Skip to content

Latest commit

 

History

History
255 lines (209 loc) · 7.05 KB

mtv-aio-buffer.adoc

File metadata and controls

255 lines (209 loc) · 7.05 KB
Raw

Increasing asynchronous I/O (AIO) sizes and buffer counts for NBD transport mode

This document describes how to change NBD transport NFC parameters for increased migration performance when using the {project-first} product.

Warning
Cold Migration Only

Using AIO buffering is only suitable for Cold Migration use cases.

Key findings

  • The best migration performance was achieved by migrating using multiple VMs (10) on a single ESXi host with the following values:

    • VixDiskLib.nfcAio.Session.BufSizeIn64KB=16

    • vixDiskLib.nfcAio.Session.BufCount=4

  • The following improvements were noted when using AIO buffer (Asynchronous Buffer Counts) settings:

    • Migration time was reduced by 31.1%, from 0:24:32 to 0:16:54.

    • Read rate was increased from 347.83 MB/s to 504.93 MB/s.

  • There was no significant improvement observed when using AIO buffer settings with a single VM.

  • There was no significant improvement observed when using AIO buffer settings with multiple VMs from multiple hosts.

Enabling AIO buffer configuration

Validating Controller Pod support for AIO values

  • Ensure that the forklift-controller pod in the openshift-mtv namespace supports the AIO buffer values.

    Since the pod name prefix is dynamic, check the pod name first by running the following command:

    oc get pods -n openshift-mtv | grep forklift-controller | awk '{print $1}'

    The example output is as follows:

    forklift-controller-667f57c8f8-qllnx
    Note

    This is the pod name prefix from the example: forklift-controller-667f57c8f8-qllnx

  • Check the environment variables of the pod by running:

    oc get pod forklift-controller-667f57c8f8-qllnx -n openshift-mtv -o yaml

  • Check for the following lines in the output:

    ...
\- name: VIRT\_V2V\_EXTRA\_ARGS
\- name: VIRT\_V2V\_EXTRA\_CONF\_CONFIG\_MAP
...
Editing ForkliftController Configuration

  • In the openshift-mtv namespace, edit the ForkliftController object to include the AIO buffer values by running the following command:

    oc edit forkliftcontroller -n openshift-mtv

    Add the following under the spec section:

    virt_v2v_extra_args: "--vddk-config /mnt/extra-v2v-conf/input.conf"
virt_v2v_extra_conf_config_map: "perf"
Creating a ConfigMap named perf

  • Create the required ConfigMap using the following command:

    oc -n openshift-mtv create cm perf
Preparing the ConfigMap content

  • Convert the desired buffer configuration values to Base64. For example, for 16/4:

    echo -e "VixDiskLib.nfcAio.Session.BufSizeIn64KB=16\nvixDiskLib.nfcAio.Session.BufCount=4" | base64

    The output will be similar to the following:

    Vml4RGlza0xpYi5uZmNBaW8uU2Vzc2lvbi5CdWZTaXplSW42NEtCPTE2CnZpeERpc2tMaWIubmZjQWlvLlNlc3Npb24uQnVmQ291bnQ9NAo=
Editing the ConfigMap

  • Update the perf ConfigMap with the Base64 string under the binaryData section, for example:

    apiVersion: v1
kind: ConfigMap
binaryData:
  input.conf: Vml4RGlza0xpYi5uZmNBaW8uU2Vzc2lvbi5CdWZTaXplSW42NEtCPTE2CnZpeERpc2tMaWIubmZjQWlvLlNlc3Npb24uQnVmQ291bnQ9NAo=
metadata:
  name: perf
  namespace: openshift-mtv
Restarting the Forklift Controller Pod

  • Restart the forklift-controller pod to apply the new configuration.

  • Ensure the VIRT_V2V_EXTRA_ARGS environment variable reflects the updated settings.

Verifying migration logs

  • Run a migration plan and check the logs of the migration pod. Confirm that the AIO buffer settings are passed as parameters, particularly the --vddk-config value.

    For example:

    exec: /usr/bin/virt-v2v … --vddk-config /mnt/extra-v2v-conf/input.conf

    Sample log excerpt:

    Buffer size calc for 16 value:
(16 * 64 * 1024 = 1048576)
nbdkit: vddk[1]: debug: [NFC VERBOSE] NfcAio_OpenSession:
Opening an AIO session.
nbdkit: vddk[1]: debug: [NFC INFO] NfcAioInitSession:
Disabling
read-ahead buffer since the AIO buffer size of 1048576 is >=
the read-ahead buffer size of 65536. Explicitly setting flag
'`NFC_AIO_SESSION_NO_NET_READ_AHEAD`'
nbdkit: vddk[1]: debug: [NFC VERBOSE] NfcAioInitSession: AIO Buffer Size is 1048576
nbdkit: vddk[1]: debug: [NFC VERBOSE] NfcAioInitSession: AIO Buffer
Count is 4
    Note

    The above logs were when using debug_level = 4
Inspecting ConfigMap values Content are in the Migration Pod

  • Log in to the migration pod and verify the buffer settings using the following command:

    cat /mnt/extra-v2v-conf/input.conf

    The example output is as follows:

    VixDiskLib.nfcAio.Session.BufSizeIn64KB=16
vixDiskLib.nfcAio.Session.BufCount=4
Enabling Debugging (optional)

  • To enable debug logs, convert the configuration to Base64, including a high log level:

    echo -e
"`VixDiskLib.nfcAio.Session.BufSizeIn64KB=16\nVixDiskLib.nfcAio.Session.BufCount=4\nVixDiskLib.nfc.LogLevel=4`"
| base64
    Note

    Adding a high log level will degrade performance and is for debugging purposes only.

Disabling AIO Buffer Configuration

To disable the AIO buffer configuration, complete the following steps:

  • Edit the ForkliftController Object: Remove the previously added lines from the spec section in the ForkliftController object:

    oc edit forkliftcontroller -n openshift-mtv

  • Remove the following lines:

    virt_v2v_extra_args: "`–vddk-config /mnt/extra-v2v-conf/input.conf`"
virt_v2v_extra_conf_config_map: "`perf`"

  • Delete the ConfigMap: Remove the perf ConfigMap that was created earlier:

    oc delete cm perf -n openshift-mtv

  • Restart the Forklift Controller Pod (Optional).

If needed, ensure the changes take effect by restarting the forklift-controller pod.

Key requirements for AIO Buffer (Asynchronous Buffer Counts) support

VDDK and vSphere Versions

Support is based upon tests performed using the following versions: