Article ID: 117816, created on Oct 9, 2013, last review on Jun 17, 2016

  • Applies to:
  • Virtuozzo 6.0

Symptoms

The hardware node and virtual environments hosted on it show different results in I/O benchmark:

  • Hardware node:

    ~# dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
        1.37909 s, 779 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync
        1.38039 s, 778 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=1M count=1k oflag=dsync
        3.03817 s, 353 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=64k count=16k oflag=dsync
        7.68641 s, 140 MB/s
    
  • Container:

    ~# dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
        2.75598 s, 390 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync
        2.77653 s, 387 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=1M count=1k oflag=dsync
        105.612 s, 10.2 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=64k count=16k oflag=dsync
        114.956 s, 9.3 MB/s
    
  • VM:

    ~# dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
        6.72346 s, 160 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync
        2.18709 s, 491 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=1M count=1k oflag=dsync
        3.69151 s, 291 MB/s
    ~# dd if=/dev/zero of=sb-io-test bs=64k count=16k oflag=dsync
        14.0475 s, 76.4 MB/s
    

Cause

  1. Reads from disk are not only using disk resources, CPU is also involved into any I\O operations. Therefore the results may be affected by the CPU settings assigned to the VEs.

  2. For containers in ploop format, it's a result of the automatic ploop image file expansion upon writing data to it. Virtual machines have a special optimization that simply drops I/O writes if the zero content like from /dev/zero, thus the results will be better for them.

  3. An overhead in disk write performance between the hardware node and virtual environments (ploop containers and virtual machines) is acceptable due to the virtualization technology nature, but the performance of VMs and CTs should be on the same level.

Resolution

  1. To compare the performance between virtual environments and the node itself - ensure your VEs have a comparable amount of resources.

  2. It is possible to tune containers to ignore I/O writes from /dev/zero. This is a boolean setting, stored in /sys/block/ploopNNNNN/ptune/check_zeros, so you can enable it the following way :

    ~# echo 1 > /sys/block/ploopNNNNN/ptune/check_zeros
    

    This setting is enabled by default since 2.6.32-042stab084.12 kernel.

    The mapping between ploop numbers and containers can be obtained from ploop list command.

    To do this for all containers on the current hardware node :

    ~# ploop list | awk '{print$1}' | while read ploop ; do echo 1 > /sys/block/$ploop/ptune/check_zeros ; done
    
  3. It is recommended to use /dev/urandom or some big file with random content to measure real performance.

Search Words

performance

/dev/null

i/o

benchmark

io

/dev/zero

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef

Email subscription for changes to this article
Save as PDF