Article ID: 123037, created on Oct 1, 2014, last review on Jun 17, 2016

  • Applies to:
  • Virtuozzo 6.0
  • Virtuozzo containers for Linux 4.7
  • Virtuozzo hypervisor

Symptoms you may observe

  1. Containers are non-responsive (cannot be pinged, accessed from the network; services inside containers do not work).
  2. High load average on the node:

    # uptime
    02:55:27 up  2:19,  0 users,  load average: 2128.67, 2065.07, 1851.27
    
  3. There are many processes in D state in "sb_start_write" WCHAN :

        # vzps axww -eLo veid,ppid,pid,tid,wchan:32,rsz,vsz,state,cmd | awk '$8~/[DR]/'
        0       1    2930    2930 sb_start_write                   34908 2569276 D /usr/sbin/prl_disp_service --mode server --pidfile /var/run/prl_disp_service.pid
     6558   31303   32073   32073 sb_start_write                    640  1824 D syslogd -m 0
     6558   31303   32133   32134 sb_start_write                   2524 76776 D /usr/bin/milter-greylist -P /var/milter-greylist/milter-greylist.pid -p /var/milter-greylist/milter-greylist.sock
     7939   34886   35522   35522 sb_start_write                    640  1824 D syslogd -m 0
     9480   36487   37132   37132 sb_start_write                    588  1832 D syslogd -m 0
     9480   37428   37435   37435 sb_start_write                    380  1768 D /usr/local/psa/admin/sbin/pipelog 7081
     9480   37438   37439   37439 sb_start_write                   1848  4996 D nginx: worker process
     9480   36487   37777   37777 sb_start_write                   1828  8304 D /usr/libexec/postfix/master
     9480   36487   38315   38315 sb_start_write                   2580 16080 D /usr/local/psa/admin/bin/modules/watchdog/monit -Ic /usr/local/psa/etc/modules/watchdog/monitrc
     9487   38553   39319   39319 sb_start_write                    588  1832 D syslogd -m 0
     9487   38553   39438   39438 sb_start_write                   12860 35360 D /usr/sbin/httpd
     9487   39438   39444   39444 sb_start_write                    380  1768 D /usr/local/psa/admin/sbin/pipelog 7081
     9487   39438   39489   39489 sb_start_write                   29920 52252 D /usr/sbin/httpd
     9487   39515   39597  507383 sb_start_write                   17548 116796 D /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --log-error=/var/log/mysqld.log --socket=/var/lib/mysql/mysql.sock
     9487   38553   39870   39870 sb_start_write                   1620  8304 D /usr/libexec/postfix/master
     9480   37428   39968   39968 sb_start_write                   22372 64924 D /usr/sbin/httpd
     9487   38553   40551   40551 sb_start_write                   2568 16048 D /usr/local/psa/admin/bin/modules/watchdog/monit -Ic /usr/local/psa/etc/modules/watchdog/monitrc
     9554   40447   40919   40919 sb_start_write                   3424 282256 D /usr/local/psa/admin/bin/modules/watchdog/monit -Ic /usr/local/psa/etc/modules/watchdog/monitrc
     9554   40447   41578   41578 sb_start_write                   2140 58264 D /usr/libexec/postfix/master
     9554   41578   41624   41624 sb_start_write                   2436 58572 D tlsmgr -l -t unix -u 
    

Cause

Under certain conditions a backup operation of a VZFS3-based container could lead to the filesystem which hosts the container's private area to be left frozen, blocking the I/O operations to the filesystem.

Resolution

The problem is recognized as product bug PSBM-29171, the fix is planned to be included in one of the future product updates.

The workaround is to reboot the node and upgrade the container's VZFS version to VZFS4. Instructions on the upgrade can be found in the 'Resolution' section of this article.

Search Words

backup

unresponsive

processes in D state

sb_start_write

vz frozen

filesystem

freeze

stuck

la

INFO: task syslogd:27966 blocked for more than 120 seconds.

a26b38f94253cdfbf1028d72cf3a498b 2897d76d56d2010f4e3a28f864d69223 e8e50b42231236b82df27684e7ec0beb d02f9caf3e11b191a38179103495106f 0dd5b9380c7d4884d77587f3eb0fa8ef 0c05f0c76fec3dd785e9feafce1099a9 c62e8726973f80975db0531f1ed5c6a2

Email subscription for changes to this article
Save as PDF