Article ID: 128359, created on Feb 17, 2016, last review on Feb 17, 2016

  • Applies to:
  • Virtuozzo 6.0
  • Virtuozzo containers for Linux 4.7

Information

The hardware node has high LA values for short or long period of times, high memory usage, etc. Is there any information logged for such events?

Resolution

The first thing to check is the generic configuration of the server:

  1. Performance degraded if NUMA is disabled.
  2. Memory limits in Parallels Virtuozzo Containers for Linux
  3. Verify if the SSD disk is healthy if containers are on Pstorage with write journals
  4. Check and monitor swapping (vmstat 5), whether the server experience any memory shortage
  5. Ensure there is no 100% CPU usage in top for wa% and/or si% (hit '1' in top to get per-CPU usage)

Looking into saved log files

To find out if there were events with the high load on the node registered, one can check the files in the directory "/var/log/vzstat/", for example - LA over 200:

~# awk '$19>200' /var/log/vzstat/vzstatmon.log
02/04/14 00:03:31: 13 4 0 10688 713600 12234032 70610792 0 4 257 3295 65254 27734 16 15 59 248.27 105.39 56.55 13/13078 19900 -8034 496
02/04/14 00:11:47: 5 2 0 12668 2870980 12148808 69825968 1 4 38 3136 59720 24955 14 13 59 105.67 149.81 112.07 6/12872 15272 27840 496
02/04/14 00:20:03: 3 4 0 14488 522772 12092548 69219536 0 4 29 3160 59683 24872 12 13 60 416.70 280.89 173.99 5/13399 8405 25601 496

The numbers are written from output of vmstat and cat /proc/loadavg commands. Fields meaning:

  1. 1-2: time stamp: data, time;
  2. 3-5: number of processes: running, blocked, waiting;
  3. 6-9: memory usage: used swap, free memory, used buffers, used caches;
  4. 10-11: swap in/out activity: swapped in blocks, swapped out blocks;
  5. 12-13: disk read/write activity: read blocks, written blocks;
  6. 14-15: system actiwity: number of interrupts served, process context switches;
  7. 16-18: CPU usage: user time, system time, idle time;
  8. 19-23: system load average: 1 min, 5 min, 15 min, amount of running/sleeping processes, pid of cat;
  9. 24-25: differential data since previous run: number of forks (new process creation), seconds passed.

Basing on these values, history of the system load can be obtained. For the case with high number of blocked/running processes, or for high LA values, the file "/var/log/vzstat/vzstatmon.full.log" collects the list of R//D state processes, so the load can be analyzed further.

However, the output is not useful that much, and in the most cases it is better to check additionally with this command:

~# vzps axww -eLo veid,ppid,pid,tid,wchan:32,rsz,vsz,state,cmd | awk '$8~/[DR]/'
  1. refrigerator: should it be seen, this is High load average during backup-related operations;
  2. sb_start_write or similar may point to R1Soft/Idera incompatibility, try to disable it.

Search Words

vzstat

pcs memory usage

high la

hv suddently high load

shaman crash

parallels high load

vzstatmon

swap problem

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef d02f9caf3e11b191a38179103495106f e8e50b42231236b82df27684e7ec0beb 0c05f0c76fec3dd785e9feafce1099a9

Email subscription for changes to this article
Save as PDF