Article ID: 127891, created on Dec 25, 2015, last review on Jun 7, 2016

  • Applies to:
  • Virtuozzo 6.0

Symptoms

Node hangs with CPU soft lockup on snapshotting of a container with namespaces enabled.
Similar lines can be found in messages log:

   [4950267.557397] BUG: soft lockup - CPU#9 stuck for 67s! [vzctl:214228]
   [4950267.557818] Pid: 214228, comm: vzctl veid: 0 Tainted: P           ---------------    2.6.32-042stab108.7 #1 042stab108_7
   [4950267.557828] RIP: 0010:[<ffffffffa0484fe3>]  [<ffffffffa0484fe3>] lookup_cpt_object+0x33/0x50 [vzcpt]
   [4950267.557841] RSP: 0018:ffff883fbec83bd8  EFLAGS: 00000206
   [4950267.557845] RAX: ffff88206fd7df40 RBX: ffff883fbec83bd8 RCX: 0000000000000000
   [4950267.557850] RDX: ffff881fa30722e8 RSI: ffff884065dff100 RDI: 0000000000000110
   [4950267.557854] RBP: ffffffff8100bc4e R08: ffff884073801138 R09: 0000000000000ffa
   [4950267.557859] R10: 0000000000000000 R11: 0000000000000000 R12: ffff883fbec83c68
   [4950267.557864] R13: ffff883fbec83ba8 R14: ffffffff811d8ca0 R15: ffff883fbec83b58
   [4950267.557869] FS:  00007fce2224a320(0000) GS:ffff882100d00000(0000) knlGS:0000000000000000
   [4950267.557874] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
   [4950267.557876] CR2: 00007f69e32103a0 CR3: 0000004072347000 CR4: 00000000000407e0
   [4950267.557878] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
   [4950267.557881] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
   [4950267.557884] Process vzctl (pid: 214228, veid: 0, threadinfo ffff883fbec82000, task ffff88405c9844c0)
   [4950267.557886] Stack:
   [4950267.557887]  ffff883fbec83cb8 ffffffffa048c3aa ffff883fbec83c38 0000000000000282
   [4950267.557890] <d> ffff883fbec83c00 ffff881f6d9dd840 ffff88406251b000 ffff881f6d9dd3c0
   [4950267.557893] <d> ffffffffffffffff 0001003800000409 6040000000001020 ffffffffffffffff
   [4950267.557897] Call Trace:
   [4950267.557905]  [<ffffffffa048c3aa>] ? dump_vfsmount+0x13a/0x9e0 [vzcpt]
   [4950267.557910]  [<ffffffffa048cd41>] ? cpt_dump_namespace+0xf1/0x200 [vzcpt]
   [4950267.557914]  [<ffffffffa0481629>] ? cpt_dump+0x649/0xa70 [vzcpt]
   [4950267.557918]  [<ffffffffa0480b5d>] ? cpt_ioctl+0x69d/0x960 [vzcpt]
   [4950267.557922]  [<ffffffffa04804c0>] ? cpt_ioctl+0x0/0x960 [vzcpt]
   [4950267.557930]  [<ffffffff81225b0e>] ? proc_reg_unlocked_ioctl+0xde/0x100
   [4950267.557934]  [<ffffffff811ca442>] ? vfs_ioctl+0x22/0xa0
   [4950267.557938]  [<ffffffff811ca926>] ? do_vfs_ioctl+0x3c6/0x5b0
   [4950267.557942]  [<ffffffff811b4052>] ? vfs_write+0x132/0x1a0
   [4950267.557944]  [<ffffffff811cab5f>] ? sys_ioctl+0x4f/0x80
   [4950267.557949]  [<ffffffff8100b122>] ? system_call_fastpath+0x16/0x1b
   [4950267.557975] Call Trace:
   [4950267.557979]  [<ffffffffa048c3aa>] ? dump_vfsmount+0x13a/0x9e0 [vzcpt]
   [4950267.557983]  [<ffffffffa048cd41>] ? cpt_dump_namespace+0xf1/0x200 [vzcpt]
   [4950267.557987]  [<ffffffffa0481629>] ? cpt_dump+0x649/0xa70 [vzcpt]
   [4950267.557991]  [<ffffffffa0480b5d>] ? cpt_ioctl+0x69d/0x960 [vzcpt]
   [4950267.557995]  [<ffffffffa04804c0>] ? cpt_ioctl+0x0/0x960 [vzcpt]
   [4950267.557999]  [<ffffffff81225b0e>] ? proc_reg_unlocked_ioctl+0xde/0x100
   [4950267.558001]  [<ffffffff811ca442>] ? vfs_ioctl+0x22/0xa0
   [4950267.558004]  [<ffffffff811ca926>] ? do_vfs_ioctl+0x3c6/0x5b0
   [4950267.558007]  [<ffffffff811b4052>] ? vfs_write+0x132/0x1a0
   [4950267.558009]  [<ffffffff811cab5f>] ? sys_ioctl+0x4f/0x80
   [4950267.558012]  [<ffffffff8100b122>] ? system_call_fastpath+0x16/0x1b

Cause

This behavior recognized as product issue PSBM-40430

Resolution

The fix is included in the kernel 2.6.32-042stab113.17 and later, as a resolution for PSBM-40430 and PSBM-42471.

Please install updates and reboot the server to the latest installed kernel.

Search Words

cpt_dump_namespace+0xf1/0x200

cpu soft lookup

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef

Email subscription for changes to this article
Save as PDF