Article ID: 127391, created on Nov 11, 2015, last review on Jul 22, 2016

  • Applies to:
  • Virtuozzo 6.0

Symptoms

A destination hardware node crashes during online container migration. Similar messages can be seen in dmesg:

[597391.171816] BUG: unable to handle kernel paging request at ffff8820b1021001
[597391.171952] IP: [<ffffffffa05b038d>] rst_skb+0x48d/0x7b0 [vzrst]
[597391.172033] PGD 1a86063 PUD 0 
[597391.172104] Oops: 0000 [#1] SMP 
[597391.172182] last sysfs file: /sys/module/vzcptpram/initstate
[597391.172254] CPU 0 
[597391.172262] Modules linked in: vzcptpram(P)(U) mpt3sas mpt2sas scsi_transport_sas raid_class mptctl mptbase dell_rbu cls_fw simfs sch_cbq prl_usb_connect(U) prl_netbridge(U) prl_vtdhook(P)(U) prl_hypervisor(P)(U) ip_vzredir(P)(U) vzredir(P)(U) vzcompat(P)(U) vzrst vznetdev ip6_vzprivnet(P)(U) ip6_vzredir(P)(U) ip6_vznetstat(P)(U) pio_kaio pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop ip_vzprivnet(P)(U) vziolimit vzsnap(P)(U) vzfs(P)(U) vzdquota vzcpt vzlinkdev(P)(U) vzethdev vzevent vzlist(P)(U) vzstat(P)(U) vzmon ip_vznetstat(P)(U) vznetstat(P)(U) vzdev ip_set_hash_netport ip_set_hash_net ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ip ip_set nfnetlink xt_string xt_owner ipt_addrtype xt_comment ipt_MASQUERADE xt_recent ipt_REDIRECT nf_nat_irc xt_DSCP nf_conntrack_irc xt_conntrack ipt_LOG nf_nat_ftp nf_conntrack_ftp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_helper xt_state xt_length xt_hl xt_tcpmss xt_TCPMSS iptable_mangle xt_multiport xt_limit xt_dscp ebt_among ebtable_filter ebtables bridge bonding 8021q garp stp llc ipt_REJECT iptable_filter ip_tables nf_conntrack_ipv6 nf_conntrack nf_defrag_ipv6 ip6t_LOG ip6t_REJECT ip6table_mangle ipv6 ip6table_filter ip6_tables ipmi_devintf tun fuse nfs lockd fscache auth_rpcgss nfs_acl sunrpc power_meter acpi_ipmi ipmi_si ipmi_msghandler microcode iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core i7core_edac edac_core igb dca i2c_algo_bit i2c_core ptp pps_core ses enclosure sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif wmi megaraid_sas dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
[597391.174175] 
[597391.174239] Pid: 517925, comm: init veid: 101 Tainted: P           ---------------    2.6.32-042stab108.8 #1 042stab108_8 Dell Inc. PowerEdge R510/0DPRKF
[597391.174384] RIP: 0010:[<ffffffffa05b038d>]  [<ffffffffa05b038d>] rst_skb+0x48d/0x7b0 [vzrst]
[597391.174523] RSP: 0018:ffff88102600fbf0  EFLAGS: 00010246
[597391.174596] RAX: 0000000000000002 RBX: ffff881c1c09c580 RCX: 000000007a086800
[597391.174732] RDX: ffff882036f9a800 RSI: ffff8819565327b8 RDI: ffff882036071800
[597391.174867] RBP: ffff88102600fcf0 R08: 0000000000000003 R09: ffffea0037bba888
[597391.175001] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000439a10
[597391.175138] R13: ffff881ffdc55000 R14: 0000000000000004 R15: 0000000000000000
[597391.175266] FS:  00007f1c61dd3320(0000) GS:ffff881080800000(0000) knlGS:0000000000000000
[597391.175395] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[597391.175467] CR2: ffff8820b1021001 CR3: 0000000fbd967000 CR4: 00000000000007f0
[597391.175593] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[597391.175724] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[597391.175852] Process init (pid: 517925, veid: 101, threadinfo ffff88102600e000, task ffff881037661000)
[597391.175981] Stack:
[597391.176042]  0000000000000010 00000000004398a0 ffff881c1c09c5b8 00000004a05a0c75
[597391.176121] <d> ffff88102600fd4c ffff88147531a780 0000000000000170 0001007800000406
[597391.176266] <d> 00000002000000b0 0000000000000000 000000e000000000 7a08680000000020
[597391.176468] Call Trace:
[597391.176542]  [<ffffffffa05b2ef5>] rst_socket_in+0x2c5/0x11b0 [vzrst]
[597391.176624]  [<ffffffff815399eb>] ? _spin_unlock_bh+0x1b/0x20
[597391.176701]  [<ffffffffa05b0ff5>] open_socket+0x495/0x7c0 [vzrst]
[597391.176783]  [<ffffffffa05b1f96>] rst_sockets+0x516/0x9c0 [vzrst]
[597391.176862]  [<ffffffffa059f9c2>] hook+0x11d2/0x1b10 [vzrst]
[597391.176933]  [<ffffffff810795ae>] ? __mmdrop+0x4e/0x80
[597391.177001]  [<ffffffff810703b8>] ? finish_task_switch+0xc8/0x120
[597391.177078]  [<ffffffffa059e7f0>] ? hook+0x0/0x1b10 [vzrst]
[597391.177151]  [<ffffffffa059c7ff>] child_rip+0xa/0x1b [vzrst]
[597391.177221]  [<ffffffffa059e7f0>] ? hook+0x0/0x1b10 [vzrst]
[597391.177291]  [<ffffffffa059e7f0>] ? hook+0x0/0x1b10 [vzrst]
[597391.177362]  [<ffffffffa059c7f5>] ? child_rip+0x0/0x1b [vzrst]
[597391.177428] Code: 08 48 89 4b 40 c7 85 1c ff ff ff 00 00 00 00 e9 f4 fc ff ff 66 2e 0f 1f 84 00 00 00 00 00 8b 8b c8 00 00 00 48 8b 93 d8 00 00 00 <44> 0f b6 74 0a 01 8b 8b c4 00 00 00 4c 8d 63 38 83 f8 03 45 0f 
[597391.177778] RIP  [<ffffffffa05b038d>] rst_skb+0x48d/0x7b0 [vzrst]
[597391.177853]  RSP <ffff88102600fbf0>
[597391.177915] CR2: ffff8820b1021001

Cause

The problem is recognized as a product bug PSBM-39332.

Resolution

The bug is fixed in Virtuozzo 6 update 11 hotfix 8. To avoid the issue in the future, update the destination server and reboot into the latest kernel.

Search Words

rst_skb

unable to handle kernel paging request

kernel panic

migration

rst_socket_in

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef

Email subscription for changes to this article
Save as PDF