Article ID: 121866, created on Jun 2, 2014, last review on Jun 2, 2014

  • Applies to:
  • Virtuozzo 6.0

Symptoms

The node crashed after connecting a network adapter in bridged mode to a virtual machine.

The following call stack can be found in dmesg:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000588
IP: [<ffffffffa007010e>] ipt_forward_hook+0xe/0x30 [iptable_mangle]
...
Pid: 1018555, comm: prl_vm_app veid: 0 Tainted: P        W  ---------------    2.6.32-042stab088.4 #1 042stab088_4 Supermicro X9DRW-3LN4F+/X9DRW-3TF+/X9DRW-3LN4F+/X9DRW-3TF+
RIP: 0010:[<ffffffffa007010e>]  [<ffffffffa007010e>] ipt_forward_hook+0xe/0x30 [iptable_mangle]
RSP: 0018:ffff882100d838f8  EFLAGS: 00010206
RAX: 0000000000000002 RBX: ffff882100d83990 RCX: ffff88206ebaa020
RDX: 0000000000000000 RSI: ffff883c884d9080 RDI: ffff883c884d9080
RBP: ffff882100d838f8 R08: ffffffffa02671a0 R09: ffff882100d83990
R10: 000000000062efad R11: 0000000000000000 R12: 0000000080000000
R13: ffffffff81c0ecc0 R14: ffff883c884d9080 R15: 0000000000000002
FS:  00007f3c561fc700(0000) GS:ffff882100d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000588 CR3: 0000003f59bcf000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process prl_vm_app (pid: 1018555, veid: 0, threadinfo ffff88362b47c000, task ffff882c8e650500)
Stack:
ffff882100d83948 ffffffff81495c09 ffff88206ebaa020 0000000000000000
<d> ffff882100d83a18 ffff883c884d9080 0000000000000000 ffffffffa02671a0
<d> 0000000000000002 ffff883c884d9080 ffff882100d839c8 ffffffff81495dc6
Call Trace:
<IRQ> 
[<ffffffff81495c09>] nf_iterate+0x69/0xb0
[<ffffffffa02671a0>] ? br_nf_forward_finish+0x0/0x140 [bridge]
[<ffffffff81495dc6>] nf_hook_slow+0x76/0x120
[<ffffffffa02671a0>] ? br_nf_forward_finish+0x0/0x140 [bridge]
[<ffffffffa0267748>] br_nf_forward_ip+0x1f8/0x390 [bridge]
[<ffffffff81495c09>] nf_iterate+0x69/0xb0
[<ffffffffa0260cb0>] ? br_forward_finish+0x0/0x60 [bridge]
[<ffffffff81495dc6>] nf_hook_slow+0x76/0x120
[<ffffffffa0260cb0>] ? br_forward_finish+0x0/0x60 [bridge]
[<ffffffffa0260d10>] ? __br_forward+0x0/0xd0 [bridge]
[<ffffffffa0260d10>] ? __br_forward+0x0/0xd0 [bridge]
[<ffffffffa0260d8e>] __br_forward+0x7e/0xd0 [bridge]
[<ffffffff8145c3ee>] ? skb_clone+0x6e/0xd0
[<ffffffffa026093e>] deliver_clone+0x3e/0x60 [bridge]
[<ffffffff8146a8a0>] ? netif_receive_skb+0x0/0x60
[<ffffffffa0260b59>] br_flood+0x79/0xd0 [bridge]
[<ffffffffa0260bcc>] br_flood_forward+0x1c/0x20 [bridge]
[<ffffffffa02620c0>] br_handle_frame_finish+0x2d0/0x320 [bridge]
[<ffffffff8105be1c>] ? find_busiest_group+0x27c/0xa70
[<ffffffffa0267f78>] br_nf_pre_routing_finish+0x238/0x350 [bridge]
[<ffffffffa026852a>] br_nf_pre_routing+0x49a/0x7d0 [bridge]
[<ffffffff81495c09>] nf_iterate+0x69/0xb0
[<ffffffffa0261df0>] ? br_handle_frame_finish+0x0/0x320 [bridge]
[<ffffffff81495dc6>] nf_hook_slow+0x76/0x120
[<ffffffffa0261df0>] ? br_handle_frame_finish+0x0/0x320 [bridge]
[<ffffffffa026229c>] br_handle_frame+0x18c/0x260 [bridge]
[<ffffffff81466bf9>] __netif_receive_skb+0x459/0x7a0
[<ffffffff81466fda>] process_backlog+0x9a/0x100
[<ffffffff8146bfe0>] net_rx_action+0x110/0x300
[<ffffffff8107e40d>] __do_softirq+0x10d/0x250
[<ffffffff8100c44c>] call_softirq+0x1c/0x30
<EOI> 
[<ffffffff81010195>] ? do_softirq+0x65/0xa0
[<ffffffff8146c468>] netif_rx_ni+0x28/0x30
[<ffffffffa02f29cd>] hw_send+0xdd/0x1d0 [prl_netbridge]
[<ffffffffa02efe3a>] vi_send_arr+0x39a/0x710 [prl_netbridge]
[<ffffffff810a683d>] ? hrtimer_try_to_cancel+0x3d/0xd0
[<ffffffffa02f055d>] prlnet_ioctl_imp+0x10d/0x290 [prl_netbridge]
[<ffffffff811f75f4>] ? ep_scan_ready_list+0x194/0x1a0
[<ffffffffa02f0715>] prlnet_unlocked_ioctl+0x15/0x20 [prl_netbridge]
[<ffffffff811c1292>] vfs_ioctl+0x22/0xa0
[<ffffffff811c1434>] do_vfs_ioctl+0x84/0x5b0
[<ffffffff81015029>] ? read_tsc+0x9/0x20
[<ffffffff810ad461>] ? ktime_get_ts+0xb1/0xf0
[<ffffffff811c19af>] sys_ioctl+0x4f/0x80
[<ffffffff810a0b50>] ? sys_clock_gettime+0xc0/0xd0
[<ffffffff8100b102>] system_call_fastpath+0x16/0x1b

Cause

Under certain conditions a race can occur. While forwarding a packet, the input interface was removed from the bridge or not yet fully connected to the bridge which eventually caused the crash.

Resolution

The problem has been forwarded to mainstream kernel developers in the scope of the internal request #PSBM-26931.

The fix is considered to be included in one of the next PCS updates.

Search Words

ipt_forward_hook

ipt_forward_hook+0xe/0x30

BUG: unable to handle kernel NULL pointer dereference at 0000000000000588

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef

Email subscription for changes to this article
Save as PDF