PowerPath serbest kaldıktan sonra kullanım paniği

VPLEX direktör çökmesi ile VPLEX NDU

İşletim Sistemi: Red Hat Enterprise Linux 6.4
Yazılımı: Linux 6.0 için PowerPath

VPLEX NDU sırasında Red Hat Enterprise Linux sunucu paniği

Pid: 632, comm: fc_wq_2 Tainted: P        W  ---------------    2.6.32-504.23.4.el6.x86_64 #1 HP ProLiant DL380 Gen9
RIP: 0010:[<ffffffffa0164c83>]  [<ffffffffa0164c83>] sd_remove+0x43/0xc0 [sd_mod]
RSP: 0018:ffff88204d60bcc0  EFLAGS: 00010286
RAX: 006e647300000010 RBX: ffff8804d485e938 RCX: ffffffff81b0b790
RDX: ffffffff81ebc5dc RSI: ffffffff81388a90 RDI: ffff8804d485e938
RBP: ffff88204d60bce0 R08: 0000000000000000 R09: ffffffff816490c0
R10: 0000000000000000 R11: 0000000000000000 R12: ffff881a02cb8000
R13: ffff881a02cb8010 R14: ffff88204f282060 R15: ffff88204da53010
FS:  0000000000000000(0000) GS:ffff880028240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007fa01228dd50 CR3: 000000031a926000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process fc_wq_2 (pid: 632, threadinfo ffff88204d60a000, task ffff88204f0d8040)
Stack:
 ffff88204f282060 ffff8804d485e938 ffffffffa016a9c8 ffffffff81b0bca0
<d> ffff88204d60bd00 ffffffff8136c72f ffff8804d485e998 ffff8804d485e938
<d> ffff88204d60bd20 ffffffff8136c89d ffff8804d485e938 ffff880c20c39c28
Call Trace:
 [<ffffffff8136c72f>] __device_release_driver+0x6f/0xe0
 [<ffffffff8136c89d>] device_release_driver+0x2d/0x40
 [<ffffffff8136b7b3>] bus_remove_device+0xa3/0x100
 [<ffffffff813692ad>] device_del+0x12d/0x1e0
 [<ffffffff8138e245>] __scsi_remove_device+0xc5/0xd0
 [<ffffffff8138e280>] scsi_remove_device+0x30/0x50
 [<ffffffff8138e422>] scsi_remove_target+0x162/0x200
 [<ffffffffa005e570>] ? fc_starget_delete+0x0/0x30 [scsi_transport_fc]
 [<ffffffffa005e596>] fc_starget_delete+0x26/0x30 [scsi_transport_fc]
 [<ffffffff81098100>] worker_thread+0x170/0x2a0
 [<ffffffff8109ec20>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff81097f90>] ? worker_thread+0x0/0x2a0
 [<ffffffff8109e78e>] kthread+0x9e/0xc0
 [<ffffffff8100c28a>] child_rip+0xa/0x20
 [<ffffffff8109e6f0>] ? kthread+0x0/0xc0
 [<ffffffff8100c280>] ? child_rip+0x0/0x20
Code: 00 48 89 fb 48 c7 c7 80 b7 b0 81 e8 a8 28 f4 e0 48 89 df e8 b0 76 20 e1 49 89 c4 48 8b 40 08 48 c7 c6 90 8a 38 81 4d 8d 6c 24 10 <48> 8b 78 08 e8 34 13 11 e1 49 8b 44 24 08 31 f6 48 8b 78 08 e8
RIP  [<ffffffffa0164c83>] sd_remove+0x43/0xc0 [sd_mod]
 RSP <ffff88204d60bcc0>

PowerPath hatası

VPLEX NDU sırasında, VPLEX yöneticisi geri geldikten birkaç saniye sonra yanıt vermez. Bu, PowerPath'te cihaz açılışlarıyla ilgili bir hatayı ortaya çıkardı ve bu da PowerPath'in fazladan bir sürüm yayınlamasına neden oldu. Bu ek sürüm, kullanım sonrası kullanım durumuna ve Linux çekirdeği paniğine neden olur.

PowerPath 6.1 HF1 ve PowerPath'in gelecekteki sürümlerinde bu hata için bir düzeltme vardır.

Hem Red Hat hem de PowerPath Engineering, VPLEX NDU'dan bağımsız olarak laboratuvarda durumu yeniden oluşturabilir. Müşteri sırt izine ek olarak, görülen birkaç farklı panik geri izi vardı.
Ücretsiz kullanımdan sonra kullanımın doğası gereği, gerçek panik birkaç yerde görülebilir.

Müşteri geri izleme #2

Pid: 0, comm: swapper Tainted: P        W  ---------------    2.6.32-504.23.4.el6.x86_64 #1 HP ProLiant DL380 Gen9
RIP: 0010:[<ffffffff8137f2fc>]  [<ffffffff8137f2fc>] scsi_finish_command+0xac/0x130
RSP: 0018:ffff880028203e40  EFLAGS: 00010286
RAX: 0000000000010001 RBX: ffff88106dd7b580 RCX: 000000000000721b
RDX: ffff88402005c800 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff880028203e60 R08: ffff884021a8d288 R09: ffff8840269fd090
R10: ffff880028203d08 R11: 0000000000000001 R12: 0000000000004000
R13: ffff88401f46d800 R14: ffff8840219d2000 R15: 0000000000000004
FS:  0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000010079 CR3: 0000000ecc6a0000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper (pid: 0, threadinfo ffffffff81a00000, task ffffffff81a8d020)
Stack:
 ffff88106dd7b580 0000000000002002 0000000000007530 0000000000000005
<d> ffff880028203e90 ffffffff81389637 ffff880028203ea0 ffffffff81a830a0
<d> 0000000000000020 0000000000000100 ffff880028203ec0 ffffffff812787f5
Call Trace:
 <IRQ>
 [<ffffffff81389637>] scsi_softirq_done+0x147/0x170
 [<ffffffff812787f5>] blk_done_softirq+0x85/0xa0
 [<ffffffff8107d901>] __do_softirq+0xc1/0x1e0
 [<ffffffff810eac70>] ? handle_IRQ_event+0x60/0x170
 [<ffffffff8100c38c>] call_softirq+0x1c/0x30
 [<ffffffff8100fbd5>] do_softirq+0x65/0xa0
 [<ffffffff8107d7b5>] irq_exit+0x85/0x90
 [<ffffffff81533ba5>] do_IRQ+0x75/0xf0
 [<ffffffff8100ba53>] ret_from_intr+0x0/0x11
 <EOI>
 [<ffffffff812eab5e>] ? intel_idle+0xde/0x170
 [<ffffffff812eab41>] ? intel_idle+0xc1/0x170
 [<ffffffff81426167>] cpuidle_idle_call+0xa7/0x140
 [<ffffffff81009fc6>] cpu_idle+0xb6/0x110
 [<ffffffff8151067a>] rest_init+0x7a/0x80
 [<ffffffff81c29f8f>] start_kernel+0x424/0x430
 [<ffffffff81c2933a>] x86_64_start_reservations+0x125/0x129
 [<ffffffff81c29453>] x86_64_start_kernel+0x115/0x124
Code: 77 7c 48 8b 83 80 00 00 00 44 8b 63 68 83 78 44 02 74 2e 48 8b 90 b0 00 00 00 31 c0 48 85 d2 74 0a 48 8b 82 c8 02 00 00 48 8b 00 <48> 8b 40 78 48 85 c0 74 33 48 89 df ff d0 44 39 e0 74 29 41 89
RIP  [<ffffffff8137f2fc>] scsi_finish_command+0xac/0x130
 RSP <ffff880028203e40>
CR2: 0000000000010079

EMC Repro geri izleme #2

Pid: 289, comm: fc_wq_1 Tainted: P        W  ---------------    2.6.32-504.23.4.el6.x86_64 #1 Dell Inc. PowerEdge 2950/0H603H
RIP: 0010:[<ffffffff8120b2ec>]  [<ffffffff8120b2ec>] sysfs_find_dirent+0xc/0x50
RSP: 0018:ffff880225c41c10  EFLAGS: 00010286
RAX: ffff880225c40000 RBX: 0000000000000000 RCX: ffff88000cc501d8
RDX: 0000000000000000 RSI: ffffffff817e9361 RDI: 0000000000000000
RBP: ffff880225c41c20 R08: 0000000000001033 R09: ffffc90012e68000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff817e9361
R13: 0000000000000000 R14: ffff880226dfa060 R15: ffff880225c15010
FS:  0000000000000000(0000) GS:ffff880028300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000068 CR3: 0000000229e15000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process fc_wq_1 (pid: 289, threadinfo ffff880225c40000, task ffff880226dcd520)
Stack:
 0000000000000000 ffffffff817e9361 ffff880225c41c40 ffffffff8120c3d3
<d> ffffffff81b09ce0 ffff8802250fe420 ffff880225c41c70 ffffffff8120dd00
<d> ffff8802250fe410 ffff8802250fe410 ffff88012185c138 ffff8802250fe410
Call Trace:
 [<ffffffff8120c3d3>] sysfs_get_dirent+0x33/0x80
 [<ffffffff8120dd00>] sysfs_remove_group+0x30/0x100
 [<ffffffff81371559>] dpm_sysfs_remove+0x19/0x20
 [<ffffffff813691c9>] device_del+0x49/0x1e0
 [<ffffffffa0276ca4>] sd_remove+0x64/0xc0 [sd_mod]
 [<ffffffff8136c72f>] __device_release_driver+0x6f/0xe0
 [<ffffffff8136c89d>] device_release_driver+0x2d/0x40
 [<ffffffff8136b7b3>] bus_remove_device+0xa3/0x100
 [<ffffffff813692ad>] device_del+0x12d/0x1e0
 [<ffffffff8138e245>] __scsi_remove_device+0xc5/0xd0
 [<ffffffff8138e280>] scsi_remove_device+0x30/0x50
 [<ffffffff8138e422>] scsi_remove_target+0x162/0x200
 [<ffffffffa026b570>] ? fc_starget_delete+0x0/0x30 [scsi_transport_fc]
 [<ffffffffa026b596>] fc_starget_delete+0x26/0x30 [scsi_transport_fc]
 [<ffffffff81098100>] worker_thread+0x170/0x2a0
 [<ffffffff8109ec20>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff81097f90>] ? worker_thread+0x0/0x2a0
 [<ffffffff8109e78e>] kthread+0x9e/0xc0
 [<ffffffff8100c28a>] child_rip+0xa/0x20
 [<ffffffff8109e6f0>] ? kthread+0x0/0xc0
 [<ffffffff8100c280>] ? child_rip+0x0/0x20
Code: c6 48 c7 c7 c0 08 7d 81 31 c0 e8 14 e1 31 00 0f 0b eb fe 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 53 0f 1f 44 00 00 <48> 8b 47 68 49 89 f4 eb 1e 0f 1f 00 48 8b 70 e0 4c 89 e7 48 8d
RIP  [<ffffffff8120b2ec>] sysfs_find_dirent+0xc/0x50
 RSP <ffff880225c41c10>
CR2: 0000000000000068

EMC yeniden üretim geri izleme #4

Pid: 290, comm: fc_wq_1 Tainted: P        W  ---------------    2.6.32-504.23.4.el6.x86_64 #1 Dell Inc. PowerEdge 2950/0H603H
RIP: 0010:[<ffffffff8137eb79>]  [<ffffffff8137eb79>] scsi_device_get+0x19/0xe0
RSP: 0018:ffff880226c93aa0  EFLAGS: 00010286
RAX: ffff880226c92000 RBX: ffff880225b45400 RCX: 000000000000001e
RDX: 0000000000000000 RSI: 0000000000000002 RDI: 000000a300000001
RBP: ffff880226c93ac0 R08: fffffffcfffffffc R09: ffffffb0ffffffb0
R10: fffffffcfffffffc R11: fffffffcfffffffc R12: ffff88022762e958
R13: 0000000000000002 R14: ffff88022762e808 R15: 00000000fffffffa
FS:  0000000000000000(0000) GS:ffff880028240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 000000a3000005a1 CR3: 00000002097bb000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process fc_wq_1 (pid: 290, threadinfo ffff880226c92000, task ffff880227443520)
Stack:
 ffff8800138c7210 ffff880225b45400 ffff88022762e958 0000000000000002
<d> ffff880226c93ae0 ffffffffa0276449 ffff880223dab400 ffff880223dab400
<d> ffff880226c93b20 ffffffffa02777a8 ffff880226c93b00 ffff88022762ea30
Call Trace:
 [<ffffffffa0276449>] __scsi_disk_get+0x39/0x50 [sd_mod]
 [<ffffffffa02777a8>] sd_open+0x48/0x1d0 [sd_mod]
 [<ffffffffa079fa96>] emcp_sd_bdops_open+0x26/0x60 [emcp]
 [<ffffffffa07bb9cc>] emcp_remove+0x3ec/0x8a0 [emcp]
 [<ffffffffa07bbef5>] emcp_chg_device_notify+0x75/0x90 [emcp]
 [<ffffffff815300d5>] notifier_call_chain+0x55/0x80
 [<ffffffff810a529a>] __blocking_notifier_call_chain+0x5a/0x80
 [<ffffffff810a52d6>] blocking_notifier_call_chain+0x16/0x20
 [<ffffffff8136c718>] __device_release_driver+0x58/0xe0
 [<ffffffff8136c89d>] device_release_driver+0x2d/0x40
 [<ffffffff8136b7b3>] bus_remove_device+0xa3/0x100
 [<ffffffff813692ad>] device_del+0x12d/0x1e0
 [<ffffffff8138e245>] __scsi_remove_device+0xc5/0xd0
 [<ffffffff8138e280>] scsi_remove_device+0x30/0x50
 [<ffffffff8138e422>] scsi_remove_target+0x162/0x200
 [<ffffffffa026b570>] ? fc_starget_delete+0x0/0x30 [scsi_transport_fc]
 [<ffffffffa026b596>] fc_starget_delete+0x26/0x30 [scsi_transport_fc]
 [<ffffffff81098100>] worker_thread+0x170/0x2a0
 [<ffffffff8109ec20>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff81097f90>] ? worker_thread+0x0/0x2a0
 [<ffffffff8109e78e>] kthread+0x9e/0xc0
 [<ffffffff8100c28a>] child_rip+0xa/0x20
 [<ffffffff8109e6f0>] ? kthread+0x0/0xc0
 [<ffffffff8100c280>] ? child_rip+0x0/0x20
Code: c3 90 48 89 df e8 b8 e0 d3 ff eb dd 66 0f 1f 44 00 00 55 48 89 e5 48 83 ec 20 48 89 5d e8 4c 89 65 f0 4c 89 6d f8 0f 1f 44 00 00 <83> bf a0 05 00 00 04 48 89 fb 75 1b b8 fa ff ff ff 48 8b 5d e8
RIP  [<ffffffff8137eb79>] scsi_device_get+0x19/0xe0
 RSP <ffff880226c93aa0>
CR2: 000000a3000005a1

EMC Repro geri izleme #5

[49952.667188] Pid: 289, comm: fc_wq_1 Tainted: P           ---------------    2.6.32-504.23.4.el6.x86_64 #1 Dell Inc. PowerEdge 2950/0H603H
[49952.667188] RIP: 0010:[<ffffffff81293be9>]  [<ffffffff81293be9>] strnlen+0x9/0x40
[49952.667188] RSP: 0018:ffff88022630d930  EFLAGS: 00010086
[49952.667188] RAX: ffffffff817bcc11 RBX: ffffffff81eb9c00 RCX: 0000000000000002
[49952.667188] RDX: fffffffcfffffffc RSI: ffffffffffffffff RDI: fffffffcfffffffc
[49952.667188] RBP: ffff88022630d930 R08: 0000000000000073 R09: 00000000fffffff8
[49952.667188] R10: 0000000000000006 R11: 00000000000000c8 R12: ffffffff81eb9857
[49952.667188] R13: fffffffcfffffffc R14: 00000000ffffffff R15: 0000000000000000
[49952.667188] FS:  0000000000000000(0000) GS:ffff8800282c0000(0000) knlGS:0000000000000000
[49952.667188] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[49952.667188] CR2: fffffffcfffffffc CR3: 000000022a50e000 CR4: 00000000000007e0
[49952.667188] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[49952.667188] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[49952.667188] Process fc_wq_1 (pid: 289, threadinfo ffff88022630c000, task ffff8802276c9520)
[49952.667188] Stack:
[49952.667188]  ffff88022630d970 ffffffff81294ed0 0000000000000000 ffffffff81eb9857
[49952.667188] <d> ffffffffa084f498 ffffffffa084f496 ffff88022630dae0 ffffffff81eb9c00
[49952.667188] <d> ffff88022630da10 ffffffff81296908 0000000000000004 0000000affffffff
[49952.667188] Call Trace:
[49952.667188]  [<ffffffff81294ed0>] string+0x40/0x100
[49952.667188]  [<ffffffff81296908>] vsnprintf+0x218/0x5e0
[49952.667188]  [<ffffffff812a249e>] ? percpu_counter_destroy+0x3e/0x50
[49952.667188]  [<ffffffff81296d91>] vscnprintf+0x11/0x30
[49952.667188]  [<ffffffff81075c16>] vprintk+0xc6/0x560
[49952.667188]  [<ffffffff8128e3bd>] ? kobject_release+0x8d/0x240
[49952.667188]  [<ffffffff8128e330>] ? kobject_release+0x0/0x240
[49952.667188]  [<ffffffff8128f8a7>] ? kref_put+0x37/0x70
[49952.667188]  [<ffffffff81529425>] printk+0x41/0x44
[49952.667188]  [<ffffffffa0838d41>] ? PowerGetSemaNew+0x51/0x210 [emcp]
[49952.667188]  [<ffffffffa084b24d>] emcp_remove+0x78d/0x8d0 [emcp]
[49952.667188]  [<ffffffffa084b405>] emcp_chg_device_notify+0x75/0x90 [emcp]
[49952.667188]  [<ffffffff815300d5>] notifier_call_chain+0x55/0x80
[49952.667188]  [<ffffffff810a529a>] __blocking_notifier_call_chain+0x5a/0x80
[49952.667188]  [<ffffffff810a52d6>] blocking_notifier_call_chain+0x16/0x20
[49952.667188]  [<ffffffff8136c718>] __device_release_driver+0x58/0xe0
[49952.667188]  [<ffffffff8136c89d>] device_release_driver+0x2d/0x40
[49952.667188]  [<ffffffff8136b7b3>] bus_remove_device+0xa3/0x100
[49952.667188]  [<ffffffff813692ad>] device_del+0x12d/0x1e0
[49953.026077]  rport-2:0-4: blocked FC remote port time out: removing rport
[49952.667188]  [<ffffffff8138e245>] __scsi_remove_device+0xc5/0xd0
[49952.667188]  [<ffffffff8138e280>] scsi_remove_device+0x30/0x50
[49952.667188]  [<ffffffff8138e422>] scsi_remove_target+0x162/0x200
[49952.667188]  [<ffffffffa026b570>] ? fc_starget_delete+0x0/0x30 [scsi_transport_fc]
[49952.667188]  [<ffffffffa026b596>] fc_starget_delete+0x26/0x30 [scsi_transport_fc]
[49952.667188]  [<ffffffff81098100>] worker_thread+0x170/0x2a0
[49952.667188]  [<ffffffff8109ec20>] ? autoremove_wake_function+0x0/0x40
[49952.667188]  [<ffffffff81097f90>] ? worker_thread+0x0/0x2a0
[49952.667188]  [<ffffffff8109e78e>] kthread+0x9e/0xc0
[49952.667188]  [<ffffffff8100c28a>] child_rip+0xa/0x20
[49952.667188]  [<ffffffff8109e6f0>] ? kthread+0x0/0xc0
[49952.667188]  [<ffffffff8100c280>] ? child_rip+0x0/0x20
[49952.667188] Code: 66 90 48 83 c2 01 80 3a 00 75 f7 48 89 d0 48 29 f8 c9 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 85 f6 48 89 e5 74 2e <80> 3f 00 74 29 48 83 ee 01 48 89 f8 eb 12 66 0f 1f 84 00 00 00
[49952.667188] RIP  [<ffffffff81293be9>] strnlen+0x9/0x40
[49952.667188]  RSP <ffff88022630d930>
[49952.667188] CR2: fffffffcfffffffc

EMC Repro geri izleme #6

[ 1264.376313] Pid: 289, comm: fc_wq_1 Tainted: P        W  ---------------    2.6.32-504.23.4.el6.x86_64 #1 Dell Inc. PowerEdge 2950/0H603H
[ 1264.376313] RIP: 0010:[<ffffffff81372ae2>]  [<ffffffff81372ae2>] device_pm_remove+0x42/0xb0
[ 1264.376313] RSP: 0018:ffff88022635bc70  EFLAGS: 00010286
[ 1264.376313] RAX: ffff88022419c0b0 RBX: ffff88022419c010 RCX: 0000000000000000
[ 1264.376313] RDX: ffff880223e39cc8 RSI: 0000000000000000 RDI: ffffffff81b09d80
[ 1264.376313] RBP: ffff88022635bc80 R08: 000000000009d39c R09: 0000000000000000
[ 1264.376313] R10: 0000000000000000 R11: 00000000000000be R12: ffff88022409fee0
[ 1264.376313] R13: ffff88022419c010 R14: ffff880227762860 R15: ffff880226336010
[ 1264.376313] FS:  0000000000000000(0000) GS:ffff880028340000(0000) knlGS:0000000000000000
[ 1264.376313] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[ 1264.376313] CR2: 0000000000000008 CR3: 0000000001a85000 CR4: 00000000000007e0
[ 1264.376313] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1264.376313] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 1264.376313] Process fc_wq_1 (pid: 289, threadinfo ffff88022635a000, task ffff880226359520)
[ 1264.376313] Stack:
[ 1264.376313]  ffff8802091fc138 ffff88022419c010 ffff88022635bcb0 ffffffff813691c1
[ 1264.376313] <d> ffffffff81b0bca0 ffff8802091fc138 ffff88022419c000 ffff88022419c010
[ 1264.376313] <d> ffff88022635bce0 ffffffffa0276ca4 ffff880227762860 ffff8802091fc138
[ 1264.376313] Call Trace:
[ 1264.376313]  [<ffffffff813691c1>] device_del+0x41/0x1e0
[ 1264.376313]  [<ffffffffa0276ca4>] sd_remove+0x64/0xc0 [sd_mod]
[ 1264.376313]  [<ffffffff8136c72f>] __device_release_driver+0x6f/0xe0
[ 1264.376313]  [<ffffffff8136c89d>] device_release_driver+0x2d/0x40
[ 1264.376313]  [<ffffffff8136b7b3>] bus_remove_device+0xa3/0x100
[ 1264.376313]  [<ffffffff813692ad>] device_del+0x12d/0x1e0
[ 1264.376313]  [<ffffffff8138e245>] __scsi_remove_device+0xc5/0xd0
[ 1264.376313]  [<ffffffff8138e280>] scsi_remove_device+0x30/0x50
[ 1264.376313]  [<ffffffff8138e422>] scsi_remove_target+0x162/0x200
[ 1264.376313]  [<ffffffffa026b570>] ? fc_starget_delete+0x0/0x30 [scsi_transport_fc]
[ 1264.376313]  [<ffffffffa026b596>] fc_starget_delete+0x26/0x30 [scsi_transport_fc]
[ 1264.376313]  [<ffffffff81098100>] worker_thread+0x170/0x2a0
[ 1264.376313]  [<ffffffff8109ec20>] ? autoremove_wake_function+0x0/0x40
[ 1264.376313]  [<ffffffff81097f90>] ? worker_thread+0x0/0x2a0
[ 1264.693138]  rport-2:0-12: blocked FC remote port time out: removing rport
[ 1264.376313]  [<ffffffff8109e78e>] kthread+0x9e/0xc0
[ 1264.376313]  [<ffffffff8100c28a>] child_rip+0xa/0x20
[ 1264.376313]  [<ffffffff8109e6f0>] ? kthread+0x0/0xc0
[ 1264.376313]  [<ffffffff8100c280>] ? child_rip+0x0/0x20
[ 1264.376313] Code: fb 75 51 48 c7 c7 80 9d b0 81 e8 da 83 1b 00 48 8b 8b a0 00 00 00 48 8b 93 a8 00 00 00 48 8d 83 a0 00 00 00 48 c7 c7 80 9d b0 81 <48> 89 51 08 48 89 0a 48 89 83 a0 00 00 00 48 89 83 a8 00 00 00
[ 1264.376313] RIP  [<ffffffff81372ae2>] device_pm_remove+0x42/0xb0
[ 1264.376313]  RSP <ffff88022635bc70>
[ 1264.376313] CR2: 0000000000000008