| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| recovery-small test complete, duration 6562 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 106690 Comm: kworker/2:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1010750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 8a 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa1e501de7da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff964851f27b40 RCX: 00000006d1891ef7 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000c9c R09: 0000000000000000 R10: 000000029e711ce4 R11: ffff964844f1f000 R12: ffff964982138a00 R13: 00000000000000ba R14: ffffffff8a6587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff964982100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000a2016004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 00:55:04 (1779425704) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 00:57:16 (1779425836) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 11 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 7 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 25270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779425848 with bad export cookie 14268585464443287563 LustreError: 25270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages LustreError: MGC192.168.204.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: Skipped 8 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg452-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 308 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 126 Comm: kworker/3:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1271750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 b0 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffac6541463da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffffa009113b4f00 RCX: 00000000065a2e36 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000b23 R09: 0000000000000000 R10: 0000000002094f1a R11: ffffa0080210e000 R12: ffffa009421b8a00 R13: 00000000000000ba R14: ffffffffb0c587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffffa00942180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000005ec16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:17:46 (1779419866) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:17:48 (1779419868) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6556:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6556:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419880 with bad export cookie 5665217824841682982 LustreError: 6538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3696:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419881/real 1779419881] req@ffffa00801aa0000 x1865856552566016/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419897 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 326 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 714 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0ff7750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 8e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb29b017a7da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff98ef749baa40 RCX: 0000000006a5d870 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000dbb R09: 0000000000000000 R10: 00000000022a6d79 R11: ffff98ef7f65f000 R12: ffff98ef821b8a00 R13: 00000000000000ba R14: ffffffff8e6587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff98ef82180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000b7616006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:17:33 (1779419853) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:17:34 (1779419854) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-MDT0000 complete LustreError: 7511:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7511:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 17754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419865 with bad export cookie 14139251232081106264 LustreError: 17754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 314 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000043 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 59 Comm: kworker/1:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc10a8750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 bb 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb4b000f6fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff96f4435f86c0 RCX: 00000000065e8cd5 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000013 RBP: 0000000000000013 R08: 0000000000000d07 R09: 0000000000000000 R10: 00000000020b48ff R11: ffff96f55e7cc000 R12: ffff96f5820b8a00 R13: 00000000000000cb R14: ffffffffbb4587d4 R15: 0000000000000013 FS: 0000000000000000(0000) GS:ffff96f582080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000043 CR3: 000000006ac16002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000043 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:17:30 (1779419850) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:17:32 (1779419852) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419862 with bad export cookie 10359729752317001120 LustreError: 6538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 313 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000040 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 41 Comm: kworker/2:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1025750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 a5 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa26780dd3da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff96ab07e6dc00 RCX: 0000000006595599 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000010 RBP: 0000000000000010 R08: 00000000000005b5 R09: 0000000000000000 R10: 0000000002097d15 R11: ffff96ac0aed9000 R12: ffff96ac42138a00 R13: 00000000000000c8 R14: ffffffffa58587d4 R15: 0000000000000010 FS: 0000000000000000(0000) GS:ffff96ac42100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000040 CR3: 0000000110416005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000040 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:17:24 (1779419844) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:17:26 (1779419846) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? Lustre: server umount lustre-MDT0000 complete LustreError: 9533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419867 with bad export cookie 11214740461074828700 LustreError: 9533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3701:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419867/real 1779419867] req@ffff96ac09b64000 x1865856528445568/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419883 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 153 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000039 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 54 Comm: kworker/0:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 af 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9d6080e3bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff92194371e200 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff921a5f45a42c RDI: 0000000000000009 RBP: 0000000000000009 R08: 0000000000000000 R09: 0000000000000000 R10: ffff921a44e4f940 R11: 0000000000000007 R12: ffff921a82038a00 R13: 00000000000000c1 R14: ffffffffaf6587d4 R15: 0000000000000009 FS: 0000000000000000(0000) GS:ffff921a82000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000039 CR3: 000000001fa16002 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000039 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:14:40 (1779419680) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:14:41 (1779419681) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 6 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 12719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419686 with bad export cookie 3820038393728733913 LustreError: MGC192.168.203.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 12719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg359-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 387 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000042 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 764 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 a1 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbb6d817d3da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff98d944f2a080 RCX: 0000000007c8f985 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000012 RBP: 0000000000000012 R08: 0000000000000ead R09: 0000000000000000 R10: 00000000024334b4 R11: ffff98d848780000 R12: ffff98d9821b8a00 R13: 00000000000000ca R14: ffffffffa1c587d4 R15: 0000000000000012 FS: 0000000000000000(0000) GS:ffff98d982180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000042 CR3: 000000010d216003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000042 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:14:34 (1779419674) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:14:36 (1779419676) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419685 with bad export cookie 8842459999106472469 LustreError: 6455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: MGC192.168.201.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg149-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 382 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 137 Comm: kworker/2:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc17ac750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffacfd4150fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff91eb47bb8700 RCX: 0000000007f8acc1 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000062 R09: 0000000000000000 R10: 00000000025724a8 R11: ffff91ea45096000 R12: ffff91eb82138a00 R13: 00000000000000c0 R14: ffffffff9b0587d4 R15: 0000000000000008 FS: 0000000000000000(0000) GS:ffff91eb82100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000038 CR3: 0000000049616002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000038 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:42 (1779419622) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:43 (1779419623) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419632 with bad export cookie 18033059362918451927 LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 379 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000048 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 14240 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0ef4750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 aa 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb5434788bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff932e7a952c00 RCX: 0000000006ece9fd RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000018 RBP: 0000000000000018 R08: 000000000000034e R09: 0000000000000000 R10: 0000000002449eeb R11: ffff932d43cdc000 R12: ffff932e82038a00 R13: 00000000000000d0 R14: ffffffffaa4587d4 R15: 0000000000000018 FS: 0000000000000000(0000) GS:ffff932e82000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000048 CR3: 0000000030616002 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000048 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:14 (1779419594) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:15 (1779419595) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6592:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6592:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6575:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419605 with bad export cookie 16375692614540683600 LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 428 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 10052 Comm: kworker/0:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc114e750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 8f 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbc620193bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffffa088c4422a40 RCX: 000000000712858e RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000004bb R09: 0000000000000000 R10: 0000000002552d01 R11: ffffa088d0aaf000 R12: ffffa08902038a00 R13: 00000000000000ba R14: ffffffff8f8587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffffa08902000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000004f016004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:13 (1779419593) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:14 (1779419594) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 6609:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6609:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 4 previous similar messages Lustre: Skipped 2 previous similar messages LustreError: 6591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419600 with bad export cookie 14975376926122361037 LustreError: 6591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages LustreError: MGC192.168.204.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg404-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 375 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000042 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 10242 Comm: kworker/3:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0ea2750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 a1 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb4bc8730fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff96900075f740 RCX: 0000000006e793e8 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000012 RBP: 0000000000000012 R08: 00000000000000ef R09: 0000000000000000 R10: 000000000243c3ea R11: ffff968ec3537000 R12: ffff9690021b8a00 R13: 00000000000000ca R14: ffffffffa10587d4 R15: 0000000000000012 FS: 0000000000000000(0000) GS:ffff969002180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000042 CR3: 000000003b616002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000042 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:14 (1779419594) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:15 (1779419595) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419602 with bad export cookie 9764614854308611092 LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 405 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000036 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 20704 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1ad9750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 8c 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffae7242dc3da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff93e243e7d3c0 RCX: 0000000007d4c073 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000006 RBP: 0000000000000006 R08: 000000000000082d R09: 0000000000000000 R10: 000000000247f6b9 R11: ffff93e34837d000 R12: ffff93e382038a00 R13: 00000000000000be R14: ffffffff8c2587d4 R15: 0000000000000006 FS: 0000000000000000(0000) GS:ffff93e382000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000036 CR3: 000000003c816004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000036 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:16 (1779419596) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:17 (1779419597) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 13596:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419605 with bad export cookie 17931212880905623467 LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 13596:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 430 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 41 Comm: kworker/2:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0fcb750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 9e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb73c80dd3da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9b4a58bbc480 RCX: 0000000007036a57 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000e90 R09: 0000000000000000 R10: 00000000024e725b R11: ffff9b4a7af91000 R12: ffff9b4a82138a00 R13: 00000000000000ba R14: ffffffff9e0587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9b4a82100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000134016001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:52 (1779419572) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:54 (1779419574) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 1 previous similar message LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 20939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419584 with bad export cookie 1656654847173005901 LustreError: MGC192.168.204.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 20939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 188 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000033 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 489 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc19ec750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 85 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffacb6815efda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff942f85053e80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff942faeeac42c RDI: 0000000000000003 RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000 R10: ffff942f9f8ecda0 R11: 0000000000000007 R12: ffff942fc21b8a00 R13: 00000000000000bb R14: ffffffff858587d4 R15: 0000000000000003 FS: 0000000000000000(0000) GS:ffff942fc2180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000033 CR3: 000000002fc16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000033 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:13:05 (1779419585) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:13:06 (1779419586) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 1 previous similar message LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: server umount lustre-MDT0000 complete LustreError: 6446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419591 with bad export cookie 2664696113297837865 LustreError: MGC192.168.203.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg314-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 415 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 11573 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1886750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 a5 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9dbf046d3da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff8dc16f2c7780 RCX: 000000000825f684 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000001cb R09: 0000000000000000 R10: 000000000269df2f R11: ffff8dc045ff6000 R12: ffff8dc182038a00 R13: 00000000000000ba R14: ffffffffa5a587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8dc182000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000001d416005 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:00 (1779419580) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:01 (1779419581) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 12091:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 12091:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 7214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419590 with bad export cookie 2657689133608812435 LustreError: MGC192.168.201.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg159-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 426 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000036 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 18921 Comm: kworker/0:8 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1aa2750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 97 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa87582adbda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff8b77c5df01c0 RCX: 0000000007f6fd04 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000006 RBP: 0000000000000006 R08: 0000000000000447 R09: 0000000000000000 R10: 00000000025607ad R11: ffff8b76c48dd000 R12: ffff8b7802038a00 R13: 00000000000000be R14: ffffffff978587d4 R15: 0000000000000006 FS: 0000000000000000(0000) GS:ffff8b7802000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000036 CR3: 0000000066216004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000036 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:56 (1779419576) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:58 (1779419578) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: server umount lustre-MDT0000 complete LustreError: 16195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419581 with bad export cookie 6561105667533004626 LustreError: MGC192.168.204.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 16195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg458-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 429 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 10863 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1923750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 b3 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb2acc0d4bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff9fa0f6560bc0 RCX: 00000000080dd379 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 000000000000015d R09: 0000000000000000 R10: 000000000262a58b R11: ffff9fa0c868e000 R12: ffff9fa1020b8a00 R13: 00000000000000ba R14: ffffffffb30587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9fa102080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000002a016005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:54 (1779419574) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:55 (1779419575) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 6 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 14829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419580 with bad export cookie 5773536506112438405 LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 14829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 418 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 15087 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1aa4750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 bc 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb94380c8fda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff9173b59d7ac0 RCX: 0000000008193cb8 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000a29 R09: 0000000000000000 R10: 00000000026581cd R11: ffff9173c1775000 R12: ffff9173c21b8a00 R13: 00000000000000ba R14: ffffffffbc4587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9173c2180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000006a016002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix serio_raw ghash_clmulni_intel libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:51 (1779419571) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:52 (1779419572) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 17283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419582 with bad export cookie 13734895319022175057 LustreError: MGC192.168.204.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 17283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg412-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 442 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 33 Comm: kworker/3:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc18d0750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 85 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffacf580d7fda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff8f1585d84d80 RCX: 0000000008305843 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000d15 R09: 0000000000000000 R10: 00000000026d9935 R11: ffff8f16bd0ef000 R12: ffff8f16c21b8a00 R13: 00000000000000ba R14: ffffffff856587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8f16c2180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000006c616001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:53 (1779419573) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:54 (1779419574) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 17797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419580 with bad export cookie 3615524415381355458 LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 17797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg450-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 432 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 15300 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0e7b750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 b7 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff99f580d9bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff8da32bcb2dc0 RCX: 0000000007141a76 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 000000000000034e R09: 0000000000000000 R10: 0000000002550f57 R11: ffff8da202e31000 R12: ffff8da3420b8a00 R13: 00000000000000ba R14: ffffffffb74587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8da342080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000001d216004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:13:02 (1779419582) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:03 (1779419583) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 6608:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 2 previous similar messages LustreError: 6608:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 1 previous similar message LustreError: 7541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419587 with bad export cookie 18287314738695081994 LustreError: MGC192.168.204.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg427-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 417 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000039 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 19510 Comm: kworker/2:6 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0f16750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 b5 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa5bf81bfbda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff99e7845b5280 RCX: 000000000700771c RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009 RBP: 0000000000000009 R08: 0000000000000254 R09: 0000000000000000 R10: 00000000024df012 R11: ffff99e884338000 R12: ffff99e8c2138a00 R13: 00000000000000c1 R14: ffffffffb52587d4 R15: 0000000000000009 FS: 0000000000000000(0000) GS:ffff99e8c2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000039 CR3: 0000000063616001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000039 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:35 (1779419555) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:37 (1779419557) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 8514:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 8514:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419568 with bad export cookie 17622012309791231698 LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 406 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 363 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc122b750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 ae 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb37b0154fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffffa0273724ca80 RCX: 0000000006f6b3bb RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000733 R09: 0000000000000000 R10: 00000000024b5115 R11: ffffa0273f1ed000 R12: ffffa027421b8a00 R13: 00000000000000ba R14: ffffffffaec587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffffa02742180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000139616003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:59 (1779419579) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:13:01 (1779419581) === Lustre: server umount lustre-MDT0000 complete LustreError: 6555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419584 with bad export cookie 7907188334541576406 LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) LustreError: 6572:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6572:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 4 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 428 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000042 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 252 Comm: kworker/0:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 a7 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff97014154bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff8891b5dfc4c0 RCX: 0000000008016dd0 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000012 RBP: 0000000000000012 R08: 0000000000000f95 R09: 0000000000000000 R10: 00000000025c4950 R11: ffff8891afcae000 R12: ffff8891c2038a00 R13: 00000000000000ca R14: ffffffffa76587d4 R15: 0000000000000012 FS: 0000000000000000(0000) GS:ffff8891c2000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000042 CR3: 000000003ca16005 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix serio_raw ghash_clmulni_intel libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000042 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:43 (1779419563) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:44 (1779419564) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419568 with bad export cookie 2046872908669618596 LustreError: 6473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 419 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000034 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 20382 Comm: kworker/2:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc106d750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 8b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa86187e07da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff914744242200 RCX: 00000000070fb81d RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004 RBP: 0000000000000004 R08: 000000000000015d R09: 0000000000000000 R10: 000000000253821d R11: ffff914742eb9000 R12: ffff914882138a00 R13: 00000000000000bc R14: ffffffff8b2587d4 R15: 0000000000000004 FS: 0000000000000000(0000) GS:ffff914882100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000034 CR3: 000000004ac16003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000034 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:31 (1779419551) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:32 (1779419552) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419563 with bad export cookie 18122113969178730519 LustreError: MGC192.168.202.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg218-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 433 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 20982 Comm: kworker/2:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1188750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 88 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffab4fc7df7da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9701699bfa80 RCX: 0000000007043248 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 000000000000091f R09: 0000000000000000 R10: 0000000002514ea6 R11: ffff970044268000 R12: ffff970182138a00 R13: 00000000000000ba R14: ffffffff888587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff970182100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000afe16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:29 (1779419549) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:31 (1779419551) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7832:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7832:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 18443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419562 with bad export cookie 256909516590664971 LustreError: MGC192.168.202.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 18443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg215-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 434 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000036 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 10751 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1ab7750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 8a 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb6c3c1853da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff8dd0d1120b00 RCX: 0000000008220e7f RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000006 RBP: 0000000000000006 R08: 0000000000000e0f R09: 0000000000000000 R10: 000000000267ca07 R11: ffff8dcfc52c3000 R12: ffff8dd102038a00 R13: 00000000000000be R14: ffffffff8a0587d4 R15: 0000000000000006 FS: 0000000000000000(0000) GS:ffff8dd102000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000036 CR3: 00000000a3e16004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000036 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:37 (1779419557) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:39 (1779419559) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419566 with bad export cookie 6971407745864645022 LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 434 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 41 Comm: kworker/2:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1219750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 9f 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb70840dd3da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff937dc46ac080 RCX: 0000000006f78a58 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000d7b R09: 0000000000000000 R10: 000000000248e935 R11: ffff937dc33ce000 R12: ffff937f02138a00 R13: 00000000000000ba R14: ffffffff9f0587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff937f02100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000030a16001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:54 (1779419574) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:55 (1779419575) === Lustre: server umount lustre-MDT0000 complete LustreError: 16881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419579 with bad export cookie 2862197107015726201 LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 16881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 6615:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages LustreError: 6615:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 418 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 10082 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1adf750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 ac 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbc7088e1bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9a8cdf5ab240 RCX: 000000000804cfe0 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000002c9 R09: 0000000000000000 R10: 00000000025dd812 R11: ffff9a8cfc507000 R12: ffff9a8d020b8a00 R13: 00000000000000ba R14: ffffffffacc587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9a8d02080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000b3a16003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:33 (1779419553) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:35 (1779419555) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7216:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419564 with bad export cookie 14957898685189533766 LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7216:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 428 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 18894 Comm: kworker/0:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1aea750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 a5 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffab15c0ddbda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff8cb0c5f4bb00 RCX: 00000000081c98da RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000005b5 R09: 0000000000000000 R10: 000000000265b60c R11: ffff8cb0c3e39000 R12: ffff8cb202038a00 R13: 00000000000000ba R14: ffffffffa56587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8cb202000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000b0616006 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:38 (1779419558) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:39 (1779419559) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 7256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419564 with bad export cookie 6095240030694864353 LustreError: MGC192.168.203.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg360-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 437 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000043 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 27 Comm: kworker/2:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc18ec750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 a3 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb37940d4bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffffa0713d815d80 RCX: 00000000082205f5 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000013 RBP: 0000000000000013 R08: 000000000000015d R09: 0000000000000000 R10: 0000000002669497 R11: ffffa0713c9b6000 R12: ffffa07142138a00 R13: 00000000000000cb R14: ffffffffa3c587d4 R15: 0000000000000013 FS: 0000000000000000(0000) GS:ffffa07142100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000043 CR3: 000000002d616006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic serio_raw ghash_clmulni_intel ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000043 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:32 (1779419552) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:33 (1779419553) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419561 with bad export cookie 8643620604190088853 LustreError: MGC192.168.202.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg241-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 424 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 261 Comm: kworker/3:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc17fa750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 94 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffaaed814a3da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff9b4382543540 RCX: 000000000805a787 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000d07 R09: 0000000000000000 R10: 00000000025fd995 R11: ffff9b44c1bbb000 R12: ffff9b44c21b8a00 R13: 00000000000000ba R14: ffffffff948587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9b44c2180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000125e16001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:49 (1779419569) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:50 (1779419570) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419575 with bad export cookie 6295799767150754740 LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: 6493:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages LustreError: 6493:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 1 previous similar message Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 421 sec | BUG: unable to handle kernel NULL pointer dereference at 000000000000003d PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 252 Comm: kworker/1:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1967750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 b4 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffba6e4147bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9a9ff5b7f200 RCX: 000000000817a7a3 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000d RBP: 000000000000000d R08: 0000000000000b81 R09: 0000000000000000 R10: 0000000002647490 R11: ffff9a9ff3e3c000 R12: ffff9aa0020b8a00 R13: 00000000000000c5 R14: ffffffffb46587d4 R15: 000000000000000d FS: 0000000000000000(0000) GS:ffff9aa002080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000000003d CR3: 000000000f416006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 000000000000003d | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:26 (1779419546) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:27 (1779419547) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 13719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419556 with bad export cookie 4988638849932837410 LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 13719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg323-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 416 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 20092 Comm: kworker/2:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1029750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 95 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa92a81d13da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff987fbe149140 RCX: 00000000071703e6 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000009ab R09: 0000000000000000 R10: 000000000255d76e R11: ffff987fb577e000 R12: ffff987fc2138a00 R13: 00000000000000ba R14: ffffffff952587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff987fc2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000011e216002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:23 (1779419543) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:24 (1779419544) === Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 6584:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: Skipped 4 previous similar messages LustreError: 6584:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 23055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419550 with bad export cookie 9681615963050355956 LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 23055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 435 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000039 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 10814 Comm: kworker/2:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb4d8c108fda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff97735871c880 RCX: 00000000085420d9 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009 RBP: 0000000000000009 R08: 0000000000000541 R09: 0000000000000000 R10: 00000000027c6297 R11: ffff977367c4c000 R12: ffff977382138a00 R13: 00000000000000c1 R14: ffffffff9b8587d4 R15: 0000000000000009 FS: 0000000000000000(0000) GS:ffff977382100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000039 CR3: 0000000074e16002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000039 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:21 (1779419541) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:22 (1779419542) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419547 with bad export cookie 2132650712891078283 LustreError: MGC192.168.201.103@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg103-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 420 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 64 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1a42750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 99 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffaf704145bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff8a84103ecd40 RCX: 00000000083a5a55 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000004bb R09: 0000000000000000 R10: 000000000270d846 R11: ffff8a8439069000 R12: ffff8a84420b8a00 R13: 00000000000000ba R14: ffffffff998587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8a8442080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000000b816005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:07 (1779419527) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:08 (1779419528) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7300:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7300:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419537 with bad export cookie 7927785499230268796 LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 282 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 14389 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc10b4750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 90 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa967c1477da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff8e92bfa2d000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8e92bf92002c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8e92b1512900 R11: 0000000000000007 R12: ffff8e92c2038a00 R13: 00000000000000ba R14: ffffffff900587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8e92c2000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000019216005 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:11:03 (1779419463) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:11:05 (1779419465) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419477 with bad export cookie 405497630518468849 LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 268 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000034 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 418 Comm: kworker/1:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc11b5750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 8e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb8f401513da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff8d2944cad100 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8d29428c182c RDI: 0000000000000004 RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8d2943676a20 R11: 0000000000000007 R12: ffff8d2a820b8a00 R13: 00000000000000bc R14: ffffffff8e2587d4 R15: 0000000000000004 FS: 0000000000000000(0000) GS:ffff8d2a82080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000034 CR3: 0000000080416002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? free_unref_page_commit+0xa6/0x170 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000034 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:46 (1779419446) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:48 (1779419448) === Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 6596:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6596:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 8 previous similar messages Lustre: Skipped 4 previous similar messages LustreError: 8519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419457 with bad export cookie 13694964780692568953 LustreError: MGC192.168.202.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg205-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 284 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000036 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 14420 Comm: kworker/2:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0fa4750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 8f 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9c40c7c5fda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff91183576ba40 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9118012ab82c RDI: 0000000000000006 RBP: 0000000000000006 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9118180e3d80 R11: 00000000ffffffe5 R12: ffff911842138a00 R13: 00000000000000be R14: ffffffff8f2587d4 R15: 0000000000000006 FS: 0000000000000000(0000) GS:ffff911842100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000036 CR3: 000000010d616004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000036 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:33 (1779419433) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:35 (1779419435) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages LustreError: 6608:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6608:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419449 with bad export cookie 10467197850111053380 LustreError: MGC192.168.201.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg109-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 270 sec | BUG: unable to handle kernel NULL pointer dereference at 000000000000003a PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 125 Comm: kworker/2:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 b7 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa49e41473da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9197986c0880 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9197b337d82c RDI: 000000000000000a RBP: 000000000000000a R08: 0000000000000000 R09: 0000000000000000 R10: ffff91968392dd20 R11: 0000000000000007 R12: ffff9197c2138a00 R13: 00000000000000c2 R14: ffffffffb78587d4 R15: 000000000000000a FS: 0000000000000000(0000) GS:ffff9197c2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000000003a CR3: 00000000a4c16005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 000000000000003a | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:35 (1779419435) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:36 (1779419436) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6586:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6586:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 12 previous similar messages LustreError: 6570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419448 with bad export cookie 97610491901618001 LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3718:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419450/real 1779419450] req@ffff9197ac6cea00 x1865856012532480/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419466 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 270 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 11296 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc122f750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 a0 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbaaa80dabda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff90555883ad80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff905442cc9c2c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff905443e20700 R11: 00000000ffffffe6 R12: ffff905582038a00 R13: 00000000000000ba R14: ffffffffa00587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff905582000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000011216006 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:19 (1779419419) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:22 (1779419422) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6591:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6591:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419433 with bad export cookie 13181450963373530675 LustreError: 6573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 254 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 1010 Comm: kworker/0:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1034750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 e5 b4 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb2a7c15a3da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff8ab17a6aa980 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8ab04886382c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8ab14bc21140 R11: 00000000fffffff0 R12: ffff8ab182038a00 R13: 00000000000000ba R14: ffffffffb4e587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8ab182000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000134616004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:27 (1779419427) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:29 (1779419429) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 3 previous similar messages LustreError: 6617:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 6617:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 11 previous similar messages LustreError: 6598:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419442 with bad export cookie 16321237424194636452 LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6598:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3750:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419443/real 1779419443] req@ffff8ab049666680 x1865856024061056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419459 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 261 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 19578 Comm: kworker/2:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1128750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 ac 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa7b04198fda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff96397c62c800 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9638443ed42c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff963844f98660 R11: 00000000ffffffec R12: ffff963982138a00 R13: 00000000000000ba R14: ffffffffaca587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff963982100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000031216003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:11 (1779419411) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:15 (1779419415) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6602:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6602:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419430 with bad export cookie 14922900354298207719 LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3740:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419431/real 1779419431] req@ffff96396bc40000 x1865856005195776/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419447 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg133-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 263 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 15938 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0fbc750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 a7 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffab13c18efda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9252f9dd4d80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9252dda5242c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9252d2eea7e0 R11: 0000000000000007 R12: ffff9253020b8a00 R13: 00000000000000ba R14: ffffffffa74587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff925302080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000012fc16005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:09 (1779419409) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:14 (1779419414) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 9134:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 9134:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 14400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419431 with bad export cookie 10976852890879945735 LustreError: MGC192.168.201.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3748:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419432/real 1779419432] req@ffff925301e7d880 x1865856003100928/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419448 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg157-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 283 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 11370 Comm: kworker/3:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1117750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 9a 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa73800d93da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff999cfb9945c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff999cf4846c2c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff999cd01df720 R11: 0000000000000007 R12: ffff999d021b8a00 R13: 00000000000000ba R14: ffffffff9ac587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff999d02180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000a3e16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:19 (1779419419) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:22 (1779419422) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 4 previous similar messages LustreError: 6615:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6615:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 15 previous similar messages LustreError: 6595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419431 with bad export cookie 2864088190238836336 LustreError: MGC192.168.201.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3747:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419433/real 1779419433] req@ffff999bc4292680 x1865855991569024/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419449 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg130-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 246 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000036 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 125 Comm: kworker/2:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1ad5750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 88 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9f928146bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff8d6cbb981f40 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8d6b84f13c2c RDI: 0000000000000006 RBP: 0000000000000006 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8d6b85d9e700 R11: 0000000000000007 R12: ffff8d6cc2138a00 R13: 00000000000000be R14: ffffffff882587d4 R15: 0000000000000006 FS: 0000000000000000(0000) GS:ffff8d6cc2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000036 CR3: 000000001fe16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000036 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:07 (1779419407) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:09 (1779419409) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 12804:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 12804:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 13618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419422 with bad export cookie 15708265354343142125 LustreError: MGC192.168.204.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 13618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg416-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 272 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000033 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 17467 Comm: kworker/2:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0fa4750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 97 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff96e8c1c3bda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff8a8b81814b80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8a8b8187a82c RDI: 0000000000000003 RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8a8cbf9df7c0 R11: 00000000fffffff4 R12: ffff8a8cc2138a00 R13: 00000000000000bb R14: ffffffff974587d4 R15: 0000000000000003 FS: 0000000000000000(0000) GS:ffff8a8cc2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000033 CR3: 000000007e216003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000033 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:08 (1779419408) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:11 (1779419411) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6601:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6601:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419424 with bad export cookie 8441728126971229480 LustreError: MGC192.168.204.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3737:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419427/real 1779419427] req@ffff8a8ca79b8a80 x1865855995754752/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419443 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: 3740:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419428/real 1779419428] req@ffff8a8cc148c700 x1865855995755008/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419444 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg440-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 249 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 27 Comm: kworker/2:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1b0e750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 9d 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9ee5c0d4bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff90844eca3000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff90844ee5342c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff908445e07120 R11: 00000000ffffffee R12: ffff908482138a00 R13: 00000000000000ba R14: ffffffff9d8587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff908482100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000117e16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:14 (1779419414) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:16 (1779419416) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 15272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419423 with bad export cookie 7663366273834683332 LustreError: MGC192.168.204.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 15272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg408-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 268 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 18140 Comm: kworker/1:10 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0fdc750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 b7 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb95b01dc7da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9f3c7fbd1e00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9f3c5d7c602c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9f3c4b69fca0 R11: 00000000ffffffdc R12: ffff9f3c820b8a00 R13: 00000000000000ba R14: ffffffffb76587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9f3c82080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000005ca16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:12 (1779419412) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:16 (1779419416) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6602:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6602:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 11 previous similar messages LustreError: 14286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419429 with bad export cookie 10445793790326176044 LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 14286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3736:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419431/real 1779419431] req@ffff9f3b413b1180 x1865856005184128/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419447 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 246 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 11068 Comm: kworker/0:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1895750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 97 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa3b10147fda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff901aea61fc00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff901ac2e34c2c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff901ad1578880 R11: 00000000fffffffd R12: ffff901b02038a00 R13: 00000000000000ba R14: ffffffff974587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff901b02000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000004b416006 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:49 (1779419389) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:56 (1779419396) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6483:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6483:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 1 previous similar message LustreError: 16729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419415 with bad export cookie 8520098430159093793 LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 16729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 241 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 16455 Comm: kworker/0:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1252750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 86 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffac44c0c87da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff94c95a108840 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff94c9772b882c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff94c948d95320 R11: 00000000fffffff8 R12: ffff94c982038a00 R13: 00000000000000ba R14: ffffffff860587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff94c982000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000000c016005 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:47 (1779419387) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:51 (1779419391) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 6 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? Lustre: server umount lustre-MDT0000 complete LustreError: 6570:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6570:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 15 previous similar messages LustreError: 7504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419419 with bad export cookie 5302568595236351798 LustreError: 7504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3704:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419423/real 1779419423] req@ffff94c842f0df80 x1865856008312832/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419439 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: 3706:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419424/real 1779419424] req@ffff94c842f0f100 x1865856008313088/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419440 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg345-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 252 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 16375 Comm: kworker/2:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1857750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 86 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbd93c226bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff9d83058505c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9d842f75502c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9d842f7f2820 R11: 00000000fffffff5 R12: ffff9d8442138a00 R13: 00000000000000ba R14: ffffffff86c587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9d8442100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000052416001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:02 (1779419402) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:05 (1779419405) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 4 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7288:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7288:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 9783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419421 with bad export cookie 939250178053248850 LustreError: 9783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg245-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 265 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000031 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 364 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1928750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 93 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9e5c8155fda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff926b04166a80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff926c2c554c2c RDI: 0000000000000001 RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000 R10: ffff926c10431ca0 R11: 0000000000000007 R12: ffff926c420b8a00 R13: 00000000000000b9 R14: ffffffff930587d4 R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff926c42080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000031 CR3: 000000006e816004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000031 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:45 (1779419385) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:47 (1779419387) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 1 previous similar message Lustre: Skipped 6 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6478:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6478:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419399 with bad export cookie 1541187681048811588 LustreError: MGC192.168.204.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg410-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 232 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 14290 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc0fc1750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 a9 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffac0147defda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff93a2cb55e540 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff93a2f67fb82c RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000000 R09: 0000000000000000 R10: ffff93a1c3a38b80 R11: 0000000000000007 R12: ffff93a302038a00 R13: 00000000000000c0 R14: ffffffffa96587d4 R15: 0000000000000008 FS: 0000000000000000(0000) GS:ffff93a302000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000038 CR3: 0000000046216004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000038 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:36 (1779419376) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:39 (1779419379) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 8085:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 8085:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 19410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419400 with bad export cookie 7551588518259557065 LustreError: MGC192.168.203.132@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 19410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3731:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419403/real 1779419403] req@ffff93a2f9d3c000 x1865856006220288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419419 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 3 previous similar messages Lustre: 3728:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419405/real 1779419405] req@ffff93a1c4896a00 x1865856006220544/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419421 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg332-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 231 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 14172 Comm: kworker/1:6 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1a89750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 8f 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa73100d53da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff8ee541ec8cc0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8ee40411942c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8ee506b764c0 R11: 00000000fffffff0 R12: ffff8ee5420b8a00 R13: 00000000000000ba R14: ffffffff8f0587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8ee542080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000061216006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:27 (1779419367) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:30 (1779419370) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6506:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6506:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 13710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419385 with bad export cookie 11221872675651019229 LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 13710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 254 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000043 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 13749 Comm: kworker/3:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc17a3750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 a2 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb4ad02b9bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff95647a5377c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff95647ce0bc2c RDI: 0000000000000013 RBP: 0000000000000013 R08: 0000000000000000 R09: 0000000000000000 R10: ffff95644646a1a0 R11: 00000000ffffffe4 R12: ffff9564821b8a00 R13: 00000000000000cb R14: ffffffffa28587d4 R15: 0000000000000013 FS: 0000000000000000(0000) GS:ffff956482180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000043 CR3: 0000000131e16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000043 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:29 (1779419369) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:32 (1779419372) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7595:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 17952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419384 with bad export cookie 4245760214589641319 LustreError: MGC192.168.201.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 17952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg111-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 249 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 16874 Comm: kworker/3:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1a7f750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 88 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb50981d27da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff93247a32a3c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff932345f6f82c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9324412afd80 R11: 00000000ffffffe8 R12: ffff9324821b8a00 R13: 00000000000000ba R14: ffffffff884587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff932482180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000061816004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:31 (1779419371) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:34 (1779419374) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message LustreError: 3535:0:(import.c:719:ptlrpc_connect_import_locked()) already connecting Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7650:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419386 with bad export cookie 3463025127587101675 LustreError: MGC192.168.201.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3538:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419389/real 1779419389] req@ffff93234435d880 x1865855988409344/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419405 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 247 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000048 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 763 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1afe750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 8a 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbacb41637da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff98e04050afc0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff98df0589ac2c RDI: 0000000000000018 RBP: 0000000000000018 R08: 0000000000000000 R09: 0000000000000000 R10: ffff98e004848820 R11: 00000000ffffffec R12: ffff98e0421b8a00 R13: 00000000000000d0 R14: ffffffff8aa587d4 R15: 0000000000000018 FS: 0000000000000000(0000) GS:ffff98e042180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000048 CR3: 0000000013a16005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] CR2: 0000000000000048 | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:09:31 (1779419371) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:09:34 (1779419374) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 1 previous similar message Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7577:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7577:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419390 with bad export cookie 11439687813741981344 LustreError: MGC192.168.201.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3538:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419393/real 1779419393] req@ffff98e041595c00 x1865856002036992/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419409 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg150-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |