| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| recovery-small test complete, duration 6600 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 121536 Comm: kworker/2:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 bc 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffaf7701d37da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff9d1975e97180 RCX: 0000000700155795 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000008 RBP: 0000000000000008 R08: 000000000000099d R09: 0000000000000000 R10: 00000002b13e5a65 R11: ffff9d184491a000 R12: ffff9d1982138a00 R13: 00000000000000c0 R14: ffffffffbc2587d4 R15: 0000000000000008 FS: 0000000000000000(0000) GS:ffff9d1982100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000038 CR3: 000000002b416006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000038 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 03:57:53 (1779436673) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 04:00:06 (1779436806) === LustreError: 3728:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff9d19807e0e00 x1865867487003648/t0(0) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 0 ref 1 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 LustreError: 3728:0:(client.c:1380:ptlrpc_import_delay_req()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 107704:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 11 previous similar messages LustreError: 107704:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 132 previous similar messages LustreError: 10379:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779436812 with bad export cookie 13428707713907587074 LustreError: 10379:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages LustreError: MGC192.168.202.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: Skipped 8 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg239-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 319 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 365 Comm: kworker/0:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 e5 b6 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffc10241547da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9d083b6722c0 RCX: 00000000065b2c7d RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000df9 R09: 0000000000000000 R10: 000000000209ea3a R11: ffff9d0703279000 R12: ffff9d0842038a00 R13: 00000000000000ba R14: ffffffffb6e587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9d0842000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000050816001 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:18:22 (1779430702) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:18:24 (1779430704) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 6 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? Lustre: server umount lustre-MDT0000 complete LustreError: 6541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430723 with bad export cookie 1898894844430534587 LustreError: MGC192.168.204.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3697:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430723/real 1779430723] req@ffff9d083a47e680 x1865867912835200/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430739 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg404-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 314 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 20270 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 8c 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa9bac158fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9d5102ca8c00 RCX: 00000000065dd5f1 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000725 R09: 0000000000000000 R10: 00000000020d7da1 R11: ffff9d5225209000 R12: ffff9d5242038a00 R13: 00000000000000ba R14: ffffffff8c6587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9d5242000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000029e16004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:18:10 (1779430690) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:18:11 (1779430691) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 7497:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: Skipped 1 previous similar message LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 14332:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430695 with bad export cookie 9133900908147730090 LustreError: MGC192.168.203.137@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 14332:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg337-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 320 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000044 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 17862 Comm: kworker/3:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 b0 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb70301a03da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff93f50928c780 RCX: 00000000074334a7 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000014 RBP: 0000000000000014 R08: 00000000000008ab R09: 0000000000000000 R10: 00000000020bde95 R11: ffff93f508ffa000 R12: ffff93f5463b8a00 R13: 00000000000000cc R14: ffffffffb0c587d4 R15: 0000000000000014 FS: 0000000000000000(0000) GS:ffff93f546380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000044 CR3: 0000000141616002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000044 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:17:31 (1779430651) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:17:32 (1779430652) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 16117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430657 with bad export cookie 11647957180458780911 LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 16117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 407 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 20217 Comm: kworker/1:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 8e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbbd741cc7da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9c4c5e0ce400 RCX: 000000000700b935 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000006bf R09: 0000000000000000 R10: 00000000024daa8c R11: ffff9c4c4352f000 R12: ffff9c4c820b8a00 R13: 00000000000000ba R14: ffffffff8e4587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9c4c82080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000aa016005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:59 (1779430439) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:14:00 (1779430440) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 6 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430450 with bad export cookie 9614024728863014042 LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 427 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 808 Comm: kworker/0:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 98 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa43c8156fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff974b7a6d6440 RCX: 00000000074a2aeb RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000006c0 R09: 0000000000000000 R10: 00000000026d269d R11: ffff974a432d8000 R12: ffff974b82038a00 R13: 00000000000000ba R14: ffffffff984587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff974b82000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000029a16006 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:45 (1779430425) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:46 (1779430426) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? Lustre: server umount lustre-MDT0000 complete LustreError: 6574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430444 with bad export cookie 16640848644849654853 LustreError: MGC192.168.201.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg105-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 432 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 42 Comm: kworker/2:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 88 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb0c440ddbda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff949b473b7a40 RCX: 000000000829b1ca RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000633 R09: 0000000000000000 R10: 00000000026d4f80 R11: ffff949b7ebc5000 R12: ffff949b82138a00 R13: 00000000000000ba R14: ffffffff882587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff949b82100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000b2c16003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:14:01 (1779430441) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:14:03 (1779430443) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 19344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430451 with bad export cookie 5255017561400149703 LustreError: 19344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages LustreError: MGC192.168.202.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg219-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 434 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 19548 Comm: kworker/3:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 8e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb18940d7fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffffa04b714553c0 RCX: 0000000007360d36 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000254 R09: 0000000000000000 R10: 0000000002664304 R11: ffffa04a436dd000 R12: ffffa04b821b8a00 R13: 00000000000000ba R14: ffffffff8e0587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffffa04b82180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000b6e16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:52 (1779430432) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:54 (1779430434) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 7891:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7891:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 4 previous similar messages Lustre: Skipped 5 previous similar messages LustreError: 6586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430438 with bad export cookie 9235532807865201076 LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 402 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000039 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 25526 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 99 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb94b81c9bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9e0408c023c0 RCX: 0000000006ec7999 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009 RBP: 0000000000000009 R08: 00000000000001cb R09: 0000000000000000 R10: 00000000024450e4 R11: ffff9e0510fb5000 R12: ffff9e05421b8a00 R13: 00000000000000c1 R14: ffffffff998587d4 R15: 0000000000000009 FS: 0000000000000000(0000) GS:ffff9e0542180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000039 CR3: 0000000121e16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000039 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:53 (1779430433) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:54 (1779430434) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 2 previous similar messages LustreError: 19468:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 19468:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430440 with bad export cookie 16937814501893856818 LustreError: MGC192.168.203.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg305-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 415 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 6 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 95 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffaeac80c87da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff90c409a11680 RCX: 000000000820c053 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000825 R09: 0000000000000000 R10: 0000000002655b19 R11: ffff90c50917f000 R12: ffff90c542038a00 R13: 00000000000000ba R14: ffffffff950587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff90c542000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000120416001 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:54 (1779430434) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:55 (1779430435) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 13587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430440 with bad export cookie 15455784488534540054 LustreError: 13587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages LustreError: MGC192.168.204.103@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg403-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 415 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 15745 Comm: kworker/3:6 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 85 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb65041d0fda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff8916c4a24000 RCX: 00000000071a6123 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000062 R09: 0000000000000000 R10: 000000000259c147 R11: ffff8916c443d000 R12: ffff8918021b8a00 R13: 00000000000000ba R14: ffffffff85c587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff891802180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000076416004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:47 (1779430427) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:49 (1779430429) === Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 12049:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: Skipped 4 previous similar messages LustreError: 12049:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 1 previous similar message LustreError: 6584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430433 with bad export cookie 17004876481657456947 LustreError: 6584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 434 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000066 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 55 Comm: kworker/2:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 8c 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb28e40e3bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff99a1c7dab3c0 RCX: 000000000813c734 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000036 RBP: 0000000000000036 R08: 0000000000000633 R09: 0000000000000000 R10: 00000000026302bf R11: ffff99a1cbd43000 R12: ffff99a202138a00 R13: 00000000000000ee R14: ffffffff8c8587d4 R15: 0000000000000036 FS: 0000000000000000(0000) GS:ffff99a202100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000066 CR3: 0000000036616004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000066 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:51 (1779430431) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:52 (1779430432) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430437 with bad export cookie 3122608717661596139 LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 459 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000033 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 254 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 e5 a3 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffae4081463da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff95678ae91440 RCX: 0000000007ed7641 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000003 RBP: 0000000000000003 R08: 00000000000000ef R09: 0000000000000000 R10: 0000000002563979 R11: ffff9567bdea1000 R12: ffff9567c21b8a00 R13: 00000000000000bb R14: ffffffffa3e587d4 R15: 0000000000000003 FS: 0000000000000000(0000) GS:ffff9567c2180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000033 CR3: 0000000118616002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000033 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:50 (1779430430) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:51 (1779430431) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430440 with bad export cookie 8458797646242920603 LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 450 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000042 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 15053 Comm: kworker/0:7 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffba6ac1547da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9f6a0954da80 RCX: 00000000080dbf39 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000012 RBP: 0000000000000012 R08: 0000000000000b23 R09: 0000000000000000 R10: 00000000025f5ae1 R11: ffff9f6b06f67000 R12: ffff9f6b42038a00 R13: 00000000000000ca R14: ffffffff9b0587d4 R15: 0000000000000012 FS: 0000000000000000(0000) GS:ffff9f6b42000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000042 CR3: 0000000109816002 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000042 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:52 (1779430432) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:53 (1779430433) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430437 with bad export cookie 3367486948464947678 LustreError: MGC192.168.201.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg107-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 430 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000036 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 17933 Comm: kworker/2:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 9c 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb5d040d93da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff930604005e00 RCX: 0000000008224568 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000006 RBP: 0000000000000006 R08: 00000000000006c0 R09: 0000000000000000 R10: 0000000002695a07 R11: ffff93072602b000 R12: ffff930742138a00 R13: 00000000000000be R14: ffffffff9c6587d4 R15: 0000000000000006 FS: 0000000000000000(0000) GS:ffff930742100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000036 CR3: 000000000be16004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000036 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:46 (1779430426) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:47 (1779430427) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 9743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430433 with bad export cookie 15944703652208969478 LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 9743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 430 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 1438 Comm: kworker/0:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 8e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb2b3418dfda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffffa0134b507000 RCX: 000000000724a9aa RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000004bb R09: 0000000000000000 R10: 00000000025eb8e7 R11: ffffa0124338f000 R12: ffffa01382038a00 R13: 00000000000000ba R14: ffffffff8e6587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffffa01382000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000053e16003 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:38 (1779430418) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:39 (1779430419) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 6584:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6584:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 4 previous similar messages Lustre: Skipped 4 previous similar messages LustreError: 6564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430424 with bad export cookie 17372790085207518866 LustreError: MGC192.168.204.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg458-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 429 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000067 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 23641 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 a8 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb77cc0da3da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9de56e358600 RCX: 00000000071b89d5 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000037 RBP: 0000000000000037 R08: 0000000000000ac0 R09: 0000000000000000 R10: 000000000257e071 R11: ffff9de443986000 R12: ffff9de5820b8a00 R13: 00000000000000ef R14: ffffffffa8c587d4 R15: 0000000000000037 FS: 0000000000000000(0000) GS:ffff9de582080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000067 CR3: 000000008b416004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000067 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:36 (1779430416) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:38 (1779430418) === Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 14410:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 14410:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 10376:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430424 with bad export cookie 15913440074261935014 LustreError: MGC192.168.204.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 10376:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg412-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 441 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000040 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 20931 Comm: kworker/0:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 bb 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffc1ee82d2bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9c503b5e2fc0 RCX: 00000000080dcf8b RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000010 RBP: 0000000000000010 R08: 0000000000000254 R09: 0000000000000000 R10: 00000000025f632f R11: ffff9c503c22b000 R12: ffff9c5042038a00 R13: 00000000000000c8 R14: ffffffffbb0587d4 R15: 0000000000000010 FS: 0000000000000000(0000) GS:ffff9c5042000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000040 CR3: 000000005f816002 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000040 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:42 (1779430422) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:43 (1779430423) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-MDT0000 complete LustreError: 12749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430427 with bad export cookie 15507323827683518295 LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 12749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 429 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000043 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 19941 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 e5 b3 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa92bc0d17da8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff959d107a3280 RCX: 00000000073481dc RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000013 RBP: 0000000000000013 R08: 00000000000004bb R09: 0000000000000000 R10: 0000000002637bc7 R11: ffff959c0383c000 R12: ffff959d421b8a00 R13: 00000000000000cb R14: ffffffffb3e587d4 R15: 0000000000000013 FS: 0000000000000000(0000) GS:ffff959d42180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000043 CR3: 0000000028016006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000043 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:33 (1779430413) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:35 (1779430415) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7534:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7534:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430426 with bad export cookie 5605927353733741143 LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 436 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 138 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 9e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbcba41537da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff95662042fe40 RCX: 000000000815a1e7 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 000000000000015d R09: 0000000000000000 R10: 000000000263fc68 R11: ffff95663718e000 R12: ffff9566420b8a00 R13: 00000000000000ba R14: ffffffff9e4587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff956642080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000002a416004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:38 (1779430418) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:39 (1779430419) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 9778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430428 with bad export cookie 13650916984518861022 LustreError: MGC192.168.201.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg127-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 426 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 10547 Comm: kworker/0:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 a2 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb07741c87da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9fe40882c200 RCX: 00000000071b2db3 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000002c9 R09: 0000000000000000 R10: 00000000025a771d R11: ffff9fe53d3b1000 R12: ffff9fe542038a00 R13: 00000000000000ba R14: ffffffffa24587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9fe542000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000101c16006 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:50 (1779430430) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:51 (1779430431) === Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 6588:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: Skipped 1 previous similar message LustreError: 6575:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430435 with bad export cookie 14261195532980851648 LustreError: 6575:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: MGC192.168.204.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg425-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 408 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 17925 Comm: kworker/3:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 86 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb9e3814cfda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9f6b470cc680 RCX: 0000000006f42b8a RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000e63 R09: 0000000000000000 R10: 000000000247ae96 R11: ffff9f6b4705a000 R12: ffff9f6c863b8a00 R13: 00000000000000ba R14: ffffffff868587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9f6c86380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000013fe16003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:26 (1779430406) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:27 (1779430407) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6587:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6587:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430417 with bad export cookie 4820950684914804327 LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 420 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 20372 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 88 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff996442263da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff88ef446e0a40 RCX: 0000000007fe114f RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000062 R09: 0000000000000000 R10: 00000000025aa382 R11: ffff88f081cd8000 R12: ffff88f0820b8a00 R13: 00000000000000ba R14: ffffffff886587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff88f082080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000032c16002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_generic serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:27 (1779430407) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:28 (1779430408) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 4 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430417 with bad export cookie 725720893404804181 LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 426 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 13745 Comm: kworker/3:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 91 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb93b00d4bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff93cf02e05ec0 RCX: 0000000008311def RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 00000000000000ef R09: 0000000000000000 R10: 0000000002704240 R11: ffff93cf3cfbb000 R12: ffff93cf421b8a00 R13: 00000000000000ba R14: ffffffff918587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff93cf42180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000060216006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:30 (1779430410) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:31 (1779430411) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 6475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430416 with bad export cookie 10151308988211177296 LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 419 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000047 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 11108 Comm: kworker/3:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 b6 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa1f701c5fda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff8b6b08e6d280 RCX: 00000000082316bb RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000017 RBP: 0000000000000017 R08: 00000000000007b9 R09: 0000000000000000 R10: 00000000026a2fce R11: ffff8b6a0514a000 R12: ffff8b6b421b8a00 R13: 00000000000000cf R14: ffffffffb6a587d4 R15: 0000000000000017 FS: 0000000000000000(0000) GS:ffff8b6b42180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000047 CR3: 000000007cc16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000047 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:24 (1779430404) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:25 (1779430405) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-MDT0000 complete LustreError: 7231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430411 with bad export cookie 15781842918692719555 LustreError: 7231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: 3523:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430412/real 1779430412] req@ffff8b6b3591d500 x1865867393889792/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430428 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: 3524:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430412/real 1779430412] req@ffff8b6b3591ea00 x1865867393889920/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430428 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: 3523:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430416/real 1779430416] req@ffff8b6b3591f100 x1865867393890432/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430432 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 434 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 20776 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 a2 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb37e00d17da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9e8d44031280 RCX: 00000000085e56c9 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000c89 R09: 0000000000000000 R10: 0000000002802b98 R11: ffff9e8e7ece7000 R12: ffff9e8e820b8a00 R13: 00000000000000ba R14: ffffffffa2c587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9e8e82080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000038416006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:23 (1779430403) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:24 (1779430404) === Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 17109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430409 with bad export cookie 13070438931086420452 LustreError: MGC192.168.201.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 17109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg104-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 419 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000037 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 10452 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 af 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffadf248b4bda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffffa0390a5014c0 RCX: 00000000082e0944 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000007 RBP: 0000000000000007 R08: 00000000000006bf R09: 0000000000000000 R10: 00000000026c7c83 R11: ffffa0390a9b6000 R12: ffffa03a420b8a00 R13: 00000000000000bf R14: ffffffffaf4587d4 R15: 0000000000000007 FS: 0000000000000000(0000) GS:ffffa03a42080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000037 CR3: 000000010be16003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000037 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:13:23 (1779430403) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:13:24 (1779430404) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-MDT0000 complete LustreError: 6451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430408 with bad export cookie 18397334638328969190 LustreError: MGC192.168.203.103@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg303-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 469 sec | BUG: unable to handle kernel NULL pointer dereference at 000000000000003a PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 764 Comm: kworker/2:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 b4 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffaa36c15cbda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff92fd845577c0 RCX: 0000000007691223 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000a RBP: 000000000000000a R08: 00000000000007b1 R09: 0000000000000000 R10: 00000000027b7f7c R11: ffff92febf57a000 R12: ffff92fec2138a00 R13: 00000000000000c2 R14: ffffffffb48587d4 R15: 000000000000000a FS: 0000000000000000(0000) GS:ffff92fec2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000000003a CR3: 0000000078416001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 000000000000003a | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:12:36 (1779430356) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:12:37 (1779430357) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 6 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430368 with bad export cookie 13562056021939569873 LustreError: MGC192.168.203.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg353-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 456 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 254 Comm: kworker/3:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 bd 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa747c149fda8 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff9259d0774100 RCX: 00000000087808d3 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000062 R09: 0000000000000000 R10: 00000000028f8111 R11: ffff9259c53a1000 R12: ffff925a021b8a00 R13: 00000000000000ba R14: ffffffffbd0587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff925a02180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000a3816003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? snprintf+0x51/0x70 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === recovery-small: start cleanup 02:12:26 (1779430346) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 02:12:28 (1779430348) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 8926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430353 with bad export cookie 13416074020880264187 LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 266 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 21714 Comm: kworker/1:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 e5 9e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa18cc1ca3da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9449e23d0080 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9449eb42f42c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9449e2156b40 R11: 0000000000000007 R12: ffff944a020b8a00 R13: 00000000000000ba R14: ffffffff9ee587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff944a02080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000a3416003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:32 (1779430292) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:35 (1779430295) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6593:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6593:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 11 previous similar messages LustreError: 9572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430307 with bad export cookie 3044266657742929237 LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 9572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 260 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 20682 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 b0 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffac9c41c3bda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff8dee90ed4e40 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8dee9e36c02c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8deec1b670a0 R11: 00000000ffffffec R12: ffff8deec20b8a00 R13: 00000000000000ba R14: ffffffffb02587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8deec2080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000008f816005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:31 (1779430291) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:33 (1779430293) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 8720:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 8720:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 15 previous similar messages LustreError: 7984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430306 with bad export cookie 13497940949060736010 LustreError: MGC192.168.202.137@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg237-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 268 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 768 Comm: kworker/1:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 8c 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbbef417bbda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9771b2859000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9771baaf282c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9771bfac7980 R11: 00000000ffffffe3 R12: ffff9771c20b8a00 R13: 00000000000000ba R14: ffffffff8c6587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff9771c2080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000026616006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:21 (1779430281) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:24 (1779430284) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 16113:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 16113:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 15 previous similar messages LustreError: 16347:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430299 with bad export cookie 987231830876800497 LustreError: MGC192.168.204.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 16347:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg439-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 224 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 3220 Comm: kworker/3:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 8e 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9ebf818abda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff904c84517900 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff904d9022082c RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000000 R09: 00000000000000b0 R10: ffff904dbab49f20 R11: 00000000fffffff6 R12: ffff904dc21b8a00 R13: 00000000000000c0 R14: ffffffff8e4587d4 R15: 0000000000000008 FS: 0000000000000000(0000) GS:ffff904dc2180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000038 CR3: 000000007b416003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000038 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:27 (1779430287) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:29 (1779430289) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 8014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430299 with bad export cookie 9529262215970159494 LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 238 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 21 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 84 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb1f5c0d17da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff991b458619c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff991c46580c2c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff991b43a57c60 R11: 00000000fffffff4 R12: ffff991c820b8a00 R13: 00000000000000ba R14: ffffffff848587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff991c82080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000000d416004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:26 (1779430286) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:29 (1779430289) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430297 with bad export cookie 7780496318856405064 LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 241 sec | BUG: unable to handle kernel NULL pointer dereference at 00000000000000a3 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 16373 Comm: kworker/0:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 05 b4 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffc1c5441bfda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff9d34053888c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9d3403bd882c RDI: 0000000000000073 RBP: 0000000000000073 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9d3510616740 R11: 0000000000000007 R12: ffff9d3542038a00 R13: 000000000000012b R14: ffffffffb40587d4 R15: 0000000000000073 FS: 0000000000000000(0000) GS:ffff9d3542000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000a3 CR3: 000000006c216002 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 00000000000000a3 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:20 (1779430280) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:23 (1779430283) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430291 with bad export cookie 6194988183592890771 LustreError: MGC192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 7217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 257 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 1410 Comm: kworker/2:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 a0 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9b4201d5fda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff8ee868ce1800 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8ee7485d582c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8ee749a767c0 R11: 00000000ffffffec R12: ffff8ee882138a00 R13: 00000000000000ba R14: ffffffffa04587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8ee882100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000122816003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:14 (1779430274) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:18 (1779430278) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-MDT0000 complete LustreError: 13681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430292 with bad export cookie 14168295420554528047 LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 13681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 224 sec | BUG: unable to handle kernel NULL pointer dereference at 000000000000003d PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 126 Comm: kworker/1:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb56581483da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff9d15465f8280 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9d157fe6902c RDI: 000000000000000d RBP: 000000000000000d R08: 0000000000000000 R09: 0000000000000000 R10: ffff9d144a2d2da0 R11: 0000000000000007 R12: ffff9d15820b8a00 R13: 00000000000000c5 R14: ffffffff9b6587d4 R15: 000000000000000d FS: 0000000000000000(0000) GS:ffff9d1582080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000000003d CR3: 0000000133e16003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 000000000000003d | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:14 (1779430274) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:17 (1779430277) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6487:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6487:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 8894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430289 with bad export cookie 2836812006958386052 LustreError: MGC192.168.203.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg321-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 281 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 19569 Comm: kworker/0:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 a2 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffabed01517da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff8af185075440 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8af2a05da82c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 00000000000000b0 R10: ffff8af2900028a0 R11: 00000000ffffffde R12: ffff8af2c2038a00 R13: 00000000000000ba R14: ffffffffa2a587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8af2c2000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000039416001 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:00 (1779430260) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:03 (1779430263) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6602:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6602:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 17 previous similar messages LustreError: 6589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430282 with bad export cookie 5144081363440275216 LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 264 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 16485 Comm: kworker/2:6 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 89 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa94347ddbda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff95877abf1c00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff958642a1f02c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff958643daac00 R11: 00000000ffffffe8 R12: ffff958782138a00 R13: 00000000000000ba R14: ffffffff898587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff958782100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000b7c16002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:08 (1779430268) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:11 (1779430271) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6594:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6594:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430283 with bad export cookie 5845118048678329167 LustreError: MGC192.168.201.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3729:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430286/real 1779430286] req@ffff958643418380 x1865867395878528/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430302 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg117-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 248 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 27 Comm: kworker/2:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 85 ab 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb12f40d4bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff9895f9755b80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9895c7834c2c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9894ca177680 R11: 0000000000000007 R12: ffff989602138a00 R13: 00000000000000ba R14: ffffffffab8587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff989602100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000134616004 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:09 (1779430269) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:12 (1779430272) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 4 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6478:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6478:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 8892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430287 with bad export cookie 15012100583578153950 LustreError: MGC192.168.204.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg452-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 253 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 14431 Comm: kworker/2:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffac624164fda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff98f7df8e8a00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff98f7de62142c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff98f800f0a0e0 R11: 0000000000000007 R12: ffff98f802138a00 R13: 00000000000000ba R14: ffffffff9b2587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff98f802100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 000000007f016006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:13 (1779430273) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:17 (1779430277) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6596:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6596:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6576:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430294 with bad export cookie 16863891291504709675 LustreError: MGC192.168.204.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6576:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg459-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 254 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 18819 Comm: kworker/2:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 88 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9e7342b53da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff89889cedb240 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff89888a22b42c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff898786332960 R11: 00000000fffffff0 R12: ffff8988c2138a00 R13: 00000000000000ba R14: ffffffff886587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8988c2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000016816006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:06 (1779430266) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:09 (1779430269) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 14158:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 14158:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 8021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430279 with bad export cookie 392238944462480390 LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 245 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 11214 Comm: kworker/3:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 9d 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa44c02defda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff962b03f0bcc0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff962b3547d42c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff962b04e219a0 R11: 00000000fffffff4 R12: ffff962b421b8a00 R13: 00000000000000ba R14: ffffffff9da587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff962b42180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000096216003 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:11:08 (1779430268) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:11 (1779430271) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-MDT0000 complete LustreError: 12782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430284 with bad export cookie 17577486981325341243 LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 12782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 244 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 54 Comm: kworker/0:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 9a 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa75fc0e33da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff8ed84500d780 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8ed84539202c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8ed948d8f980 R11: 0000000000000007 R12: ffff8ed982038a00 R13: 00000000000000ba R14: ffffffff9a4587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8ed982000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 00000000a8416006 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_generic serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:53 (1779430253) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:58 (1779430258) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6476:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6476:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430275 with bad export cookie 14458197897102217030 LustreError: MGC192.168.203.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg304-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 254 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 2 PID: 137 Comm: kworker/2:2 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 bb 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffa74881527da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff95a2bc0a6e80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff95a2bcd1e42c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff95a2bbb8c820 R11: 00000000fffffffe R12: ffff95a2c2138a00 R13: 00000000000000ba R14: ffffffffbbc587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff95a2c2100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000062616001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:59 (1779430259) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:11:00 (1779430260) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6582:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6582:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 6567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430277 with bad export cookie 10721166181470581032 LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3722:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430280/real 1779430280] req@ffff95a1830fa680 x1865867399015168/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430296 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: 3719:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430281/real 1779430281] req@ffff95a183225180 x1865867399015424/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430297 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 258 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 18076 Comm: kworker/0:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff9cff4798bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff91f9411e6b00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff91f92fde802c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff91f804edadc0 R11: 00000000fffffff0 R12: ffff91f942038a00 R13: 00000000000000ba R14: ffffffff9bc587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff91f942000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000069e16001 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:51 (1779430251) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:53 (1779430253) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 6 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430262 with bad export cookie 3164976962298547526 LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 250 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000075 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 15300 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 e5 a6 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb75ec1c77da8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff91cdc4612500 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff91cef2537c2c RDI: 0000000000000045 RBP: 0000000000000045 R08: 0000000000000000 R09: 0000000000000000 R10: ffff91cec52b7600 R11: 00000000ffffffe4 R12: ffff91cf020b8a00 R13: 00000000000000fd R14: ffffffffa6e587d4 R15: 0000000000000045 FS: 0000000000000000(0000) GS:ffff91cf02080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000075 CR3: 000000005ee16005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000075 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:44 (1779430244) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:48 (1779430248) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6481:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6481:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 3 previous similar messages LustreError: 8907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430262 with bad export cookie 3170459302683606336 LustreError: MGC192.168.203.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg328-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 241 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 33 Comm: kworker/3:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 9c 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffadf380d7fda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff88a743d62780 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff88a7456fe42c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff88a7437ea880 R11: 00000000ffffffdd R12: ffff88a8821b8a00 R13: 00000000000000ba R14: ffffffff9c6587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff88a882180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000068a16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:33 (1779430233) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:35 (1779430235) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 7249:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 7249:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 6463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430246 with bad export cookie 290363235436018015 LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 246 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000037 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 53 Comm: kworker/0:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 b3 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb1b240e2bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff9976d1a111c0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9975c3b50c2c RDI: 0000000000000007 RBP: 0000000000000007 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9976fe58d380 R11: 0000000000000007 R12: ffff997702038a00 R13: 00000000000000bf R14: ffffffffb3a587d4 R15: 0000000000000007 FS: 0000000000000000(0000) GS:ffff997702000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000037 CR3: 000000007ae16004 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? 0xffffffffb0200000 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000037 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:31 (1779430231) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:34 (1779430234) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430245 with bad export cookie 14042667980068902085 LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 264 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000032 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 16395 Comm: kworker/3:7 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 ba 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffad6180e2bda8 EFLAGS: 00010206 RAX: 0000000000000001 RBX: ffff8df56dbd3880 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff8df5683ff82c RDI: 0000000000000002 RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 R10: ffff8df5433fa0e0 R11: 00000000ffffffde R12: ffff8df5821b8a00 R13: 00000000000000ba R14: ffffffffbaa587d4 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8df582180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000032 CR3: 0000000083816002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel serio_raw ata_piix libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000032 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:24 (1779430224) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:27 (1779430227) === Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 2 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6483:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6483:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 7 previous similar messages LustreError: 16257:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430242 with bad export cookie 12946936092449581822 LustreError: MGC192.168.202.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 16257:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg254-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 229 sec | BUG: unable to handle kernel NULL pointer dereference at 0000000000000034 PGD 0 P4D 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 15415 Comm: kworker/3:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffba8640c8fda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9e6aefc11b80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9e6afa49182c RDI: 0000000000000004 RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9e6ae020b920 R11: 00000000fffffff0 R12: ffff9e6b021b8a00 R13: 00000000000000bc R14: ffffffff9bc587d4 R15: 0000000000000004 FS: 0000000000000000(0000) GS:ffff9e6b02180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000034 CR3: 000000000ac16006 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? no_context+0x30f/0x5a0 ? __bad_area_nosemaphore+0x1c6/0x260 ? bad_area_nosemaphore+0x1a/0x30 ? do_user_addr_fault+0x540/0x8a0 ? __free_pages+0x102/0x160 ? __do_page_fault+0x6b/0xa0 ? do_page_fault+0x87/0x30f ? page_fault+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] gssiam_delete_sec_context+0x22/0x80 [ptlrpc_gss] lgss_delete_sec_context+0x60/0x2d0 [ptlrpc_gss] __rsc_free+0x9b/0x140 [ptlrpc_gss] rsc_entry_free_work+0x21/0xf0 [ptlrpc_gss] process_one_work+0x2c8/0x700 worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc_gss(O-) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: mgc] CR2: 0000000000000034 | Lustre: DEBUG MARKER: === replay-single: start cleanup 02:10:19 (1779430219) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 02:10:21 (1779430221) === Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 3 previous similar messages Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779430229 with bad export cookie 9813332119592248189 LustreError: MGC192.168.204.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3542:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779430229/real 1779430229] req@ffff9e6afc44d180 x1865867400038016/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779430245 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg414-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |