| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| recovery-small test complete, duration 412 sec | stack segment: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 15732 Comm: kworker/0:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 86 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffba74c1a73da8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff8ecfbd2ffb40 RCX: 0000000006fa4a8c RDX: 0000000000000000 RSI: 0000000000000000 RDI: 7479625f6e695f65 RBP: 7479625f6e695f65 R08: 00000000000003b0 R09: 0000000000000000 R10: 00000000024d7541 R11: ffff8ece82207000 R12: ffff8ecfc2038a00 R13: 7479625f6e69601d R14: ffffffff86c587d4 R15: 7479625f6e695f65 FS: 0000000000000000(0000) GS:ffff8ecfc2000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055e0d72319f8 CR3: 000000001d616001 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? die+0x3a/0x6f ? do_trap+0x118/0x150 ? do_error_trap+0x8b/0x130 ? do_stack_segment+0x29/0x40 ? stack_segment+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul serio_raw crc32c_intel ata_piix ghash_clmulni_intel libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:42 (1779419562) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:43 (1779419563) === Lustre: server umount lustre-MDT0000 complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 10186:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: Skipped 6 previous similar messages LustreError: 10186:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 2 previous similar messages LustreError: 14271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419570 with bad export cookie 1689989591323316010 LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 14271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 436 sec | stack segment: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 24715 Comm: kworker/3:1 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 45 9b 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffff94ca4145bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff88f7c683bb00 RCX: 000000000720c51d RDX: 0000000000000000 RSI: 0000000000000000 RDI: 6d65747379732f30 RBP: 6d65747379732f30 R08: 0000000000000b42 R09: 0000000000000000 R10: 00000000025c32d6 R11: ffff88f7c8a71000 R12: ffff88f9063b8a00 R13: 6d65747379732fe8 R14: ffffffff9b4587d4 R15: 6d65747379732f30 FS: 0000000000000000(0000) GS:ffff88f906380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fe547f100fa CR3: 0000000142e16001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? die+0x3a/0x6f ? do_trap+0x118/0x150 ? do_error_trap+0x8b/0x130 ? do_stack_segment+0x29/0x40 ? stack_segment+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:38 (1779419558) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:39 (1779419559) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 12231:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: Skipped 3 previous similar messages LustreError: 8513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419565 with bad export cookie 16578151224234332984 LustreError: MGC192.168.202.132@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg232-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| recovery-small test complete, duration 423 sec | stack segment: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 3 PID: 252 Comm: kworker/3:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 25 8f 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbc5b0149bda8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff9d63f7917d00 RCX: 000000000813ee13 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 6d2e646565736432 RBP: 6d2e646565736432 R08: 0000000000000633 R09: 0000000000000000 R10: 0000000002601f7d R11: ffff9d62c5c3b000 R12: ffff9d64021b8a00 R13: 6d2e6465657364ea R14: ffffffff8f2587d4 R15: 6d2e646565736432 FS: 0000000000000000(0000) GS:ffff9d6402180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000555d1d9549f8 CR3: 0000000077016002 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? die+0x3a/0x6f ? do_trap+0x118/0x150 ? do_error_trap+0x8b/0x130 ? do_stack_segment+0x29/0x40 ? stack_segment+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) zfs(O) spl(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] | Lustre: DEBUG MARKER: === recovery-small: start cleanup 23:12:27 (1779419547) === Lustre: DEBUG MARKER: === recovery-small: finish cleanup 23:12:28 (1779419548) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419553 with bad export cookie 514168117665558300 LustreError: MGC192.168.203.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg349-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 253 sec | stack segment: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 21 Comm: kworker/1:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events rsc_entry_free_work [ptlrpc_gss] RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 a5 8f 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbbb2c0d17da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff9fc26ffa0f80 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff9fc280c5642c RDI: 7479625f6e695f65 RBP: 7479625f6e695f65 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9fc280aa9be0 R11: 0000000000000007 R12: ffff9fc2820b8a00 R13: 7479625f6e69601d R14: ffffffff8fa587d4 R15: 7479625f6e695f65 FS: 0000000000000000(0000) GS:ffff9fc282080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00005555f02989f8 CR3: 0000000053816005 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? die+0x3a/0x6f ? do_trap+0x118/0x150 ? do_error_trap+0x8b/0x130 ? do_stack_segment+0x29/0x40 ? stack_segment+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:59 (1779419459) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:11:00 (1779419460) === Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 5 previous similar messages Lustre: Skipped 2 previous similar messages LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: server umount lustre-MDT0000 complete LustreError: 6572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419472 with bad export cookie 9501014833320592611 LustreError: MGC192.168.204.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 6572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages LustreError: 6589:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message LustreError: 6589:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 1 previous similar message Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg422-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 280 sec | stack segment: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 0 PID: 18118 Comm: kworker/0:4 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1166750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 c5 a1 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffbc7401c0fda8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff99f042c51900 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff99f14512342c RDI: 657361656c657264 RBP: 657361656c657264 R08: 0000000000000000 R09: 0000000000000000 R10: ffff99f1714b1260 R11: 0000000000000007 R12: ffff99f182038a00 R13: 657361656c65731c R14: ffffffffa1c587d4 R15: 657361656c657264 FS: 0000000000000000(0000) GS:ffff99f182000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000557f0457e9f8 CR3: 000000009c616001 CR4: 0000000000170ef0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? die+0x3a/0x6f ? do_trap+0x118/0x150 ? do_error_trap+0x8b/0x130 ? do_stack_segment+0x29/0x40 ? stack_segment+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:27 (1779419427) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:29 (1779419429) === LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 6596:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 6596:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 11 previous similar messages LustreError: 10374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419438 with bad export cookie 8830694585469422894 LustreError: MGC192.168.201.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 10374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg156-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |
| replay-single test complete, duration 263 sec | stack segment: 0000 [#1] SMP DEBUG_PAGEALLOC CPU: 1 PID: 19536 Comm: kworker/1:5 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Workqueue: events 0xffffffffc1170750 RIP: 0010:debugfs_remove_recursive+0x45/0x260 Code: 41 57 49 89 ff 41 56 49 c7 c6 d4 87 65 9a 41 55 4c 8d af b8 00 00 00 41 54 55 48 89 fd 53 48 83 ec 08 48 83 05 eb 9a 17 03 01 <48> 8b 45 30 4c 8d 65 58 48 8d b8 c0 00 00 00 e8 c7 3e 8f 00 48 83 RSP: 0018:ffffb308c0d87da8 EFLAGS: 00010202 RAX: 0000000000000001 RBX: ffff89536b448280 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffff89536fb4902c RDI: 7479625f6e695f65 RBP: 7479625f6e695f65 R08: 0000000000000000 R09: 0000000000000000 R10: ffff895344a728a0 R11: 0000000000000007 R12: ffff8953820b8a00 R13: 7479625f6e69601d R14: ffffffff9a6587d4 R15: 7479625f6e695f65 FS: 0000000000000000(0000) GS:ffff895382080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055992294a9f8 CR3: 000000001ec16001 CR4: 0000000000170ee0 Call Trace: ? show_regs.cold.9+0x22/0x2f ? __die_body+0x22/0x90 ? __die+0x33/0x4a ? die+0x3a/0x6f ? do_trap+0x118/0x150 ? do_error_trap+0x8b/0x130 ? do_stack_segment+0x29/0x40 ? stack_segment+0x1e/0x30 ? debugfs_remove_recursive+0x45/0x260 ? _cond_resched+0x1d/0x50 ? mutex_lock+0x31/0x70 lprocfs_nodemap_remove+0x1a/0x160 [ptlrpc] nodemap_destroy+0x45/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] nodemap_destroy+0x298/0x700 [ptlrpc] nodemap_putref+0xd3/0x130 [ptlrpc] ? process_one_work+0x2c8/0x700 ? worker_thread+0x296/0x6e0 ? rescuer_thread+0x570/0x570 ? kthread+0x1d1/0x200 ? set_kthread_struct+0x70/0x70 ? ret_from_fork+0x1f/0x30 Modules linked in: zfs(O) spl(O) ldiskfs(O) lquota(O) ec(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul ata_generic crc32_pclmul crc32c_intel ata_piix ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: ptlrpc_gss] | Lustre: DEBUG MARKER: === replay-single: start cleanup 23:10:29 (1779419429) === Lustre: DEBUG MARKER: === replay-single: finish cleanup 23:10:32 (1779419432) === Lustre: server umount lustre-MDT0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 LustreError: 8174:0:(ldlm_lib.c:1182:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 8174:0:(ldlm_lib.c:1182:target_handle_connect()) Skipped 8 previous similar messages Lustre: Skipped 1 previous similar message LustreError: 8488:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779419441 with bad export cookie 7498032259750880567 LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 8488:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0001 complete Lustre: server umount lustre-OST0000 complete Lustre: 3710:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779419442/real 1779419442] req@ffff895242ff5c00 x1865856014622464/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779419458 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0001 complete Lustre: DEBUG MARKER: oleg334-server.virtnet: executing unload_modules_local Key type lgssc unregistered | Link to test |