| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| conf-sanity test 151a: damaged local config doesn't prevent mounting | LustreError: 1723833:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 1723833:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 0 UID: 0 PID: 1723833 Comm: osp-pre-0-3 Kdump: loaded Tainted: G OE ------- --- 6.12.0-55.37.1_lustre.el10.x86_64 #1 Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: MGC10.240.25.242@tcp: Connection to MGS (at 10.240.25.242@tcp) was lost; in progress operations using this service will fail LustreError: Skipped 4 previous similar messages Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) Lustre: Skipped 26 previous similar messages LDISKFS-fs (dm-3): unmounting filesystem e59003d4-8b15-4ae4-988c-7ec3a45bc2f9. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 LDISKFS-fs (dm-4): unmounting filesystem 3d159987-eab5-4f37-b9d3-3b8bb04c1154. Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; LNet: 2 peer NIs in recovery (showing 2): 10.240.25.234@tcp, 10.240.25.233@tcp Autotest: Test running for 655 minutes (lustre-reviews_full-dne-part-3_117374.1005) Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem e59003d4-8b15-4ae4-988c-7ec3a45bc2f9 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem 3d159987-eab5-4f37-b9d3-3b8bb04c1154 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid LNet: 1 peer NIs in recovery (showing 1): 10.240.25.233@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-70vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-70vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-70vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-3): unmounting filesystem e59003d4-8b15-4ae4-988c-7ec3a45bc2f9. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 858392:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 858392:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 1 UID: 0 PID: 858392 Comm: osp-pre-0-2 Kdump: loaded Tainted: P OE ------- --- 6.12.0-55.37.1_lustre.el10.x86_64 #1 Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds1' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt1 >/dev/null 2>&1 || IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:7b:d8:2a detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:7b:d8:2a detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:70:8f:16 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:5d:37:f6 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:5d:37:f6 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:a2:24:f7 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds3' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt3 >/dev/null 2>&1 || Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds1 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt1/mdt1 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:40:90:28 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:40:90:28 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:f3:0b:b7 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:1e:20:56 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ce:a8:37 detected! Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds1; mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 LustreError: 848316:0:(ldlm_lib.c:1176:target_handle_connect()) lustre-MDT0002: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 848316:0:(ldlm_lib.c:1176:target_handle_connect()) Skipped 130 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt1/mdt1 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt1/mdt1 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds3 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:40:90:28 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:71:b1:99 detected! Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:4d:a5:9e detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:1e:20:56 detected! Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt3/mdt3 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds3; mount -t lustre -o localrecov lustre-mdt3/mdt3 /mnt/lustre-mds3 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt3/mdt3 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt3/mdt3 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:dc:b1 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:70:8f:16 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:4d:a5:9e detected! LNet: 1 peer NIs in recovery (showing 1): 10.240.22.180@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:71:b1:99 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:7b:d8:2a detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:b1:c9:77 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:66:71:6c detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:b1:c9:77 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ce:a8:37 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:5d:37:f6 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:dc:b1 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:f3:0b:b7 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:23:2c:92 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:9d:f7:86 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:a1:d3:2c detected! IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:5f:09:20 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:dc:b1 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:f3:0b:b7 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:23:2c:92 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:9d:f7:86 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:a1:d3:2c detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds1' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds1 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:5f:09:20 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:dc:b1 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:f3:0b:b7 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:23:2c:92 detected! Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:9d:f7:86 detected! Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt1 >/dev/null 2>&1 || IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:a1:d3:2c detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds3' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds3 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:5f:09:20 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:5d:37:f6 detected! LNet: 1 peer NIs in recovery (showing 1): 10.240.22.181@tcp IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:71:b1:99 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:4d:a5:9e detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:71:b1:99 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:40:90:28 detected! LustreError: 854881:0:(obd_class.h:478:obd_check_dev()) Device 19 not setup LustreError: 854881:0:(obd_class.h:478:obd_check_dev()) Skipped 119 previous similar messages Lustre: server umount lustre-MDT0002 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt3 >/dev/null 2>&1 || IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:b1:c9:77 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:a1:d3:2c detected! Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: lsmod | grep -q libcfs Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: /usr/sbin/lctl set_param fail_loc=0x80000716 Lustre: DEBUG MARKER: mount -t lustre lustre-mdt1/mdt1 /mnt/lustre-mds1 -o localrecov Lustre: DEBUG MARKER: mount -t lustre lustre-mdt1/mdt1 /mnt/lustre-mds1 -o localrecov LustreError: 857403:0:(tgt_mount.c:2435:server_fill_super()) cfs_race id 716 sleeping LustreError: 857404:0:(tgt_mount.c:2435:server_fill_super()) cfs_fail_race id 716 waking LustreError: 857403:0:(tgt_mount.c:2435:server_fill_super()) cfs_fail_race id 716 awake: rc=4996 LustreError: 857403:0:(tgt_mount.c:2446:server_fill_super()) Unable to start osd on lustre-mdt1/mdt1: -114 LustreError: 857403:0:(super25.c:198:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -114 Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 10 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl set_param fail_loc=0x0 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:70:8f:16 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:4d:a5:9e detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:b1:c9:77 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds3 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt3/mdt3 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds3; mount -t lustre -o localrecov lustre-mdt3/mdt3 /mnt/lustre-mds3 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt3/mdt3 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt3/mdt3 2>/dev/null IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:40:90:28 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:4d:a5:9e detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:1e:20:56 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:66:71:6c detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:40:90:28 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:4d:a5:9e detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-138vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds1' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds1 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:7b:d8:2a detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:40:90:28 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:4d:a5:9e detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:23:2c:92 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:1e:20:56 detected! Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt1 >/dev/null 2>&1 || LustreError: 857409:0:(ldlm_lockd.c:2559:ldlm_cancel_handler()) ldlm_cancel from 10.240.22.190@tcp arrived at 1760374268 with bad export cookie 10123350714329343988 LustreError: 857409:0:(ldlm_lockd.c:2559:ldlm_cancel_handler()) Skipped 16 previous similar messages IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:a2:24:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:71:b1:99 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:5d:37:f6 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:42:85:b2 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds3' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds3 LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: Skipped 5 previous similar messages | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 502513:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 502513:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 0 UID: 0 PID: 502513 Comm: osp-pre-0-3 Kdump: loaded Tainted: P OE ------- --- 6.12.0-55.37.1_lustre.el10.x86_64 #1 Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] ? __schedule+0x261/0x640 osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt4 >/dev/null 2>&1 || IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:70:d2:ad detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:6e:84:1b detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ce:a8:37 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 LustreError: 494140:0:(ldlm_lib.c:1176:target_handle_connect()) lustre-MDT0003: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 494140:0:(ldlm_lib.c:1176:target_handle_connect()) Skipped 64 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:b0:a4:14 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:c9:db:05 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ef:b9:9d detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov lustre-mdt4/mdt4 /mnt/lustre-mds4 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:96:0d:d7 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:67:f7:87 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:c2:03:ed detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:6e:84:1b detected! Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:8e:de:c8 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:ef:b9:9d detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:96:0d:d7 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:be:0e:4b detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:90:ae:8f detected! LNet: 1 peer NIs in recovery (showing 1): 10.240.22.189@tcp IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:c9:db:05 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:70:d2:ad detected! Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ef:b9:9d detected! Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6f:67:58 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:67:f7:87 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:94:6d:09 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6e:84:1b detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:df:bc:dd detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:62:83:ca detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ef:b9:9d detected! Autotest: Test running for 285 minutes (lustre-reviews_full-dne-zfs-part-3_117330.1008) IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:70:d2:ad detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:be:0e:4b detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:be:0e:4b detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:cc:8b:05 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:6e:84:1b detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:9d:e7:af detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6f:26:66 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:67:f7:87 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:6f:26:66 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:67:f7:87 detected! Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:8e:de:c8 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:96:0d:d7 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:94:6d:09 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:96:0d:d7 detected! Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt4 >/dev/null 2>&1 || Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-138vm3.onyx.whamcloud.com: executing load_modules_local IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:ab:0b:1d detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:70:d2:ad detected! LNet: 1 peer NIs in recovery (showing 1): 10.240.22.190@tcp IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:b0:a4:14 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:ab:0b:1d detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:90:ae:8f detected! Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:ef:b9:9d detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:70:d2:ad detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:ab:0b:1d detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:90:ae:8f detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov lustre-mdt4/mdt4 /mnt/lustre-mds4 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-138vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:cc:8b:05 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:38:ce:05 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:ab:0b:1d detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:df:bc:dd detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:8e:de:c8 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-138vm3.onyx.whamcloud.com: executing lsmod IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6f:67:58 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:6f:67:58 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:c9:db:05 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: 500837:0:(client.c:1369:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff8e59b9d24340 x1845650757471104/t0(0) o900->lustre-MDT0000-lwp-MDT0001@10.240.22.189@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6a:9b:e8 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:df:bc:dd detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:67:f7:87 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:df:bc:dd detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:c2:03:ed detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:8e:de:c8 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 544788:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 544788:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 0 UID: 0 PID: 544788 Comm: osp-pre-0-3 Kdump: loaded Tainted: G B OE ------- --- 6.12.0-55.37.1_lustre.el10.x86_64 #1 Tainted: [B]=BAD_PAGE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] ? srso_alias_return_thunk+0x5/0xfbef5 osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xcf/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x31/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6e:54:f5 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:6e:84:1b detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:62:83:ca detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:08:a7:b2 detected! Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:08:a7:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:38:ce:05 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:df:bc:dd detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:6e:84:1b detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm62.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm62.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem 9b486186-eac3-43cb-bf75-4d5abdf0e01d r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:c9:db:05 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:b0:a4:14 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:c9:db:05 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:e7:e8:9e detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm62.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm62.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem b07915e5-8081-4589-801b-19e3625d8d04 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:2d:b7:94 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:f3:b5:c2 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:90:ae:8f detected! Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:ab:0b:1d detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:6e:54:f5 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:ab:0b:1d detected! Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:6f:26:66 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:cc:8b:05 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:3a:12:2f detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:c9:db:05 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm61.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm61.onyx.whamcloud.com: executing set_default_debug -1 all LNet: 1 peer NIs in recovery (showing 1): 10.240.31.65@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-155vm59.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-155vm60.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:f3:0b:b7 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:3a:12:2f detected! IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:1c:b7:bd detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:bd:ee:d0 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:6e:54:f5 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:6f:26:66 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:f3:b5:c2 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:c9:db:05 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: MGC10.240.31.66@tcp: Connection to MGS (at 10.240.31.66@tcp) was lost; in progress operations using this service will fail LustreError: Skipped 5 previous similar messages IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6a:9b:e8 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:2d:b7:94 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:6f:26:66 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:6e:54:f5 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:c2:03:ed detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:5a:08:f7 detected! IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:1c:b7:bd detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:f3:0b:b7 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:6f:67:58 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:9d:e7:af detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:08:a7:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:6f:67:58 detected! Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-3): unmounting filesystem 9b486186-eac3-43cb-bf75-4d5abdf0e01d. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:42:85:b2 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:cc:8b:05 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:df:bc:dd detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:70:d2:ad detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:42:85:b2 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 LDISKFS-fs (dm-4): unmounting filesystem b07915e5-8081-4589-801b-19e3625d8d04. Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; LNet: 1 peer NIs in recovery (showing 1): 10.240.31.63@tcp Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:ce:a8:37 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:6f:67:58 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:e7:e8:9e detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:b0:a4:14 detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm62.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm61.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm60.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-155vm62.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-155vm61.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-155vm60.onyx.whamcloud.com: executing load_modules_local Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem 9b486186-eac3-43cb-bf75-4d5abdf0e01d r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:07:8a:4b detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:e1:ca:d5 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:c2:03:ed detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:70:d2:ad detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:1c:b7:bd detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:90:ae:8f detected! Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm62.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm62.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:ab:0b:1d detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:1c:b7:bd detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:90:ae:8f detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:cc:8b:05 detected! Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem b07915e5-8081-4589-801b-19e3625d8d04 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:5a:08:f7 detected! Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-155vm63.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-155vm61.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-155vm61.onyx.whamcloud.com: executing lsmod IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:6a:9b:e8 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:b0:a4:14 detected! IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:0a:ea:38 detected! Autotest: Test running for 350 minutes (lustre-reviews_full-dne-part-3_117330.1005) Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: 542843:0:(client.c:1369:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ff42115477aec340 x1845636306385280/t0(0) o900->lustre-MDT0000-lwp-MDT0001@10.240.31.66@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 IPv6: eth0: IPv6 duplicate address fe80::a54:2e6:bd0d:d0c5 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::8baa:a02:b1f6:f72d used by 52:54:00:0a:ea:38 detected! LDISKFS-fs (dm-3): unmounting filesystem 9b486186-eac3-43cb-bf75-4d5abdf0e01d. Lustre: server umount lustre-MDT0001 complete IPv6: eth0: IPv6 duplicate address fe80::194a:e2fd:b72e:b661 used by 52:54:00:0a:ea:38 detected! Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && LustreError: 542820:0:(ldlm_lib.c:1176:target_handle_connect()) lustre-MDT0001: not available for connect from 10.240.31.65@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 542820:0:(ldlm_lib.c:1176:target_handle_connect()) Skipped 50 previous similar messages IPv6: eth0: IPv6 duplicate address fe80::2d8b:44d9:a914:a976 used by 52:54:00:0a:ea:38 detected! IPv6: eth0: IPv6 duplicate address fe80::201:7204:228b:bddd used by 52:54:00:62:83:ca detected! Lustre: DEBUG MARKER: modprobe dm-flakey; IPv6: eth0: IPv6 duplicate address fe80::5a2a:afa3:69c:8b6a used by 52:54:00:5a:08:f7 detected! Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 ================================================================== BUG: KFENCE: use-after-free read in __ptlrpc_req_put.isra.0+0x4d/0x2d0 [ptlrpc] Use-after-free read at 0x000000009751a903 (in kfence-#140): __ptlrpc_req_put.isra.0+0x4d/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xcf/0x100 ret_from_fork+0x31/0x50 ret_from_fork_asm+0x1a/0x30 kfence-#140: 0x000000001ce84918-0x00000000a2a7fb62, size=816, cache=ptlrpc_cache allocated by task 544788 on cpu 0 at 21093.328541s (1.573892s ago): ptlrpc_request_cache_alloc+0x2c/0x110 [ptlrpc] __ptlrpc_request_alloc+0x16/0x3b0 [ptlrpc] ptlrpc_request_alloc_internal+0x16/0xe0 [ptlrpc] ptlrpc_request_alloc_pack+0x23/0x80 [ptlrpc] fld_client_rpc+0x1c5/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xcf/0x100 ret_from_fork+0x31/0x50 ret_from_fork_asm+0x1a/0x30 freed by task 544788 on cpu 0 at 21093.328644s (1.574654s ago): __ptlrpc_free_req+0x341/0x550 [ptlrpc] __ptlrpc_req_put.isra.0+0xb2/0x2d0 [ptlrpc] fld_client_rpc+0xb7/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xcf/0x100 ret_from_fork+0x31/0x50 ret_from_fork_asm+0x1a/0x30 CPU: 0 UID: 0 PID: 544788 Comm: osp-pre-0-3 Kdump: loaded Tainted: G OE ------- --- 6.12.0-55.37.1_lustre.el10.x86_64 #1 Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014 ================================================================== | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 502531:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 502531:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 1 UID: 0 PID: 502531 Comm: osp-pre-0-3 Kdump: loaded Tainted: P B OE ------- --- 6.12.0-55.34.1_lustre.el10.x86_64 #1 Tainted: [P]=PROPRIETARY_MODULE, [B]=BAD_PAGE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true LNet: 2 peer NIs in recovery (showing 2): 10.240.27.60@tcp, 10.240.27.69@tcp Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt4 >/dev/null 2>&1 || Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 LustreError: 494212:0:(ldlm_lib.c:1176:target_handle_connect()) lustre-MDT0003: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 494212:0:(ldlm_lib.c:1176:target_handle_connect()) Skipped 70 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov lustre-mdt4/mdt4 /mnt/lustre-mds4 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LNet: 1 peer NIs in recovery (showing 1): 10.240.27.62@tcp Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt4 >/dev/null 2>&1 || Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm3.onyx.whamcloud.com: executing load_modules_local Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov lustre-mdt4/mdt4 /mnt/lustre-mds4 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-150vm3.onyx.whamcloud.com: executing lsmod LNet: 1 peer NIs in recovery (showing 1): 10.240.27.62@tcp Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: 500855:0:(client.c:1369:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff9697b69bc680 x1844771658588416/t0(0) o900->lustre-MDT0000-lwp-MDT0001@10.240.27.69@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 ================================================================== BUG: KFENCE: use-after-free read in __ptlrpc_req_put.isra.0+0x4d/0x2d0 [ptlrpc] Use-after-free read at 0x0000000055f53f48 (in kfence-#35): __ptlrpc_req_put.isra.0+0x4d/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xd2/0x100 ret_from_fork+0x34/0x50 ret_from_fork_asm+0x1a/0x30 kfence-#35: 0x00000000493f9c17-0x00000000d0d497b8, size=816, cache=ptlrpc_cache allocated by task 502531 on cpu 1 at 18024.513057s (1.568933s ago): ptlrpc_request_cache_alloc+0x2c/0x110 [ptlrpc] __ptlrpc_request_alloc+0x16/0x3b0 [ptlrpc] ptlrpc_request_alloc_internal+0x16/0xe0 [ptlrpc] ptlrpc_request_alloc_pack+0x23/0x80 [ptlrpc] fld_client_rpc+0x1c5/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xd2/0x100 ret_from_fork+0x34/0x50 ret_from_fork_asm+0x1a/0x30 freed by task 502531 on cpu 1 at 18024.513159s (1.569973s ago): __ptlrpc_free_req+0x341/0x550 [ptlrpc] __ptlrpc_req_put.isra.0+0xb2/0x2d0 [ptlrpc] fld_client_rpc+0xb7/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xd2/0x100 ret_from_fork+0x34/0x50 ret_from_fork_asm+0x1a/0x30 CPU: 1 UID: 0 PID: 502531 Comm: osp-pre-0-3 Kdump: loaded Tainted: P OE ------- --- 6.12.0-55.34.1_lustre.el10.x86_64 #1 Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 ================================================================== | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 513626:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 513626:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 1 UID: 0 PID: 513626 Comm: osp-pre-0-3 Kdump: loaded Tainted: G OE ------- --- 6.12.0-55.34.1_lustre.el10.x86_64 #1 Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem 5637bddb-385a-48af-8b48-d7f9480682f1 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem 4655ccd5-cd7a-4f2f-bae0-54243dc4d78c r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid LNet: 2 peer NIs in recovery (showing 2): 10.240.26.200@tcp, 10.240.26.199@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm5.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm5.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-81vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: MGC10.240.26.207@tcp: Connection to MGS (at 10.240.26.207@tcp) was lost; in progress operations using this service will fail LustreError: Skipped 5 previous similar messages LDISKFS-fs (dm-3): unmounting filesystem 5637bddb-385a-48af-8b48-d7f9480682f1. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; LustreError: 507451:0:(ldlm_lib.c:1176:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 507451:0:(ldlm_lib.c:1176:target_handle_connect()) Skipped 47 previous similar messages Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 LDISKFS-fs (dm-4): unmounting filesystem 4655ccd5-cd7a-4f2f-bae0-54243dc4d78c. Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; LNet: 2 peer NIs in recovery (showing 2): 10.240.26.200@tcp, 10.240.26.199@tcp Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm12.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm5.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm4.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-81vm12.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-81vm5.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-81vm4.onyx.whamcloud.com: executing load_modules_local Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem 5637bddb-385a-48af-8b48-d7f9480682f1 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm12.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem 4655ccd5-cd7a-4f2f-bae0-54243dc4d78c r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-81vm13.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-81vm5.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-81vm5.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: 511681:0:(client.c:1369:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff989d2aa5f740 x1844765972349184/t0(0) o900->lustre-MDT0000-lwp-MDT0001@10.240.26.207@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 LustreError: 511681:0:(client.c:1369:ptlrpc_import_delay_req()) Skipped 3 previous similar messages Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-3): unmounting filesystem 5637bddb-385a-48af-8b48-d7f9480682f1. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; LNet: 3 peer NIs in recovery (showing 3): 10.240.26.200@tcp, 10.240.26.199@tcp, 10.240.26.207@tcp Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 502338:0:(client.c:2837:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 502338:0:(client.c:2837:__ptlrpc_req_put()) LBUG CPU: 0 UID: 0 PID: 502338 Comm: osp-pre-0-3 Kdump: loaded Tainted: P OE ------- --- 6.12.0-55.34.1_lustre.el10.x86_64 #1 Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt4 >/dev/null 2>&1 || Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov lustre-mdt4/mdt4 /mnt/lustre-mds4 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-77vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-77vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Autotest: Test running for 280 minutes (lustre-reviews_full-dne-zfs-part-3_117076.1008) Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt4 >/dev/null 2>&1 || Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-77vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-77vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-77vm3.onyx.whamcloud.com: executing load_modules_local Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt2/mdt2 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: lsmod | grep zfs >&/dev/null || modprobe zfs; Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov lustre-mdt4/mdt4 /mnt/lustre-mds4 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-77vm14.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname lustre-mdt4/mdt4 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-77vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-77vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: 500662:0:(client.c:1369:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff8a4133d66700 x1844751689782016/t0(0) o900->lustre-MDT0000-lwp-MDT0001@10.240.26.126@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: ! zpool list -H lustre-mdt2 >/dev/null 2>&1 || Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 | Link to test |
| conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one | LustreError: 513970:0:(client.c:2833:__ptlrpc_req_put()) ASSERTION( request->rq_pill.rc_reqmsg != ((void *)0x5a5a5a5a5a5a5a5aL) ) failed: LustreError: 513970:0:(client.c:2833:__ptlrpc_req_put()) LBUG CPU: 0 UID: 0 PID: 513970 Comm: osp-pre-0-3 Kdump: loaded Tainted: G B OE ------- --- 6.12.0-55.33.1_lustre.el10.x86_64 #1 Tainted: [B]=BAD_PAGE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x42 [libcfs] __ptlrpc_req_put.isra.0+0x2ca/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] ? __schedule+0x261/0x640 ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_osp_precreate_thread+0x10/0x10 [osp] kthread+0xd2/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> | Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem 4b9ecd7c-89ed-4563-9a12-58ddf53a6503 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null LNet: 1 peer NIs in recovery (showing 1): 10.240.27.62@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem 6ce859de-5b8e-4331-8095-871a74f8bcff r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-\*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount FULL mdc.lustre-MDT0003-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm3.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: onyx-150vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LustreError: MGC10.240.27.69@tcp: Connection to MGS (at 10.240.27.69@tcp) was lost; in progress operations using this service will fail LustreError: Skipped 5 previous similar messages LDISKFS-fs (dm-3): unmounting filesystem 4b9ecd7c-89ed-4563-9a12-58ddf53a6503. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; LNet: 1 peer NIs in recovery (showing 1): 10.240.27.61@tcp LustreError: 504799:0:(ldlm_lib.c:1129:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: 504799:0:(ldlm_lib.c:1129:target_handle_connect()) Skipped 44 previous similar messages Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 LDISKFS-fs (dm-4): unmounting filesystem 6ce859de-5b8e-4331-8095-871a74f8bcff. Lustre: server umount lustre-MDT0003 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm2.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm3.onyx.whamcloud.com: executing load_modules_local Lustre: DEBUG MARKER: onyx-150vm2.onyx.whamcloud.com: executing load_modules_local Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds2_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds2; mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 LDISKFS-fs (dm-3): mounted filesystem 4b9ecd7c-89ed-4563-9a12-58ddf53a6503 r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds2_flakey 2>/dev/null Autotest: Test running for 335 minutes (lustre-reviews_full-dne-part-3_116860.1005) Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm10.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4 Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey >/dev/null 2>&1 Lustre: DEBUG MARKER: dmsetup status /dev/mapper/mds4_flakey 2>&1 Lustre: DEBUG MARKER: test -b /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-mds4; mount -t lustre -o localrecov /dev/mapper/mds4_flakey /mnt/lustre-mds4 LDISKFS-fs (dm-4): mounted filesystem 6ce859de-5b8e-4331-8095-871a74f8bcff r/w with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: onyx-150vm11.onyx.whamcloud.com: executing set_default_debug -1 all Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' Lustre: DEBUG MARKER: e2label /dev/mapper/mds4_flakey 2>/dev/null Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-150vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: onyx-150vm3.onyx.whamcloud.com: executing lsmod Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds2' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds2 LNet: 3 peer NIs in recovery (showing 3): 10.240.27.61@tcp, 10.240.27.69@tcp, 10.240.27.62@tcp LustreError: 511973:0:(client.c:1369:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff9bda73016700 x1844105356758016/t0(0) o900->lustre-MDT0000-lwp-MDT0001@10.240.27.69@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 LDISKFS-fs (dm-3): unmounting filesystem 4b9ecd7c-89ed-4563-9a12-58ddf53a6503. Lustre: server umount lustre-MDT0001 complete Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds4' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d -f /mnt/lustre-mds4 ================================================================== BUG: KFENCE: use-after-free read in __ptlrpc_req_put.isra.0+0x4d/0x2d0 [ptlrpc] Use-after-free read at 0x000000008e01d4c7 (in kfence-#86): __ptlrpc_req_put.isra.0+0x4d/0x2d0 [ptlrpc] fld_client_rpc+0x33d/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xd2/0x100 ret_from_fork+0x34/0x50 ret_from_fork_asm+0x1a/0x30 kfence-#86: 0x0000000067b1b6aa-0x0000000096b6ee99, size=816, cache=ptlrpc_cache allocated by task 513970 on cpu 0 at 20192.294247s (1.769031s ago): ptlrpc_request_cache_alloc+0x2c/0x110 [ptlrpc] __ptlrpc_request_alloc+0x16/0x3b0 [ptlrpc] ptlrpc_request_alloc_internal+0x16/0xe0 [ptlrpc] ptlrpc_request_alloc_pack+0x23/0x80 [ptlrpc] fld_client_rpc+0x1c5/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xd2/0x100 ret_from_fork+0x34/0x50 ret_from_fork_asm+0x1a/0x30 freed by task 513970 on cpu 0 at 20192.294329s (1.770093s ago): __ptlrpc_free_req+0x341/0x550 [ptlrpc] __ptlrpc_req_put.isra.0+0xb2/0x2d0 [ptlrpc] fld_client_rpc+0xb7/0x680 [fld] fld_server_lookup+0x217/0x390 [fld] osp_precreate_send+0x88a/0xe80 [osp] osp_precreate_thread+0x5a6/0x1260 [osp] kthread+0xd2/0x100 ret_from_fork+0x34/0x50 ret_from_fork_asm+0x1a/0x30 CPU: 0 UID: 0 PID: 513970 Comm: osp-pre-0-3 Kdump: loaded Tainted: G OE ------- --- 6.12.0-55.33.1_lustre.el10.x86_64 #1 Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 ================================================================== | Link to test |