| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed7ea2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 5752 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff88012a2db330 ti: ffff88012e968000 task.ti: ffff88012e968000 RIP: 0010:[<ffffffffa0ed7ea2>] [<ffffffffa0ed7ea2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88012e96b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88012e96b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88012efa2e90 R13: 0000000000000000 R14: ffff8800933c5000 R15: ffff88009ed8cb40 FS: 0000000000000000(0000) GS:ffff88013e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0efd050>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efd42b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d83760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d8f431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dec53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df2a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de9b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df20f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0defff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05fb782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa019de35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01a1066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0625be6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a9ed4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06b23ba>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e28fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa065e3e0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0660099>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa065f4c0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 31579:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1700735379 with bad export cookie 2016394085790962610 LustreError: 166-1: MGC192.168.203.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31579:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg346-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32452:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.146@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg346-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_RC1 LNet: Added LNI 192.168.203.146@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa134edf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 20906 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff880131b51110 ti: ffff88008c304000 task.ti: ffff88008c304000 RIP: 0010:[<ffffffffa134edf2>] [<ffffffffa134edf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008c307848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008c307890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8801348de890 R13: 0000000000000000 R14: ffff880091466000 R15: ffff8800a0e390c0 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001368fa000 CR4: 00000000000006e0 Call Trace: [<ffffffffa1373e80>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137425b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e1730>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ed401>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1269a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1260b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12690f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1266ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05c9782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01c0e35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01c4066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05f3be6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0677ed4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06803ba>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02008fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa062c3e0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa062e099>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa062d4c0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 16150:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438563/real 1699438563] req@ffff880093249900 x1781990429829696/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438569 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 16806:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 20895:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438600/real 1699438600] req@ffff88009e15a300 x1781990507418304/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438606 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21545:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 25390:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438632/real 1699438632] req@ffff8800a1771e00 x1781990545166720/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438638 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 26051:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 31355:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438672/real 1699438672] req@ffff88009e158000 x1781990580818880/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438678 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32033:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 3848:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438708/real 1699438708] req@ffff88009b874b00 x1781990620664704/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438714 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 4510:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 8581:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438740/real 1699438740] req@ffff88009e15f800 x1781990659461504/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438746 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 9255:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa130cdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 1 PID: 19007 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800ba6fd550 ti: ffff8800bfc44000 task.ti: ffff8800bfc44000 RIP: 0010:[<ffffffffa130cdf2>] [<ffffffffa130cdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800bfc47848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800bfc47890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b5129890 R13: 0000000000000000 R14: ffff8800aa88f000 R15: ffff8800ab4539c0 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000006e0 Call Trace: [<ffffffffa1331e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa133222b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ba6f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11c63c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa122353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1229a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1220acd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12290f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1226ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05b0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0171e35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0175066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05da986>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa065ec74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa066715a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01c688e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0613180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0614e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0614260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 15700:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425490/real 1687425490] req@ffff8800bb88d580 x1769393806649984/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425496 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 16354:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 20426:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425527/real 1687425527] req@ffff8800aec27200 x1769393886335872/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425533 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21064:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 mount.lustre (23578) used greatest stack depth: 9448 bytes left Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 24878:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425560/real 1687425560] req@ffff8800aec276c0 x1769393926181312/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425566 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 25540:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 30824:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425601/real 1687425601] req@ffff8800bd5b3900 x1769393962881984/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425607 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 31500:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 3223:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425639/real 1687425639] req@ffff8800aec24280 x1769394003776384/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425645 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 3878:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 7811:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425672/real 1687425672] req@ffff8800bd5b6880 x1769394043621824/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425678 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 8477:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed5df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 1 PID: 6893 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff880115e02220 ti: ffff880084268000 task.ti: ffff880084268000 RIP: 0010:[<ffffffffa0ed5df2>] [<ffffffffa0ed5df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008426b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008426b890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800ad2f2a90 R13: 0000000000000000 R14: ffff8800c970f000 R15: ffff8800b7948240 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000011738c000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0efae50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efb22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d836f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d8f3c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dec53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df2a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de9acd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df20f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0defff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05e2782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01dbe35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01df066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa060c986>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0690c74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069915a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa021b88e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0645180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0646e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0646260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 28180:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1686614353 with bad export cookie 3854249869916708303 LustreError: 166-1: MGC192.168.203.171@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28180:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg371-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32592:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.171@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg371-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_RC1_1_gdef45fb LNet: Added LNI 192.168.203.171@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa131fdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 14061 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800c87ae660 ti: ffff8800b97b0000 task.ti: ffff8800b97b0000 RIP: 0010:[<ffffffffa131fdf2>] [<ffffffffa131fdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b97b3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b97b3890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bf952a90 R13: 0000000000000000 R14: ffff88010e3ee000 R15: ffff88010bd50480 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d384e000 CR4: 00000000000006f0 Call Trace: [<ffffffffa1344e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa134522b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11cd6b0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11d9381>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa123653f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa123ca00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1233acd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa123c0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1239ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05c1782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01bae35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01be066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05eb986>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa066fc74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa067815a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01fa88e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0624180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0625e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0625260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 14595:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989449/real 1685989449] req@ffff8800b15597c0 x1767888014813248/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1685989455 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 14595:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989455/real 1685989455] req@ffff8800b89e63c0 x1767888014813568/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989461 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 15252:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 19305:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989492/real 1685989492] req@ffff8800aa03b900 x1767888095548096/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989498 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 19947:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 23780:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989524/real 1685989524] req@ffff8800b155d580 x1767888134345088/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989530 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 24434:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 29674:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989561/real 1685989561] req@ffff8800b155f200 x1767888168948416/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1685989567 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 29674:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989567/real 1685989567] req@ffff8800b85de880 x1767888168948736/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989573 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 30335:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 2063:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989605/real 1685989605] req@ffff8800b89e1c80 x1767888212988800/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989611 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 2711:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 6706:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989638/real 1685989638] req@ffff8800b89e17c0 x1767888251785664/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989644 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 7364:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1352df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 1 PID: 11522 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800c6836660 ti: ffff8800b5244000 task.ti: ffff8800b5244000 RIP: 0010:[<ffffffffa1352df2>] [<ffffffffa1352df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b5247848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b5247890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b7dc3290 R13: 0000000000000000 R14: ffff88010ed4c000 R15: ffff8800ae686f00 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000006e0 Call Trace: [<ffffffffa1377e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137822b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e75f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f32c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126955f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126fa20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1266b5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126f110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126d011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05d0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01c9dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01cd006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05faa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa067f014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa068953a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa020982e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0633180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0634e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0634260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 11938:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766873/real 1668766873] req@ffff880117175f00 x1749828831816256/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1668766879 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 11938:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766879/real 1668766879] req@ffff8800b19ccc00 x1749828831816512/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766885 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 12586:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 16676:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766917/real 1668766917] req@ffff8800b19cd580 x1749828915696512/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766923 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 17304:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 21128:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766950/real 1668766950] req@ffff8800b19cf6c0 x1749828955541888/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766956 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21805:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 27068:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766989/real 1668766989] req@ffff8800b19cbdc0 x1749828991193984/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766995 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 27712:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 31880:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668767028/real 1668767028] req@ffff8800bfcf7200 x1749829031039872/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668767034 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32523:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 4049:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668767060/real 1668767060] req@ffff8800b19cb440 x1749829070885312/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668767066 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 4720:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed6df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 0 PID: 7786 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800a20c8000 ti: ffff8800a9270000 task.ti: ffff8800a9270000 RIP: 0010:[<ffffffffa0ed6df2>] [<ffffffffa0ed6df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800a9273848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800a9273890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bc953890 R13: 0000000000000000 R14: ffff8800982ec000 R15: ffff8800bc923480 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d34da000 CR4: 00000000000006f0 Call Trace: [<ffffffffa0efbe50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efc22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d845f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d902c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0ded55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df3a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0deab5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df3110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df1011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05fe782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01a7dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01ab006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0628a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06ad014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06b753a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e782e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0661180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0662e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0662260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 31559:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1668335394 with bad export cookie 5395720967403707116 LustreError: 166-1: MGC192.168.201.173@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31559:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg273-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 345:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.173@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg273-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.201.173@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 8144 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff88010b20e660 ti: ffff88009ac60000 task.ti: ffff88009ac60000 RIP: 0010:[<ffffffffa0ee8df2>] [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009ac63848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009ac63890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88009f4a9490 R13: 0000000000000000 R14: ffff8800c3ee8000 R15: ffff8800c59f2c00 FS: 0000000000000000(0000) GS:ffff88011e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d63fc000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0f0de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d965f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da22c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dff55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e05a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfcb5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e05110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e03011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01ecdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01f0006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061da06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a2014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ac53a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa022c82e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0656180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0657e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0657260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LustreError: 28742:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1668249073 with bad export cookie 1270199893924624021 LustreError: 166-1: MGC192.168.201.177@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28742:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: DEBUG MARKER: oleg277-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 711:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.177@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg277-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.201.177@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 5621 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800cf64a220 ti: ffff880083a2c000 task.ti: ffff880083a2c000 RIP: 0010:[<ffffffffa0ee8df2>] [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880083a2f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880083a2f890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800c4a01e90 R13: 0000000000000000 R14: ffff88009362c000 R15: ffff8800c4a70780 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d6fe4000 CR4: 00000000000006f0 Call Trace: [<ffffffffa0f0de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d965f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da22c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dff55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e05a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfcb5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e05110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e03011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01a9dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01ad006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061aa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa069f014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a953a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01f982e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0653180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0654e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0654260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 28794:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1668111697 with bad export cookie 12292860998847032263 LustreError: 166-1: MGC192.168.203.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28794:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg346-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 765:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.146@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg346-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_RC1_57_gad04e3f LNet: Added LNI 192.168.203.146@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 1 PID: 9329 Comm: mdt00_005 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800b59dc440 ti: ffff8800badd8000 task.ti: ffff8800badd8000 RIP: 0010:[<ffffffffa0edddf2>] [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800baddb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800baddb890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b4a78490 R13: 0000000000000000 R14: ffff8800b896b000 R15: ffff88008819f6c0 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d5952000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0f02e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0322b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d8b5f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d972c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0df455f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dfaa20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df1b5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0dfa110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df8011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05ea782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01e3dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01e7006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0614a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0699014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a353a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa022382e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa064d180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa064ee39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa064e260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 31077:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1666258324 with bad export cookie 464433880997223941 LustreError: 166-1: MGC192.168.201.164@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31077:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: DEBUG MARKER: oleg264-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 617:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.164@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg264-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_36_g9d1805c LNet: Added LNI 192.168.201.164@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 5556 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800cd014440 ti: ffff8800c9204000 task.ti: ffff8800c9204000 RIP: 0010:[<ffffffffa0ed8df2>] [<ffffffffa0ed8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800c9207848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800c9207890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800c0b4ae90 R13: 0000000000000000 R14: ffff8800c3efe000 R15: ffff8800adb6d540 FS: 0000000000000000(0000) GS:ffff88011e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001168e4000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0efde50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efe22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d86600>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d922d1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0def55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df5a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0decb5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df5110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df3011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05de782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa019fdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01a3006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0608a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa068d014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069753a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e782e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0641180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0642e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0642260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-MDT0000 complete LustreError: 31919:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1666053603 with bad export cookie 11869320381983379453 LustreError: 166-1: MGC192.168.203.172@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31919:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg372-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 712:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.172@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg372-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_RC1_40_g6e98a6e LNet: Added LNI 192.168.203.172@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 0 PID: 17588 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800becc0000 ti: ffff8800b13a0000 task.ti: ffff8800b13a0000 RIP: 0010:[<ffffffffa1335df2>] [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b13a3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b13a3890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a6fd9290 R13: 0000000000000000 R14: ffff8800b77d2000 R15: ffff8800bc80c320 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d56de000 CR4: 00000000000006f0 Call Trace: [<ffffffffa135ae60>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135b23b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11c7640>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11d35a5>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124b3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1252020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12489bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1251710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124f611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05c7782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01c2df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01c6096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05f1a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0675ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa067b02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa0202a5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa062a3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa062c059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa062b480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 21177:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179781/real 1655179781] req@ffff8800aa8eb900 x1735581722749824/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179787 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21746:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 25227:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179820/real 1655179820] req@ffff8800b2098000 x1735581811872448/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179826 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 25802:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 29040:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179855/real 1655179855] req@ffff8800a73c2140 x1735581852766656/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179861 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 29646:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 1980:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179902/real 1655179902] req@ffff8800b209df00 x1735581891564672/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179908 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 2569:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 6161:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179942/real 1655179942] req@ffff8800b209c740 x1735581938750336/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179948 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 6758:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 10157:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179977/real 1655179977] req@ffff8800b209ed40 x1735581980692928/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179983 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 10763:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1334f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw floppy virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 0 PID: 24332 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802e16e0010 ti: ffff8802f2784000 task.ti: ffff8802f2784000 RIP: 0010:[<ffffffffa1334f12>] [<ffffffffa1334f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802f2787848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802f2787890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88026ee8c978 R13: 0000000000000000 R14: ffff88029691b3f8 R15: ffff880091dd69e8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa1359ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135a3cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e0650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ec5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa124a66f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1250b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1247c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1250220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124e121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0618782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0228df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa022c096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0642a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c6ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cc02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02aca5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-24.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_157_gd043fd3 LNet: Added LNI 192.168.123.26@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f1bf12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2b1613067 PUD 2b1612067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm drm_panel_orientation_quirks ata_piix serio_raw floppy i2c_core libata virtio_blk [last unloaded: libcfs] CPU: 13 PID: 20461 Comm: mdt06_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff88029b259280 ti: ffff8802b1314000 task.ti: ffff8802b1314000 RIP: 0010:[<ffffffffa0f1bf12>] [<ffffffffa0f1bf12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b1317848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b1317890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88009af86a98 R13: 0000000000000000 R14: ffff8800931ea2a8 R15: ffff8802addfa9e8 FS: 0000000000000000(0000) GS:ffff880331d40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002cc612000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f40ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f413cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0dc7650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dd35ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e3166f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e37b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e2ec6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e37220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e35121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06c8782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025edf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0262096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f2a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0776ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077c02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa035ca5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa072d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa072c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-104.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eecf12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2ddeaa067 PUD 31eea4067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw i2c_core virtio_blk libata [last unloaded: libcfs] CPU: 6 PID: 15291 Comm: mdt03_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802d11b9280 ti: ffff8802958b8000 task.ti: ffff8802958b8000 RIP: 0010:[<ffffffffa0eecf12>] [<ffffffffa0eecf12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802958bb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802958bb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802cb6883e8 R13: 0000000000000000 R14: ffff8802ae8322a8 R15: ffff8802cc311b40 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002a1648000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f11ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f123cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d98650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da45ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e0266f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e08b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dffc6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e08220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e06121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ca782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0256df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa025a096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f4a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0778ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077e02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa035ea5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072d3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa072f059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa072e480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-54.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee2f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2c5dd3067 PUD 2a54fc067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console pcspkr virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk serio_raw i2c_core floppy libata [last unloaded: libcfs] CPU: 11 PID: 26282 Comm: mdt05_002 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802f2bc5c40 ti: ffff88031b2e8000 task.ti: ffff88031b2e8000 RIP: 0010:[<ffffffffa0ee2f12>] [<ffffffffa0ee2f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031b2eb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031b2eb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802db260dd8 R13: 0000000000000000 R14: ffff880323eba2a8 R15: ffff8800989fb890 FS: 0000000000000000(0000) GS:ffff880331cc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002b9684000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f07ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f083cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d8e650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d9a5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0df866f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dfeb30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df5c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0dfe220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0dfc121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06cf782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f9a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077dca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078302a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa033fa5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07323a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0734059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0733480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-9.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed3df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 5694 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800ca602220 ti: ffff88008757c000 task.ti: ffff88008757c000 RIP: 0010:[<ffffffffa0ed3df2>] [<ffffffffa0ed3df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008757f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008757f890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bc794090 R13: 0000000000000000 R14: ffff8800c98e0000 R15: ffff8800b0e1a708 FS: 0000000000000000(0000) GS:ffff88011e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000cf466000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0ef8e70>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0ef924b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d7f4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d8b455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0de93bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df0020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de69bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0def710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0ded611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05db782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0199dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa019d006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0605a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06898b4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa068ec2a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e68ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa063e3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0640059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa063f480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 21117:0:(client.c:1256:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff880116b7ed40 x1735093064329984/t0(0) o41->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 28887:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1654713787 with bad export cookie 13087907309105548108 LustreError: 166-1: MGC192.168.203.163@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28887:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 874:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_43_ge43f033 LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 7338 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800b8a36660 ti: ffff8800bb554000 task.ti: ffff8800bb554000 RIP: 0010:[<ffffffffa0edddf2>] [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800bb557848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800bb557890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bcf38290 R13: 0000000000000000 R14: ffff8800ba239000 R15: ffff8800c90ae898 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000cf614000 CR4: 00000000000006f0 Call Trace: [<ffffffffa0f02e60>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0323b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d894f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d95455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0df33bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dfa020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df09bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df9710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df7611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05e1782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01acdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01b0006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa060ba06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa068fc94>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069500a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01ec8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06443a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0646059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0645480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 1 previous similar message Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LustreError: 28670:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1654682054 with bad export cookie 7127315674021537718 LustreError: 166-1: MGC192.168.203.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28670:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg302-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 630:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.102@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg302-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_55_g8fc89ea LNet: Added LNI 192.168.203.102@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef4f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] CPU: 10 PID: 26406 Comm: mdt05_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802bdfc5c40 ti: ffff8802df55c000 task.ti: ffff8802df55c000 RIP: 0010:[<ffffffffa0ef4f12>] [<ffffffffa0ef4f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802df55f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802df55f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802aece97c8 R13: 0000000000000000 R14: ffff88009c21a2a8 R15: ffff880324496e18 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000b9012000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f19ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1a3cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da0650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dac5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e0a66f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e10b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e07c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e10220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0e121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06b3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06dda06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0761ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa076702a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa0347a5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07163a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0718059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0717480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-9.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef2f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw i2c_core libata [last unloaded: libcfs] CPU: 14 PID: 16624 Comm: mdt07_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800b48f1280 ti: ffff8802b9084000 task.ti: ffff8802b9084000 RIP: 0010:[<ffffffffa0ef2f12>] [<ffffffffa0ef2f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b9087848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b9087890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880098f2c2d8 R13: 0000000000000000 R14: ffff88009d4822a8 R15: ffff8802c4afb678 FS: 0000000000000000(0000) GS:ffff880331d80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031afbe000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f17ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f183cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9e650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0daa5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e0866f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0eb30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e05c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0e220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0c121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06c5782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025edf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0262096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06efa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0773ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077902a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa0359a5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07283a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa072a059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0729480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-109.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1339df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm drm_panel_orientation_quirks libata serio_raw i2c_core virtio_blk floppy [last unloaded: libcfs] CPU: 12 PID: 6149 Comm: mdt06_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800a8e524f0 ti: ffff880079988000 task.ti: ffff880079988000 RIP: 0010:[<ffffffffa1339df2>] [<ffffffffa1339df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88007998b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88007998b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802b56e21b8 R13: 0000000000000000 R14: ffff88029ac80008 R15: ffff8802c220a9e8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000b91c6000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e54f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124f3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1256020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa124c9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1255710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1253611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0619782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022ddd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0231006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0643a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c77c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06d1e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa026d8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067c3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067e059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067d480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-24.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.26@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1351df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 83fb3067 PUD 77389067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata floppy serio_raw i2c_core virtio_blk [last unloaded: libcfs] CPU: 8 PID: 14284 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff88027283a4f0 ti: ffff8803250c4000 task.ti: ffff8803250c4000 RIP: 0010:[<ffffffffa1351df2>] [<ffffffffa1351df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8803250c7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8803250c7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88026660e748 R13: 0000000000000000 R14: ffff8803251122a8 R15: ffff8802db357460 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000009a780000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1376e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137722b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11fd4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1209455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12673bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126e020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12649bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126d710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126b611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0613782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0235dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0239006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa063da06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c17c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cbe6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02b08ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06763a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0678059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0677480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-84.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.86@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: Setting parameter lustre-MDT0002.mdt.identity_upcall in log lustre-MDT0002 Lustre: srv-lustre-MDT0002: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0002: new disk, initializing Lustre: lustre-MDT0002: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2:mdt Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2:mdt] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:3e8:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f08df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk libata i2c_core floppy serio_raw [last unloaded: libcfs] CPU: 0 PID: 22762 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802da2b1280 ti: ffff880290e30000 task.ti: ffff880290e30000 RIP: 0010:[<ffffffffa0f08df2>] [<ffffffffa0f08df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880290e33848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880290e33890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88031d191128 R13: 0000000000000000 R14: ffff8802933ea2a8 R15: ffff880318f2faa8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f2de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f2e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0db44f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dc0455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e1e3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e25020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e1b9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e24710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e22611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ba782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa026add5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa026e006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06e4a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07687c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0772e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03498ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa071d3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa071f059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa071e480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-44.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1354df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 27f374067 PUD 277046067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] CPU: 12 PID: 6840 Comm: mdt06_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802d9803760 ti: ffff8800b4a68000 task.ti: ffff8800b4a68000 RIP: 0010:[<ffffffffa1354df2>] [<ffffffffa1354df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b4a6b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b4a6b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802c4616de8 R13: 0000000000000000 R14: ffff8802d33bb3f8 R15: ffff8802793ea9e8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002df98a000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1379e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137a22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e64f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f2455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa126a3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1271020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12679bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1270710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126e611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0618782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0231dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0235006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0642a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c67c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06d0e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02d98ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-19.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.21@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD a83ae067 PUD 93d38067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks virtio_blk floppy serio_raw i2c_core libata [last unloaded: libcfs] CPU: 10 PID: 23879 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802c29ddc40 ti: ffff88031ebf8000 task.ti: ffff88031ebf8000 RIP: 0010:[<ffffffffa0ee8df2>] [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031ebfb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031ebfb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802f4f65d58 R13: 0000000000000000 R14: ffff8802b5d8a2a8 R15: ffff8802d05bdb40 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000abc34000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f0de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d944f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da0455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0dfe3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e05020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfb9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e04710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e02611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0742782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0262dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0266006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa076ca06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07f07c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07fae6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa04038ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07a53a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07a7059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07a6480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-49.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-49.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-49.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-49.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1338df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2c5547067 PUD 2c5548067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix virtio_blk serio_raw i2c_core libata floppy [last unloaded: libcfs] CPU: 2 PID: 7040 Comm: mdt01_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff88031efcdc40 ti: ffff8802a7134000 task.ti: ffff8802a7134000 RIP: 0010:[<ffffffffa1338df2>] [<ffffffffa1338df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802a7137848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802a7137890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8803258b8a88 R13: 0000000000000000 R14: ffff8802df8622a8 R15: ffff8802d677e9e8 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ce9c0000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e44f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f0455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124e3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1255020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa124b9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1254710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1252611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f5782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0233dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0237006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061fa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a37c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ade6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02738ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06583a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa065a059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0659480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-89.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.91@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-89.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-89.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-89.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1362df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2a2976067 PUD 2b1818067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk libata floppy i2c_core [last unloaded: libcfs] CPU: 8 PID: 9217 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802c7ba3760 ti: ffff8802bd76c000 task.ti: ffff8802bd76c000 RIP: 0010:[<ffffffffa1362df2>] [<ffffffffa1362df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bd76f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bd76f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88028b8b38e8 R13: 0000000000000000 R14: ffff88029b0e1158 R15: ffff8802976de9e8 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ce764000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1387e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138822b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11f44f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1200455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12783bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127f020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12759bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa127e710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa127c611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa060d782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa024edd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0252006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0637a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06bb7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06c5e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02a18ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06703a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0672059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0671480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-14.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.16@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 27443d067 PUD 27bafa067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw i2c_core floppy virtio_blk libata [last unloaded: libcfs] CPU: 0 PID: 24398 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802acc88010 ti: ffff880294e6c000 task.ti: ffff880294e6c000 RIP: 0010:[<ffffffffa1335df2>] [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880294e6f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880294e6f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802df471e68 R13: 0000000000000000 R14: ffff8802ee925698 R15: ffff8802ef903460 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000029bf4a000 CR4: 00000000000007f0 Call Trace: [<ffffffffa135ae50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135b22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e14f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ed455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124b3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1252020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12489bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1251710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124f611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05de782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0225dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0229006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0608a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa068c7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0696e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02938ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06413a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0643059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0642480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-84.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.86@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: Setting parameter lustre-MDT0002.mdt.identity_upcall in log lustre-MDT0002 Lustre: srv-lustre-MDT0002: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0002: new disk, initializing Lustre: lustre-MDT0002: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2:mdt Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2:mdt] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:3e8:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef9df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_balloon virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 6 PID: 29334 Comm: mdt03_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802dbcb0010 ti: ffff8802b2ad8000 task.ti: ffff8802b2ad8000 RIP: 0010:[<ffffffffa0ef9df2>] [<ffffffffa0ef9df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b2adb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b2adb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802e2d89478 R13: 0000000000000000 R14: ffff8802eeb822a8 R15: ffff8800b5dcf248 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f1ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da54f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0db1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e0f3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e16020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e0c9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e15710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e13611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa070e782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0276dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa027a006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0738a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07bc7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07c6e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03cf8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07713a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0773059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0772480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-29.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-29.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-29.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-29.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console pcspkr i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw virtio_blk libata i2c_core [last unloaded: libcfs] CPU: 4 PID: 16563 Comm: mdt02_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802ecf48010 ti: ffff8802b8e70000 task.ti: ffff8802b8e70000 RIP: 0010:[<ffffffffa1360df2>] [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b8e73848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b8e73890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802af293c38 R13: 0000000000000000 R14: ffff8802f03eb3f8 R15: ffff8800987269e8 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1385e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138622b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11f24f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11fe455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12763bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127d020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12739bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa127c710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa127a611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0606782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0253dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0257006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0630a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06b47c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06bee6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02938ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06693a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa066b059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa066a480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-39.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.41@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-39.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-39.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-39.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1329df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix i2c_core serio_raw virtio_blk floppy libata [last unloaded: libcfs] CPU: 0 PID: 13834 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802cce3c9d0 ti: ffff88008df50000 task.ti: ffff88008df50000 RIP: 0010:[<ffffffffa1329df2>] [<ffffffffa1329df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008df53848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008df53890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a8b242d8 R13: 0000000000000000 R14: ffff880094c09158 R15: ffff88009de3e9e8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000003195d4000 CR4: 00000000000007f0 Call Trace: [<ffffffffa134ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa134f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11d54f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11e1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa123f3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1246020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa123c9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1245710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1243611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa061b782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0645a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c97c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06d3e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02d78ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067e3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0680059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067f480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-94.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.96@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_balloon pcspkr virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk floppy serio_raw libata i2c_core [last unloaded: libcfs] CPU: 6 PID: 19911 Comm: mdt03_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802bca3dc40 ti: ffff8802ce64c000 task.ti: ffff8802ce64c000 RIP: 0010:[<ffffffffa0eebdf2>] [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ce64f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ce64f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800ad43d368 R13: 0000000000000000 R14: ffff88008fd622a8 R15: ffff8800b42b4a80 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000003196d0000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f10e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1122b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d974f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da3455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e013bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e08020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfe9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e07710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e05611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06cc782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f6a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077a7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0784e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa038d8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072f3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0731059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0730480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-94.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f17df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2ee5f3067 PUD 2b23ea067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic drm_panel_orientation_quirks virtio_blk pata_acpi floppy serio_raw ata_piix i2c_core libata [last unloaded: libcfs] CPU: 4 PID: 16326 Comm: mdt02_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880322ab3760 ti: ffff8802e2fa4000 task.ti: ffff8802e2fa4000 RIP: 0010:[<ffffffffa0f17df2>] [<ffffffffa0f17df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802e2fa7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802e2fa7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880296df0dd8 R13: 0000000000000000 R14: ffff8802af2fa2a8 R15: ffff8802da8f3890 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ca692000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f3ce50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f3d22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0dc34f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dcf455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e2d3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e34020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e2a9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e33710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e31611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0721782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0261dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0265006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa074ba06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07cf7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07d9e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03808ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07843a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0786059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0785480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-59.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee9df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix i2c_core serio_raw virtio_blk libata floppy [last unloaded: libcfs] CPU: 12 PID: 14557 Comm: mdt06_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880091d949d0 ti: ffff88009c3bc000 task.ti: ffff88009c3bc000 RIP: 0010:[<ffffffffa0ee9df2>] [<ffffffffa0ee9df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009c3bf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009c3bf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88029e0717c8 R13: 0000000000000000 R14: ffff8800a426a2a8 R15: ffff8802dc2125b8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f0ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d954f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0dff3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e06020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfc9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e05710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e03611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06be782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025ddd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0261006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06e8a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa076c7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0776e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa037f8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07213a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0723059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0722480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-44.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm drm_panel_orientation_quirks floppy virtio_blk serio_raw libata i2c_core [last unloaded: libcfs] CPU: 10 PID: 28640 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802edb39280 ti: ffff8802d9fdc000 task.ti: ffff8802d9fdc000 RIP: 0010:[<ffffffffa0eebdf2>] [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802d9fdf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802d9fdf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a1ab60a8 R13: 0000000000000000 R14: ffff8802da88a2a8 R15: ffff8802b7f4b030 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f10e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1122b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d974f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da3455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e013bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e08020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfe9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e07710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e05611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06faa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077e7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0788e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03758ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07333a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0735059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0734480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-54.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eefdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks ata_piix virtio_blk i2c_core serio_raw floppy libata [last unloaded: libcfs] CPU: 0 PID: 3001 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880327053760 ti: ffff88031e328000 task.ti: ffff88031e328000 RIP: 0010:[<ffffffffa0eefdf2>] [<ffffffffa0eefdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031e32b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031e32b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802acea0a88 R13: 0000000000000000 R14: ffff880327be22a8 R15: ffff8800aae8f248 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000ba862000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f14e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1522b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9b4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da7455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e053bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0c020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e029bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0b710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e09611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d8782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa026cdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0270006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0702a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07867c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0790e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03828ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa073b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa073d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa073c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-119.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0efddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2f05fc067 PUD 2f05fd067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk libata i2c_core floppy [last unloaded: libcfs] CPU: 4 PID: 6377 Comm: mdt02_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802ac765c40 ti: ffff88009d0a4000 task.ti: ffff88009d0a4000 RIP: 0010:[<ffffffffa0efddf2>] [<ffffffffa0efddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009d0a7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009d0a7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802e850b8e8 R13: 0000000000000000 R14: ffff8800a5a222a8 R15: ffff88009102baa8 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002f054e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f22e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f2322b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da94f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0db5455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e133bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e1a020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e109bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e19710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e17611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d6782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025cdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0260006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0700a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07847c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078ee6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa036a8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07393a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa073b059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa073a480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-14.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0002: Not available for connect from 0@lo (not set up) Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 296b35067 PUD 2b7113067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic ttm pata_acpi drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw i2c_core libata [last unloaded: libcfs] CPU: 1 PID: 29272 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880092b549d0 ti: ffff880323a6c000 task.ti: ffff880323a6c000 RIP: 0010:[<ffffffffa1360df2>] [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880323a6f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880323a6f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88029710e3f8 R13: 0000000000000000 R14: ffff880091e422a8 R15: ffff8802faf6a9e8 FS: 0000000000000000(0000) GS:ffff880331a40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002c69d4000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1385e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138622b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa120c4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1218455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12763bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127d020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12739bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa127c710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa127a611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0662782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0232dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0236006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa068ca06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07107c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa071ae6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03238ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06c53a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06c7059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06c6480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-114.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_58_g7d7a06f2 LNet: Added LNI 192.168.123.116@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-114.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-114.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-114.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic pata_acpi drm_panel_orientation_quirks ata_piix serio_raw virtio_blk floppy i2c_core libata [last unloaded: libcfs] CPU: 10 PID: 28742 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800940f8010 ti: ffff88009ef44000 task.ti: ffff88009ef44000 RIP: 0010:[<ffffffffa0eeddf2>] [<ffffffffa0eeddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009ef47848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009ef47890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880092b338e8 R13: 0000000000000000 R14: ffff88009f0222a8 R15: ffff8800a5c83aa8 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f12e70>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1324b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d994f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da5455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e033bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0a020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e009bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e09710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e07611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0703782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa072da06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07b13e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07bba8a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03c48ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07663a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0768059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0767480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-54.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee3df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 1 PID: 5311 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff880116869110 ti: ffff88010c410000 task.ti: ffff88010c410000 RIP: 0010:[<ffffffffa0ee3df2>] [<ffffffffa0ee3df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88010c413848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88010c413890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88008d599e90 R13: 0000000000000000 R14: ffff88010e645000 R15: ffff8800bbb33180 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d6c00000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0f08e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0922b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d91570>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d9d091>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0df93bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e00020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df69bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0dff710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0dfd611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f6782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01a8dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01ac006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0620a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a45d4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ae7fa>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e87fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0659180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa065ae39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa065a260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 1 previous similar message Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages LustreError: 20743:0:(client.c:1256:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff8800913d3900 x1734252909587776/t0(0) o41->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete LustreError: 28502:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1653912545 with bad export cookie 5663760698040137429 LustreError: 166-1: MGC192.168.203.177@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28502:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: oleg377-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 458:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.177@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg377-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_14_g692e820 LNet: Added LNI 192.168.203.177@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeedf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr i2c_piix4 virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks i2c_core ata_piix serio_raw virtio_blk floppy libata [last unloaded: libcfs] CPU: 0 PID: 2719 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800abbb1280 ti: ffff8802ee9bc000 task.ti: ffff8802ee9bc000 RIP: 0010:[<ffffffffa0eeedf2>] [<ffffffffa0eeedf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ee9bf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ee9bf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802de2ce0a8 R13: 0000000000000000 R14: ffff8800aa8322a8 R15: ffff880327b63aa8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f13e70>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1424b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9a4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da6455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e043bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0b020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e019bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0a710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e08611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06e0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa02a7dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02ab006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa070aa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa078e3e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0798a8a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03868ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07433a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0745059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0744480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-119.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa133ef12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon pcspkr virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm virtio_blk libata drm_panel_orientation_quirks serio_raw i2c_core floppy [last unloaded: libcfs] CPU: 4 PID: 16891 Comm: mdt02_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802bc3ddc40 ti: ffff8802d2050000 task.ti: ffff8802d2050000 RIP: 0010:[<ffffffffa133ef12>] [<ffffffffa133ef12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802d2053848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802d2053890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802afb4a1b8 R13: 0000000000000000 R14: ffff8802ea118008 R15: ffff8802f58ba9e8 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1363fe0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa13643bb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ea650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f65ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa12543cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa125a890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12519cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1259f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1257e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05e2841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa022edf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0232096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa060caa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0690d74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069b24a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02768ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0645470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0647129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0646550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-104.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_141_gb58179d LNet: Added LNI 192.168.123.106@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ecef12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks virtio_blk serio_raw floppy i2c_core libata [last unloaded: libcfs] CPU: 2 PID: 9416 Comm: mdt01_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880320e95c40 ti: ffff8802bde60000 task.ti: ffff8802bde60000 RIP: 0010:[<ffffffffa0ecef12>] [<ffffffffa0ecef12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bde63848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bde63890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a2255018 R13: 0000000000000000 R14: ffff8802989fa2a8 R15: ffff8802ed83a188 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0ef4010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0ef43eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d7a650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d865ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0de43cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dea890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de19cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0de9f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0de7e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0725841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa027adf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa027e096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa074faa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07d3d74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07de24a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03e68ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0788470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa078a129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0789550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-119.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f0bf12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 9246f067 PUD 93748067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks ata_piix virtio_blk i2c_core serio_raw libata floppy [last unloaded: libcfs] CPU: 15 PID: 31596 Comm: mdt07_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802dcdfdc40 ti: ffff8802b6274000 task.ti: ffff8802b6274000 RIP: 0010:[<ffffffffa0f0bf12>] [<ffffffffa0f0bf12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b6277848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b6277890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802cac9bc38 R13: 0000000000000000 R14: ffff8802cd13a2a8 R15: ffff8802dc3b4eb0 FS: 0000000000000000(0000) GS:ffff880331dc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000bafb8000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f31010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f313eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0db7650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dc35ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e213cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e27890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e1e9cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e26f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e24e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ce841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa025fdf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0263096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f8aa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077cd74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078724a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa033e8ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0731470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0733129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0732550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-94.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f22f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD a5f7d067 PUD adaa1067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks serio_raw virtio_blk ata_piix floppy libata i2c_core [last unloaded: libcfs] CPU: 6 PID: 20679 Comm: mdt03_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802a86b9280 ti: ffff8802cf5f8000 task.ti: ffff8802cf5f8000 RIP: 0010:[<ffffffffa0f22f12>] [<ffffffffa0f22f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802cf5fb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802cf5fb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802d216a508 R13: 0000000000000000 R14: ffff8802e8d7a2a8 R15: ffff8802b997df70 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000aba9a000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f48010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f483eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0b5c650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0b685ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e383cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e3e890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e359cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e3df80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e3be81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0be4841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa0261df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0265096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0c0eaa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0c92d74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0c9d24a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa035b8ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0c47470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0c49129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0c48550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-109.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeef12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic pata_acpi ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] CPU: 6 PID: 3711 Comm: mdt03_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802d2ecdc40 ti: ffff8802ed82c000 task.ti: ffff8802ed82c000 RIP: 0010:[<ffffffffa0eeef12>] [<ffffffffa0eeef12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ed82f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ed82f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802d2b863f8 R13: 0000000000000000 R14: ffff8802e5a7a2a8 R15: ffff8800a8133678 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031d89e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f14010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f143eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9a650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da65ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e043cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0a890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e019cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e09f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e07e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ce841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa0267df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa026b096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f8aa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077cd74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078724a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03628ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0731470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0733129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0732550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-19.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |