| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa13fb062>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata [last unloaded: libcfs] CPU: 2 PID: 3556 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff8800a6e33330 ti: ffff880134a34000 task.ti: ffff880134a34000 RIP: 0010:[<ffffffffa13fb062>] [<ffffffffa13fb062>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880134a37848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880134a37890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a63c3a90 R13: 0000000000000000 R14: ffff88012e1ed000 R15: ffff880095775240 FS: 0000000000000000(0000) GS:ffff88013e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b223c000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa14203d0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa14207ab>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa128c5e0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa12982b5>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa130e72f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1314c00>] mdt_intent_layout+0x910/0xe70 [mdt] [<ffffffffa130bd2d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa13142f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa13121f1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07e97ba>] ldlm_lock_enqueue+0x3ca/0xb50 [ptlrpc] [<ffffffffa02b8545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02bb7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0813fe6>] ldlm_handle_enqueue0+0x8c6/0x1790 [ptlrpc] [<ffffffffa08988e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a0dda>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02ed9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa084c687>] ptlrpc_server_handle_request+0x257/0xc40 [ptlrpc] [<ffffffffa084e349>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa084d770>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 4086:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069039/real 1750069039] req@ffff8800a169f200 x1835080287792960/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1750069045 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 4086:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069045/real 1750069045] req@ffff880097734c00 x1835080287793344/t0(0) o251->MGC192.168.202.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1750069051 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg244-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 4767:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.144@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg244-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.202.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 8821:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069108/real 1750069108] req@ffff8800a169da40 x1835080424101248/t0(0) o251->MGC192.168.202.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1750069114 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg244-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 9476:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.144@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg244-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.202.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 13299:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069172/real 1750069172] req@ffff8801311a97c0 x1835080493307200/t0(0) o251->MGC192.168.202.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1750069178 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg244-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 13980:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.144@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg244-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.202.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 19204:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069247/real 1750069247] req@ffff8801311ad0c0 x1835080562514048/t0(0) o251->MGC192.168.202.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1750069253 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg244-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 19876:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.144@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg244-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.202.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 23953:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069304/real 1750069304] req@ffff8800a169c280 x1835080632768128/t0(0) o251->MGC192.168.202.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1750069310 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg244-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 24609:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.144@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg244-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.202.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 28453:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1750069358/real 1750069358] req@ffff880097730e40 x1835080691488064/t0(0) o251->MGC192.168.202.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1750069364 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg244-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 29117:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.144@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg244-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.202.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg244-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa142c062>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw [last unloaded: libcfs] CPU: 3 PID: 3620 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff8800a95fc440 ti: ffff8800a8484000 task.ti: ffff8800a8484000 RIP: 0010:[<ffffffffa142c062>] [<ffffffffa142c062>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800a8487848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800a8487890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88012de19c90 R13: 0000000000000000 R14: ffff8800a6652000 R15: ffff88009f2d0480 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001368ac000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa14513d0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa14517ab>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa12d65e0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa12e22b5>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa133f72f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1345c00>] mdt_intent_layout+0x910/0xe70 [mdt] [<ffffffffa133cd2d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa13452f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa13431f1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07fd7ba>] ldlm_lock_enqueue+0x3ca/0xb50 [ptlrpc] [<ffffffffa0233545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02367e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0827fe6>] ldlm_handle_enqueue0+0x8c6/0x1790 [ptlrpc] [<ffffffffa08ac8e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08b4dda>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03c99be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0860687>] ptlrpc_server_handle_request+0x257/0xc40 [ptlrpc] [<ffffffffa0862349>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0861770>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 4368:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722640/real 1749722640] req@ffff88009a28c740 x1834717058969856/t0(0) o251->MGC192.168.204.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749722646 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 4368:0:(client.c:2295:ptlrpc_expire_one_request()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 5031:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.134@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 9055:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722696/real 1749722696] req@ffff88012d756d40 x1834717190034944/t0(0) o251->MGC192.168.204.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749722702 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 9708:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.134@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: server umount lustre-OST0000 complete Lustre: 13527:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722746/real 1749722746] req@ffff88009a28da40 x1834717246657088/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1749722752 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 13527:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722752/real 1749722752] req@ffff8800951a4740 x1834717246657472/t0(0) o251->MGC192.168.204.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749722758 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 14200:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.134@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: server umount lustre-OST0000 complete Lustre: 19434:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722818/real 1749722818] req@ffff8800951a5580 x1834717308524224/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1749722824 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 19434:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722824/real 1749722824] req@ffff8800951a5580 x1834717308524672/t0(0) o251->MGC192.168.204.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749722830 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 20111:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.134@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 24112:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722881/real 1749722881] req@ffff88009fd61300 x1834717381924288/t0(0) o251->MGC192.168.204.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749722887 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 24765:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.134@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 28555:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749722941/real 1749722941] req@ffff8800951a17c0 x1834717445887296/t0(0) o251->MGC192.168.204.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749722947 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 29216:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.134@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1035062>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul squashfs ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata [last unloaded: libcfs] CPU: 3 PID: 8966 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 task: ffff8801308ba220 ti: ffff8801311a0000 task.ti: ffff8801311a0000 RIP: 0010:[<ffffffffa1035062>] [<ffffffffa1035062>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8801311a3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8801311a3890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88012b756290 R13: 0000000000000000 R14: ffff8800a8e96000 R15: ffff8801368c40c0 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b1dfa000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa105a3d0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa105a7ab>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0edf5e0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0eeb2b5>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0f4872f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f4ec00>] mdt_intent_layout+0x910/0xe70 [mdt] [<ffffffffa0f45d2d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f4e2f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f4c1f1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07e37ba>] ldlm_lock_enqueue+0x3ca/0xb50 [ptlrpc] [<ffffffffa02f3545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02f67e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa080dfe6>] ldlm_handle_enqueue0+0x8c6/0x1790 [ptlrpc] [<ffffffffa08928e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa089adda>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03319be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0846687>] ptlrpc_server_handle_request+0x257/0xc40 [ptlrpc] [<ffffffffa0848349>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0847770>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-MDT0000 complete LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LustreError: 29549:0:(ldlm_lockd.c:2526:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1749722518 with bad export cookie 545966603545843992 LustreError: 30517:0:(osp_object.c:629:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000bd2:0x1:0x0]: rc = -5 LustreError: 30517:0:(llog_cat.c:753:llog_cat_cancel_arr_rec()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 llog-records: rc = -5 LustreError: 30517:0:(llog_cat.c:789:llog_cat_cancel_records()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 of 1 llog-records: rc = -5 LustreError: 166-1: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 29549:0:(ldlm_lockd.c:2526:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 1558:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.160@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7 LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1421062>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 5649 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff8800962b5550 ti: ffff88012b214000 task.ti: ffff88012b214000 RIP: 0010:[<ffffffffa1421062>] [<ffffffffa1421062>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88012b217848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88012b217890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff880129ea5690 R13: 0000000000000000 R14: ffff88012fb2f000 R15: ffff8800990dd240 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001372ce000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa14463d0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa14467ab>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa12cb5e0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa12d72b5>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa133472f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa133ac00>] mdt_intent_layout+0x910/0xe70 [mdt] [<ffffffffa1331d2d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa133a2f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa13381f1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa08007ba>] ldlm_lock_enqueue+0x3ca/0xb50 [ptlrpc] [<ffffffffa021a545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa021d7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa082afe6>] ldlm_handle_enqueue0+0x8c6/0x1790 [ptlrpc] [<ffffffffa08af8e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08b7dda>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa032e9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0863687>] ptlrpc_server_handle_request+0x257/0xc40 [ptlrpc] [<ffffffffa0865349>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0864770>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 2881:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201492/real 1749201492] req@ffff88012f9bed40 x1834170647062784/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1749201498 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 2881:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201498/real 1749201498] req@ffff88012f9bed40 x1834170647063040/t0(0) o251->MGC192.168.204.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749201504 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 3561:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.156@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg456-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7_RC2 LNet: Added LNI 192.168.204.156@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 7596:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201534/real 1749201534] req@ffff88012ff16d40 x1834170725700480/t0(0) o251->MGC192.168.204.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749201540 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 8252:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.156@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg456-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7_RC2 LNet: Added LNI 192.168.204.156@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 12025:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201565/real 1749201565] req@ffff88012ff13440 x1834170763448768/t0(0) o251->MGC192.168.204.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749201571 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 12687:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.156@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg456-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7_RC2 LNet: Added LNI 192.168.204.156@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 17875:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201602/real 1749201602] req@ffff88009ff13440 x1834170798052160/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1749201608 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 17875:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201608/real 1749201608] req@ffff88012f9b8000 x1834170798052416/t0(0) o251->MGC192.168.204.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749201614 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 18551:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.156@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg456-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7_RC2 LNet: Added LNI 192.168.204.156@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 22636:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201644/real 1749201644] req@ffff880094408e40 x1834170841043840/t0(0) o251->MGC192.168.204.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749201650 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 23295:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.156@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg456-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7_RC2 LNet: Added LNI 192.168.204.156@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 27109:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1749201676/real 1749201676] req@ffff88009081e3c0 x1834170879840704/t0(0) o251->MGC192.168.204.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1749201682 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg456-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 27774:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.156@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg456-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.7_RC2 LNet: Added LNI 192.168.204.156@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg456-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg456-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa133ffb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon pcspkr virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic pata_acpi drm_panel_orientation_quirks serio_raw ata_piix virtio_blk floppy i2c_core libata [last unloaded: libcfs] CPU: 12 PID: 12724 Comm: mdt06_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880294a6b760 ti: ffff880294710000 task.ti: ffff880294710000 RIP: 0010:[<ffffffffa133ffb2>] [<ffffffffa133ffb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880294713848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880294713890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802b769fb28 R13: 0000000000000000 R14: ffff8802997a1158 R15: ffff8802b4e54c68 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1365230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa136560b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ea760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f6431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa125353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1259a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1250b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12590f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1256ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0671782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0263545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02667e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa069bef6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0720614>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa072548a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02909be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06d4687>] ptlrpc_server_handle_request+0x257/0xc40 [ptlrpc] [<ffffffffa06d6349>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06d5770>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-149.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_25_g0fc80ba LNet: Added LNI 192.168.123.151@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-149.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-149.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-149.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef5fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2b9563067 PUD 2c78db067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 virtio_console pcspkr ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic pata_acpi drm_panel_orientation_quirks ata_piix virtio_blk i2c_core serio_raw libata floppy [last unloaded: libcfs] CPU: 2 PID: 26957 Comm: mdt01_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802cac4dc40 ti: ffff8802f54d4000 task.ti: ffff8802f54d4000 RIP: 0010:[<ffffffffa0ef5fb2>] [<ffffffffa0ef5fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802f54d7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802f54d7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802dc056748 R13: 0000000000000000 R14: ffff88009bebd698 R15: ffff88008e5ea318 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002c6c48000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f1b230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1b60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da0760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dac431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e0953f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0fa00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e06b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0f0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0cff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06f9782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0262545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02657e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0723ef6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07a8614>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07ad48a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03a89be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa075c687>] ptlrpc_server_handle_request+0x257/0xc40 [ptlrpc] [<ffffffffa075e349>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa075d770>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-159.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1042fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 8183 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff880136a55550 ti: ffff88012d364000 task.ti: ffff88012d364000 RIP: 0010:[<ffffffffa1042fb2>] [<ffffffffa1042fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88012d367848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88012d367890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800796b1c90 R13: 0000000000000000 R14: ffff88008c7e1000 R15: ffff88007ae93180 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000129f38000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa1068230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa106860b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0eed760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0ef9431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f5653f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f5ca00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f53b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f5c0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f59ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07f0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa02f4545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02f77e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa081abc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa089f394>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a420a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa032d9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0853410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa08550c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa08544f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 27745:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744450194 with bad export cookie 8047587791055860565 LustreError: 166-1: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 27745:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32160:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.157@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_11_ged826bd LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1431fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 4318 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 task: ffff88012b0b0000 ti: ffff88012d2d8000 task.ti: ffff88012d2d8000 RIP: 0010:[<ffffffffa1431fb2>] [<ffffffffa1431fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88012d2db848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88012d2db890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88008f9e5a90 R13: 0000000000000000 R14: ffff8800908cb000 R15: ffff8800a5599900 FS: 0000000000000000(0000) GS:ffff88013e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b20c2000 CR4: 0000000000160ff0 Call Trace: [<ffffffffa1457230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa145760b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa12dc760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa12e8431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa134553f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa134ba00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1342b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa134b0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1348ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07ee782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa02c4545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02c77e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0818bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa089d3d4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a224a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa031d9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0851440>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa08530f9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0852520>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 1997:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1737409899/real 1737409899] req@ffff880095b2b440 x1821806255156032/t0(0) o251->MGC192.168.201.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1737409905 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 2638:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.101@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg101-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_1_g59ca12c LNet: Added LNI 192.168.201.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 6667:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1737409939/real 1737409939] req@ffff880095b2b440 x1821806340084608/t0(0) o251->MGC192.168.201.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1737409945 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 7314:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.101@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg101-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_1_g59ca12c LNet: Added LNI 192.168.201.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 11120:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1737409972/real 1737409972] req@ffff88012dbb84c0 x1821806379929984/t0(0) o251->MGC192.168.201.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1737409978 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 11779:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.101@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg101-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_1_g59ca12c LNet: Added LNI 192.168.201.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 16944:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1737410012/real 1737410012] req@ffff880095b2bdc0 x1821806416630720/t0(0) o251->MGC192.168.201.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1737410018 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 17606:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.101@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg101-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_1_g59ca12c LNet: Added LNI 192.168.201.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 21684:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1737410069/real 1737410069] req@ffff88009367cc00 x1821806457526272/t0(0) o251->MGC192.168.201.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1737410075 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 22324:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.101@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg101-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_1_g59ca12c LNet: Added LNI 192.168.201.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 26164:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1737410126/real 1737410126] req@ffff88009367f200 x1821806521488896/t0(0) o251->MGC192.168.201.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1737410132 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg101-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 26832:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.101@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg101-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6_1_g59ca12c LNet: Added LNI 192.168.201.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg101-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1051fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 4820 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff88009daa2220 ti: ffff880073db0000 task.ti: ffff880073db0000 RIP: 0010:[<ffffffffa1051fb2>] [<ffffffffa1051fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880073db3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880073db3890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88011ed96290 R13: 0000000000000000 R14: ffff88009d514000 R15: ffff88012f67a480 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000129cda000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa1077230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa107760b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0efc760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0f08431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f6553f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f6ba00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f62b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f6b0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f68ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07ff782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0251545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02547e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0829bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa08ae3d4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08b324a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03339be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0862440>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa08640f9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0863520>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LustreError: 28018:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1733653802 with bad export cookie 13395378074163294287 LustreError: 166-1: MGC192.168.204.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28018:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg422-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32437:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.122@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg422-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.6 LNet: Added LNI 192.168.204.122@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg422-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg422-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: Skipped 1 previous similar message | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef3422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw i2c_core virtio_blk libata [last unloaded: libcfs] CPU: 0 PID: 30506 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802ba259280 ti: ffff8802bbdb0000 task.ti: ffff8802bbdb0000 RIP: 0010:[<ffffffffa0ef3422>] [<ffffffffa0ef3422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bbdb3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bbdb3890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802cae8aef8 R13: 0000000000000000 R14: ffff88031eff5698 R15: ffff8800bafd8428 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f18c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1901b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9c810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da84e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e065df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0d240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e03bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0c930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0a831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0266545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02697e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06fdbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0782414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078728a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03329be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0736480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0738139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0737560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-184.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa133e422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic pata_acpi drm_panel_orientation_quirks ata_piix floppy serio_raw virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 0 PID: 30284 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802d75224f0 ti: ffff8802ec5e4000 task.ti: ffff8802ec5e4000 RIP: 0010:[<ffffffffa133e422>] [<ffffffffa133e422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ec5e7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ec5e7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88026a4daef8 R13: 0000000000000000 R14: ffff8802ee025698 R15: ffff880269db5298 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031a79c000 CR4: 00000000000007f0 Call Trace: [<ffffffffa1363c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa136401b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e7810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f34e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa12515df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1258240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa124ebdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1257930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1255831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa065e782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0246545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02497e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0688bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa070d414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa071228a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02739be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06c1480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06c3139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa06c2560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-204.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.206@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1329422>] lod_fill_mirrors+0x192/0x620 [lod] PGD ab977067 PUD ab978067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw i2c_core libata [last unloaded: libcfs] CPU: 8 PID: 954 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8800897f3760 ti: ffff8802b5a64000 task.ti: ffff8802b5a64000 RIP: 0010:[<ffffffffa1329422>] [<ffffffffa1329422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b5a67848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b5a67890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880087183598 R13: 0000000000000000 R14: ffff8802914c0008 R15: ffff8802ad7a1298 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000a8a3e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa134ec40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa134f01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11d2810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11de4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa123c5df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1243240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1239bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1242930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1240831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0615782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0223545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02267e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa063fbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c4414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06c928a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02509be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0678480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067a139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0679560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-189.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.191@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1359422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm drm_panel_orientation_quirks virtio_blk floppy serio_raw libata i2c_core [last unloaded: libcfs] CPU: 6 PID: 13916 Comm: mdt03_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880293409280 ti: ffff88029797c000 task.ti: ffff88029797c000 RIP: 0010:[<ffffffffa1359422>] [<ffffffffa1359422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88029797f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88029797f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802d62b3f88 R13: 0000000000000000 R14: ffff8802a54fa2a8 R15: ffff8802e94fd298 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000327a86000 CR4: 00000000000007e0 Call Trace: [<ffffffffa137ec40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137f01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa1202810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa120e4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126c5df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1273240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1269bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1272930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1270831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa061b782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0242545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02457e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0645bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06ca414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cf28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa026f9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067e480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0680139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067f560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-129.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.131@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-129.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-129.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-129.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f4d422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2d3be8067 PUD 296bd2067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk floppy i2c_core libata [last unloaded: libcfs] CPU: 5 PID: 25217 Comm: mdt02_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802ecd71280 ti: ffff8802bb104000 task.ti: ffff8802bb104000 RIP: 0010:[<ffffffffa0f4d422>] [<ffffffffa0f4d422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bb107848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bb107890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802c4ff77d8 R13: 0000000000000000 R14: ffff8802cd275698 R15: ffff8802a4d52948 FS: 0000000000000000(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002b6356000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f72c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f7301b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0b49810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0b554e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e605df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e67240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e5dbdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e66930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e64831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0c0b782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa028c545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa028f7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0c35bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0cba414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0cbf28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa039b9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0c6e480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0c70139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0c6f560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-129.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-129.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-129.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-129.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef8422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2e7523067 PUD 2cdc70067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic pata_acpi ttm drm ata_piix drm_panel_orientation_quirks floppy i2c_core libata serio_raw virtio_blk [last unloaded: libcfs] CPU: 5 PID: 3700 Comm: mdt02_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8800a3a88010 ti: ffff8802b31e0000 task.ti: ffff8802b31e0000 RIP: 0010:[<ffffffffa0ef8422>] [<ffffffffa0ef8422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b31e3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b31e3890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a9452ef8 R13: 0000000000000000 R14: ffff8802c8d9d698 R15: ffff8802c9c3bde8 FS: 0000000000000000(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ceca6000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f1dc40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1e01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da1810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dad4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e0b5df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e12240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e08bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e11930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0f831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06c7782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa027b545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa027e7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f1bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0776414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077b28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03479be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072a480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa072c139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa072b560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-154.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa133d422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 267b6f067 PUD 2635a9067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata serio_raw virtio_blk floppy i2c_core [last unloaded: libcfs] CPU: 10 PID: 12417 Comm: mdt05_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880271ae3760 ti: ffff8800a8914000 task.ti: ffff8800a8914000 RIP: 0010:[<ffffffffa133d422>] [<ffffffffa133d422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800a8917848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800a8917890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b3dfb8e8 R13: 0000000000000000 R14: ffff8802bd3c4548 R15: ffff8802cca05298 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000269566000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1362c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa136301b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e6810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f24e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa12505df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1257240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa124dbdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1256930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1254831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05ef782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0234545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02377e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0619bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa069e414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a328a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02619be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0652480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0654139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0653560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-184.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.186@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1360422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw libata virtio_blk i2c_core [last unloaded: libcfs] CPU: 2 PID: 28939 Comm: mdt01_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8800b9205c40 ti: ffff880324b64000 task.ti: ffff880324b64000 RIP: 0010:[<ffffffffa1360422>] [<ffffffffa1360422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880324b67848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880324b67890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802e554b8e8 R13: 0000000000000000 R14: ffff8800a4b922a8 R15: ffff8802b14f9298 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1385c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138601b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa1209810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa12154e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa12735df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127a240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1270bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1279930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1277831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05ee782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022c545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa022f7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0618bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa069d414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a228a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02599be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0651480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0653139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0652560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-139.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.141@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-139.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-139.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-139.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1338422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks virtio_blk libata serio_raw floppy i2c_core [last unloaded: libcfs] CPU: 6 PID: 22289 Comm: mdt03_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802683e49d0 ti: ffff880099e18000 task.ti: ffff880099e18000 RIP: 0010:[<ffffffffa1338422>] [<ffffffffa1338422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880099e1b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880099e1b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802798dd6b8 R13: 0000000000000000 R14: ffff88009ab20008 R15: ffff88009c1fb9c8 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031e67c000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135dc40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135e01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e1810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ed4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa124b5df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1252240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1248bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1251930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124f831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0682782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0236545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02397e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06acbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0731414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa073628a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03319be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06e5480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06e7139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa06e6560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-159.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.161@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: Setting parameter lustre-MDT0002.mdt.identity_upcall in log lustre-MDT0002 Lustre: srv-lustre-MDT0002: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0002: new disk, initializing Lustre: lustre-MDT0002: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2:mdt Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2:mdt] Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:3e8:ost] | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0c20422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2a6ef1067 PUD 31f056067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic ttm drm pata_acpi drm_panel_orientation_quirks ata_piix virtio_blk serio_raw i2c_core floppy libata [last unloaded: libcfs] CPU: 0 PID: 13136 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880329b89280 ti: ffff8802f5788000 task.ti: ffff8802f5788000 RIP: 0010:[<ffffffffa0c20422>] [<ffffffffa0c20422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802f578b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802f578b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802eade8098 R13: 0000000000000000 R14: ffff88029e775698 R15: ffff8802f02c6528 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002f0afe000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0c45c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0c4601b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0b60810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0b6c4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f1c5df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f23240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f19bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f22930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f20831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0cc7782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0261545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02647e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0cf1bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0d76414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0d7b28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa032d9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0d2a480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0d2c139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0d2b560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-239.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-239.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-239.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-239.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1335422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2dbc59067 PUD 27e138067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks floppy ata_piix serio_raw virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 8 PID: 27274 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802b93b9280 ti: ffff8802cd534000 task.ti: ffff8802cd534000 RIP: 0010:[<ffffffffa1335422>] [<ffffffffa1335422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802cd537848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802cd537890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a345b8e8 R13: 0000000000000000 R14: ffff880293621158 R15: ffff8800a2d68a58 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002a4322000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135ac40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135b01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11de810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ea4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa12485df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa124f240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1245bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa124e930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124c831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0606782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025b545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa025e7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0630bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06b5414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ba28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02889be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0669480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa066b139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa066a560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-159.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.161@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1355422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_console virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic ttm pata_acpi drm drm_panel_orientation_quirks ata_piix floppy libata virtio_blk i2c_core serio_raw [last unloaded: libcfs] CPU: 14 PID: 10674 Comm: mdt07_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802a2d849d0 ti: ffff88031ca18000 task.ti: ffff88031ca18000 RIP: 0010:[<ffffffffa1355422>] [<ffffffffa1355422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031ca1b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031ca1b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a7ae0098 R13: 0000000000000000 R14: ffff8802947b4548 R15: ffff88008ba45298 FS: 0000000000000000(0000) GS:ffff880331d80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000bba60000 CR4: 00000000000007e0 Call Trace: [<ffffffffa137ac40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137b01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11fe810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa120a4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa12685df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126f240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1265bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126e930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126c831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa066c782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0246545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02497e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0696bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa071b414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa072028a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa030f9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06cf480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06d1139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06d0560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-184.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.186@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-184.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f10422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 98071067 PUD 98072067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm drm_panel_orientation_quirks floppy virtio_blk serio_raw libata i2c_core [last unloaded: libcfs] CPU: 12 PID: 13880 Comm: mdt06_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802db29c9d0 ti: ffff8802dc018000 task.ti: ffff8802dc018000 RIP: 0010:[<ffffffffa0f10422>] [<ffffffffa0f10422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802dc01b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802dc01b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802dbbdb248 R13: 0000000000000000 R14: ffff8800bb36d698 R15: ffff8802d1a46318 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000009bb7c000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f35c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f3601b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0db9810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dc54e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e235df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e2a240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e20bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e29930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e27831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa073d782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0272545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02757e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0767bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07ec414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07f128a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03ec9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07a0480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07a2139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07a1560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-209.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa134d422>] lod_fill_mirrors+0x192/0x620 [lod] PGD ae542067 PUD b5284067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm libata drm_panel_orientation_quirks floppy serio_raw virtio_blk i2c_core [last unloaded: libcfs] CPU: 10 PID: 29598 Comm: mdt05_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802da8149d0 ti: ffff88009d440000 task.ti: ffff88009d440000 RIP: 0010:[<ffffffffa134d422>] [<ffffffffa134d422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009d443848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009d443890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88027ccf60a8 R13: 0000000000000000 R14: ffff8800af54a2a8 R15: ffff8800a8575298 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000008d446000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1372c40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137301b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11dc810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11e84e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa12605df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1267240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa125dbdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1266930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1264831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0648782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022f545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02327e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0672bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06f7414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06fc28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02f79be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06ab480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06ad139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06ac560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-194.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.6_RC1_19_g064c776 LNet: Added LNI 192.168.123.196@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-194.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-194.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-194.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef8422>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2bde89067 PUD 2b6d8f067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw i2c_core libata virtio_blk [last unloaded: libcfs] CPU: 8 PID: 11364 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8800bbb43760 ti: ffff88009ea6c000 task.ti: ffff88009ea6c000 RIP: 0010:[<ffffffffa0ef8422>] [<ffffffffa0ef8422>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009ea6f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009ea6f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880098e43248 R13: 0000000000000000 R14: ffff880099ad5698 R15: ffff88009f7d7de8 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002e9c3a000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f1dc40>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1e01b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da1810>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dad4e1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e0b5df>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e12240>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e08bdd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e11930>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0f831>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0707782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0263545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02667e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0731bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07b6414>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07bb28a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa032f9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa076a480>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa076c139>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa076b560>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-234.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-234.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-234.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-234.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-234.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: Skipped 4 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-234.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1345fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2cf392067 PUD 319e49067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk floppy serio_raw i2c_core libata [last unloaded: libcfs] CPU: 6 PID: 19972 Comm: mdt03_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802f0773760 ti: ffff8802fb988000 task.ti: ffff8802fb988000 RIP: 0010:[<ffffffffa1345fb2>] [<ffffffffa1345fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802fb98b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802fb98b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88026f27d368 R13: 0000000000000000 R14: ffff880298bb5698 R15: ffff88027eeb1298 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002cbf02000 CR4: 00000000000007e0 Call Trace: [<ffffffffa136b230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa136b60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11f0760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11fc431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa125953f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa125fa00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1256b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa125f0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa125cff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa060e782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022d545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02307e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0638bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06bd3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06c221a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02b29be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0671410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06730c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06724f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-139.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.141@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-139.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-139.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-139.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0b58fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 8fc2c067 PUD 99f8b067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_balloon virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk floppy libata i2c_core [last unloaded: libcfs] CPU: 2 PID: 26065 Comm: mdt01_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff88031e483760 ti: ffff8802b8ee8000 task.ti: ffff8802b8ee8000 RIP: 0010:[<ffffffffa0b58fb2>] [<ffffffffa0b58fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b8eeb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b8eeb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880090840dd8 R13: 0000000000000000 R14: ffff8802c9325698 R15: ffff8802e5b439c8 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000008dd78000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0b7e230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0b7e60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0af6760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0b02431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f0e53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f14a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f0bb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f140f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f11ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0cb9782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0258545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa025b7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0ce3bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0d683a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0d6d21a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03669be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0d1c410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0d1e0c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0d1d4f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-204.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0efefb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw virtio_blk libata i2c_core [last unloaded: libcfs] CPU: 6 PID: 18216 Comm: mdt03_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880329025c40 ti: ffff8802cb104000 task.ti: ffff8802cb104000 RIP: 0010:[<ffffffffa0efefb2>] [<ffffffffa0efefb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802cb107848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802cb107890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8803257fc628 R13: 0000000000000000 R14: ffff88009e2d5698 R15: ffff8802d0354638 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f24230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f2460b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da9760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0db5431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e1253f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e18a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e0fb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e180f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e15ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06c1782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa026f545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02727e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06ebbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07703a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077521a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa033b9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0724410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07260c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07254f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-209.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2:mdt] Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: Skipped 4 previous similar messages Lustre: DEBUG MARKER: centos-209.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1350fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm serio_raw ata_piix drm_panel_orientation_quirks libata floppy i2c_core virtio_blk [last unloaded: libcfs] CPU: 10 PID: 17562 Comm: mdt05_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff88026855dc40 ti: ffff8802c07ac000 task.ti: ffff8802c07ac000 RIP: 0010:[<ffffffffa1350fb2>] [<ffffffffa1350fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802c07af848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802c07af890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880321312858 R13: 0000000000000000 R14: ffff8802649f4548 R15: ffff8802c3871298 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1376230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137660b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e1760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ed431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126453f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126aa00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1261b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126a0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1267ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0617782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022b545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa022e7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0641bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c63a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cb21a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02589be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067a410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067c0c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067b4f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-159.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.161@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-159.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee9fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2c3dfd067 PUD 31ebc2067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console pcspkr virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic ttm pata_acpi drm drm_panel_orientation_quirks ata_piix virtio_blk serio_raw floppy libata i2c_core [last unloaded: libcfs] CPU: 4 PID: 31034 Comm: mdt02_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff88031fe49280 ti: ffff8802e44e8000 task.ti: ffff8802e44e8000 RIP: 0010:[<ffffffffa0ee9fb2>] [<ffffffffa0ee9fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802e44eb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802e44eb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802abd12ef8 R13: 0000000000000000 R14: ffff88032328d698 R15: ffff8800a3012318 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031d55e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f0f230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0f60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d94760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da0431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dfd53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e03a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfab3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e030f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e00ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06f4782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025d545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02607e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa071ebc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07a33a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07a821a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03769be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0757410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07590c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07584f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-174.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-174.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-174.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-174.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa131afb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon pcspkr i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata virtio_blk floppy i2c_core serio_raw [last unloaded: libcfs] CPU: 2 PID: 2616 Comm: mdt01_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802c0029280 ti: ffff88031f198000 task.ti: ffff88031f198000 RIP: 0010:[<ffffffffa131afb2>] [<ffffffffa131afb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031f19b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031f19b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88028805c978 R13: 0000000000000000 R14: ffff8802ab7f4548 R15: ffff8800a2f59298 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1340230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa134060b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ab760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11b7431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa122e53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1234a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa122bb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12340f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1231ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05da782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0244545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02477e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0604bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06893a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa068e21a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02719be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa063d410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa063f0c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa063e4f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-179.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.181@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-179.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-179.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-179.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa135efb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata serio_raw virtio_blk i2c_core floppy [last unloaded: libcfs] CPU: 0 PID: 22922 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8800974cc9d0 ti: ffff880297814000 task.ti: ffff880297814000 RIP: 0010:[<ffffffffa135efb2>] [<ffffffffa135efb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880297817848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880297817890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880287f0da08 R13: 0000000000000000 R14: ffff880263664548 R15: ffff8800af111298 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa1384230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138460b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa1209760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1215431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa127253f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1278a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa126fb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12780f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1275ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0610782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0227545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa022a7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa063abc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06bf3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06c421a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02549be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0673410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06750c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06744f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-154.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.156@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-154.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1334fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 3213b0067 PUD 2d39a1067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw libata i2c_core virtio_blk [last unloaded: libcfs] CPU: 5 PID: 1065 Comm: mdt02_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff88028d978010 ti: ffff8802d3e14000 task.ti: ffff8802d3e14000 RIP: 0010:[<ffffffffa1334fb2>] [<ffffffffa1334fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802d3e17848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802d3e17890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802b71b1128 R13: 0000000000000000 R14: ffff8802d5b933f8 R15: ffff8802f0d9d298 FS: 0000000000000000(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ceb9e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135a230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135a60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11df760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11eb431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa124853f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa124ea00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1245b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa124e0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124bff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa064f782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa023b545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa023e7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0679bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06fe3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa070321a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02689be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06b2410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06b40c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06b34f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-194.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.196@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-194.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-194.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-194.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeffb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2f19a6067 PUD 2b43f6067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy virtio_blk libata serio_raw i2c_core [last unloaded: libcfs] CPU: 14 PID: 28747 Comm: mdt07_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880324c2b760 ti: ffff8802bd210000 task.ti: ffff8802bd210000 RIP: 0010:[<ffffffffa0eeffb2>] [<ffffffffa0eeffb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bd213848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bd213890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802e66cea98 R13: 0000000000000000 R14: ffff8800b8e6d698 R15: ffff8802e3a898c8 FS: 0000000000000000(0000) GS:ffff880331d80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002e5c8c000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f15230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1560b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9a760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da6431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e0353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e09a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e00b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e090f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e06ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06fe782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0276545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02797e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0728bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07ad3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07b221a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03809be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0761410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07630c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07624f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-204.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1334fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk serio_raw floppy libata i2c_core [last unloaded: libcfs] CPU: 14 PID: 23098 Comm: mdt07_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802bf360010 ti: ffff8802b86e8000 task.ti: ffff8802b86e8000 RIP: 0010:[<ffffffffa1334fb2>] [<ffffffffa1334fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b86eb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b86eb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88029b324cc8 R13: 0000000000000000 R14: ffff8802af2e5698 R15: ffff8802fc070a58 FS: 0000000000000000(0000) GS:ffff880331d80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031a930000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135a230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135a60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11df760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11eb431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa124853f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa124ea00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1245b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa124e0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124bff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0610782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0267545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa026a7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa063abc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06bf3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06c421a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02949be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0673410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06750c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06744f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-164.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.166@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-164.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-164.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-164.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1342fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm drm_panel_orientation_quirks serio_raw ata_piix floppy virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 14 PID: 14221 Comm: mdt07_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802bcf5c9d0 ti: ffff8802bad14000 task.ti: ffff8802bad14000 RIP: 0010:[<ffffffffa1342fb2>] [<ffffffffa1342fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bad17848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bad17890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802b0fe7b28 R13: 0000000000000000 R14: ffff8802fba922a8 R15: ffff8802ee704a58 FS: 0000000000000000(0000) GS:ffff880331d80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000ba8b6000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1368230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa136860b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ed760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f9431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa125653f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa125ca00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1253b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa125c0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1259ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f6782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0234545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02377e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0620bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a53a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06aa21a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02619be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0659410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa065b0c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa065a4f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-219.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.221@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-219.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-219.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-219.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa135efb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2f0da1067 PUD 2ae021067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata virtio_blk serio_raw i2c_core floppy [last unloaded: libcfs] CPU: 4 PID: 316 Comm: mdt02_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802d33a3760 ti: ffff8802f0954000 task.ti: ffff8802f0954000 RIP: 0010:[<ffffffffa135efb2>] [<ffffffffa135efb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802f0957848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802f0957890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802f071dd58 R13: 0000000000000000 R14: ffff88009b240008 R15: ffff8800a2df9298 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ae02c000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1384230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138460b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa1209760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1215431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa127253f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1278a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa126fb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12780f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1275ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa064e782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0246545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02497e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0678bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06fd3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa070221a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02739be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06b1410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06b30c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa06b24f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-229.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.231@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-229.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-229.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-229.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1044fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper i2c_piix4 cryptd i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 5379 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 task: ffff8801368ce660 ti: ffff880070f74000 task.ti: ffff880070f74000 RIP: 0010:[<ffffffffa1044fb2>] [<ffffffffa1044fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880070f77848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880070f77890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff880134b2a090 R13: 0000000000000000 R14: ffff88009a367000 R15: ffff880099271d80 FS: 0000000000000000(0000) GS:ffff88013e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b19a8000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa106a230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa106a60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0eef760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0efb431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f5853f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f5ea00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f55b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f5e0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f5bff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07f2782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa02d6545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02d97e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa081cbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa08a13d4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a624a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa033c9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0855440>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa08570f9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0856520>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 28524:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1732041378 with bad export cookie 10569538946949012454 LustreError: 29492:0:(osp_object.c:629:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000bd2:0x1:0x0]: rc = -5 LustreError: 29492:0:(client.c:1256:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff88009aabbdc0 x1816176882645056/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' LustreError: 29492:0:(llog_cat.c:753:llog_cat_cancel_arr_rec()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 llog-records: rc = -5 LustreError: 29492:0:(llog_cat.c:789:llog_cat_cancel_records()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 of 1 llog-records: rc = -5 LustreError: 166-1: MGC192.168.201.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28524:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg126-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 495:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.126@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg126-server.virtnet: executing load_modules_local libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.5_41_g3e84673 LNet: Added LNI 192.168.201.126@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f1cfb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 319073067 PUD 294d15067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic ttm pata_acpi drm drm_panel_orientation_quirks ata_piix virtio_blk i2c_core serio_raw floppy libata [last unloaded: libcfs] CPU: 12 PID: 30775 Comm: mdt06_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802d6523760 ti: ffff8802b19bc000 task.ti: ffff8802b19bc000 RIP: 0010:[<ffffffffa0f1cfb2>] [<ffffffffa0f1cfb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b19bf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b19bf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880295035018 R13: 0000000000000000 R14: ffff8802ac4d5698 R15: ffff88031f1039c8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002f60da000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f42230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f4260b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0dc7760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dd3431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e3053f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e36a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e2db3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e360f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e33ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06de782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0276545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02797e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0708bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa078d3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa079221a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03609be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0741410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07430c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07424f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-124.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-124.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-124.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-124.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1333fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 pcspkr virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk floppy libata i2c_core [last unloaded: libcfs] CPU: 12 PID: 28482 Comm: mdt06_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802f54e49d0 ti: ffff880293b50000 task.ti: ffff880293b50000 RIP: 0010:[<ffffffffa1333fb2>] [<ffffffffa1333fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880293b53848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880293b53890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88028a35e0a8 R13: 0000000000000000 R14: ffff8802cb9a67e8 R15: ffff88028fb8d298 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1359230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135960b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11de760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ea431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa124753f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa124da00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1244b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa124d0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124aff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0643782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0243545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02467e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa066dbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06f23a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06f721a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02709be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06a6410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06a80c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06a74f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-219.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.221@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-219.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-219.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-219.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f2bfb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic pata_acpi drm_panel_orientation_quirks ata_piix serio_raw floppy virtio_blk libata i2c_core [last unloaded: libcfs] CPU: 9 PID: 10619 Comm: mdt04_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff88031b358010 ti: ffff8802dccd8000 task.ti: ffff8802dccd8000 RIP: 0010:[<ffffffffa0f2bfb2>] [<ffffffffa0f2bfb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802dccdb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802dccdb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802b1019478 R13: 0000000000000000 R14: ffff8802c247d698 R15: ffff8802b4692d68 FS: 0000000000000000(0000) GS:ffff880331c40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f51230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f5160b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0dd6760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0de2431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e3f53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e45a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e3cb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e450f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e42ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06fc782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0273545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02767e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0726bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07ab3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07b021a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03ab9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa075f410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07610c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07604f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-169.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-169.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-169.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-169.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0c2bfb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm serio_raw floppy virtio_blk libata drm_panel_orientation_quirks i2c_core [last unloaded: libcfs] CPU: 0 PID: 31262 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880320860010 ti: ffff8802ac35c000 task.ti: ffff8802ac35c000 RIP: 0010:[<ffffffffa0c2bfb2>] [<ffffffffa0c2bfb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ac35f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ac35f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800aedf42d8 R13: 0000000000000000 R14: ffff8802a31a5698 R15: ffff8802f54f6948 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000319f0a000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0c51230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0c5160b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0a77760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0a83431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f2853f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f2ea00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f25b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f2e0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f2bff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0cd3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025d545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02607e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0cfdbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0d823a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0d8721a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03299be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0d36410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0d380c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0d374f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-124.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-124.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-124.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-124.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa137afb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2caebe067 PUD 2fc22f067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console pcspkr i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw libata i2c_core [last unloaded: libcfs] CPU: 12 PID: 28885 Comm: mdt06_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff8802ef628010 ti: ffff8802ce634000 task.ti: ffff8802ce634000 RIP: 0010:[<ffffffffa137afb2>] [<ffffffffa137afb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ce637848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ce637890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802c9e60a88 R13: 0000000000000000 R14: ffff880298799158 R15: ffff8802c7805298 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002f258e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa13a0230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa13a060b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa1225760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1231431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa128e53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1294a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa128bb3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12940f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1291ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa061b782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022f545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02327e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0645bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06ca3a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cf21a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02ca9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067e410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06800c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067f4f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-204.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.206@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-204.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef6fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_console virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi virtio_blk drm_panel_orientation_quirks ata_piix serio_raw floppy i2c_core libata [last unloaded: libcfs] CPU: 10 PID: 1832 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff88031aea1280 ti: ffff8802a25e4000 task.ti: ffff8802a25e4000 RIP: 0010:[<ffffffffa0ef6fb2>] [<ffffffffa0ef6fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802a25e7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802a25e7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802c3af97c8 R13: 0000000000000000 R14: ffff8802a5d35698 R15: ffff88029c4c7de8 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031e4fc000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f1c230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1c60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da1760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dad431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0e0a53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e10a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e07b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e100f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0dff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0732782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa026c545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa026f7e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa075cbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07e13a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07e621a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03e19be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0795410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07970c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07964f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-189.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: Skipped 4 previous similar messages Lustre: DEBUG MARKER: centos-189.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:3e8:ost] | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1354fb2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 27b06d067 PUD 27ff14067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks ata_piix floppy virtio_blk libata serio_raw i2c_core [last unloaded: libcfs] CPU: 15 PID: 9453 Comm: mdt07_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 task: ffff880268ec8010 ti: ffff8802aa554000 task.ti: ffff8802aa554000 RIP: 0010:[<ffffffffa1354fb2>] [<ffffffffa1354fb2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802aa557848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802aa557890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802bd2bea98 R13: 0000000000000000 R14: ffff8800babfa2a8 R15: ffff880288cbd298 FS: 0000000000000000(0000) GS:ffff880331dc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002a13aa000 CR4: 00000000000007e0 Call Trace: [<ffffffffa137a230>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137a60b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ff760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa120b431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126853f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126ea00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1265b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126e0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126bff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f5782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0232545>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02357e6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061fbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a43a4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a921a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa025f9be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0658410>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa065a0c9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06594f0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-169.localnet: executing set_hostid libcfs: HW NUMA nodes: 1, HW CPU cores: 16, npartitions: 8 Lustre: Lustre: Build Version: 2.15.5_42_g9aa2620 LNet: Added LNI 192.168.123.171@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-169.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-169.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-169.localnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1045ea2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zzstd(O) zlua(O) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel squashfs lrw gf128mul glue_helper ablk_helper i2c_piix4 cryptd i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 5195 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 task: ffff8800959fa220 ti: ffff880095ab4000 task.ti: ffff880095ab4000 RIP: 0010:[<ffffffffa1045ea2>] [<ffffffffa1045ea2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880095ab7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880095ab7890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff880097256290 R13: 0000000000000000 R14: ffff8800ac841000 R15: ffff88006f9e3b40 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b519a000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa106b120>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa106b4fb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0ef0760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0efc431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f5953f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f5fa00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f56b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f5f0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f5cff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07f3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa031b585>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa031e826>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa081dbc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa08a23d4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a724a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03489be>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0856440>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa08580f9>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0857520>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 31561:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1731097127 with bad export cookie 5368110496133853315 LustreError: 166-1: MGC192.168.201.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31561:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: oleg126-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 346:0:(lib-ptl.c:992:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.126@tcp alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg126-server.virtnet: executing load_modules_local LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 Lustre: Lustre: Build Version: 2.15.5_16_g50c26cc LNet: Added LNI 192.168.201.126@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg126-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg126-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa145fea2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 22090 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff880130993330 ti: ffff88009e2e4000 task.ti: ffff88009e2e4000 RIP: 0010:[<ffffffffa145fea2>] [<ffffffffa145fea2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009e2e7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009e2e7890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88009cf14c90 R13: 0000000000000000 R14: ffff88012eacf000 R15: ffff8800a17a3c00 FS: 0000000000000000(0000) GS:ffff88013e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa1485050>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa148542b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa12f1760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa12fd431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa137353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1379a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1370b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa13790f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1376ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07ec782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa02bee35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02c20d6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0816bc6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa089b334>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a01aa>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03b98fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa084f3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0851059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0850480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 22575:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392442/real 1715392442] req@ffff88013711ed40 x1798719268728064/t0(0) o251->MGC192.168.204.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1715392448 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg407-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 23209:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.107@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg407-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_30_g03ef6c3 LNet: Added LNI 192.168.204.107@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 27250:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392506/real 1715392506] req@ffff88013711d0c0 x1798719371482752/t0(0) o251->MGC192.168.204.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1715392512 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg407-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 27882:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.107@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg407-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_30_g03ef6c3 LNet: Added LNI 192.168.204.107@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: server umount lustre-OST0000 complete Lustre: 31652:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392563/real 1715392563] req@ffff880132592ac0 x1798719438590912/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1715392569 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 31652:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392569/real 1715392569] req@ffff88009e522ac0 x1798719438591168/t0(0) o251->MGC192.168.204.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1715392575 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg407-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32325:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.107@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg407-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_30_g03ef6c3 LNet: Added LNI 192.168.204.107@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 5236:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392636/real 1715392636] req@ffff880132590980 x1798719504652032/t0(0) o251->MGC192.168.204.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1715392642 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg407-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 5886:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.107@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg407-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_30_g03ef6c3 LNet: Added LNI 192.168.204.107@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 10054:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392693/real 1715392693] req@ffff88009ea604c0 x1798719573857792/t0(0) o251->MGC192.168.204.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1715392699 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg407-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 10697:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.107@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg407-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_30_g03ef6c3 LNet: Added LNI 192.168.204.107@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 14677:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715392742/real 1715392742] req@ffff88013711df00 x1798719630480384/t0(0) o251->MGC192.168.204.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1715392748 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg407-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 15336:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.107@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg407-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_30_g03ef6c3 LNet: Added LNI 192.168.204.107@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg407-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg407-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ff8ea2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata [last unloaded: libcfs] CPU: 3 PID: 5078 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff8800b4ff2220 ti: ffff88009cd24000 task.ti: ffff88009cd24000 RIP: 0010:[<ffffffffa0ff8ea2>] [<ffffffffa0ff8ea2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009cd27848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009cd27890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff880094586690 R13: 0000000000000000 R14: ffff880129d0e000 R15: ffff88007f41f540 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b41fe000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa101e050>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa101e42b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0ea3760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0eaf431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0f0c53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0f12a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0f09b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0f120f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0f0fff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07f5782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0230e35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02340d6>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa081fbe6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa08a4354>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08a91ca>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa032e8fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa08583c0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa085a079>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa08594a0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 28308:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1713950972 with bad export cookie 6864983749103094134 LustreError: 166-1: MGC192.168.202.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28308:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg246-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32685:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.202.146@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg246-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_17_g4937c3c LNet: Added LNI 192.168.202.146@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg246-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg246-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0fe2ea2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) dm_flakey dm_mod crc_t10dif crct10dif_generic rpcsec_gss_krb5 sb_edac edac_core iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel squashfs aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata serio_raw [last unloaded: libcfs] CPU: 2 PID: 6135 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 task: ffff8800b5229110 ti: ffff8800923f8000 task.ti: ffff8800923f8000 RIP: 0010:[<ffffffffa0fe2ea2>] [<ffffffffa0fe2ea2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800923fb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800923fb890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff880072b0c490 R13: 0000000000000000 R14: ffff880079bdd000 R15: ffff88012d681000 FS: 0000000000000000(0000) GS:ffff88013e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000b19bc000 CR4: 0000000000160fe0 Call Trace: [<ffffffffa1008050>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa100842b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0b7f760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0b8b431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0ef653f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0efca00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0ef3b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0efc0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0ef9ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa07fc782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0233e35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0237066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0826be6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa08aaeb4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa08b339a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03308fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa085f3c0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0861079>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff81410a79>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa08604a0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message LustreError: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 28298:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1712050262 with bad export cookie 5691739610644657063 LustreError: 166-1: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28298:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32717:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.136@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4 LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed7ea2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 5752 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff88012a2db330 ti: ffff88012e968000 task.ti: ffff88012e968000 RIP: 0010:[<ffffffffa0ed7ea2>] [<ffffffffa0ed7ea2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88012e96b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88012e96b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88012efa2e90 R13: 0000000000000000 R14: ffff8800933c5000 R15: ffff88009ed8cb40 FS: 0000000000000000(0000) GS:ffff88013e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0efd050>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efd42b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d83760>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d8f431>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dec53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df2a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de9b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df20f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0defff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05fb782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa019de35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01a1066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0625be6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a9ed4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06b23ba>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e28fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa065e3e0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0660099>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa065f4c0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 31579:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1700735379 with bad export cookie 2016394085790962610 LustreError: 166-1: MGC192.168.203.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31579:0:(ldlm_lockd.c:2521:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg346-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32452:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.146@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg346-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.4_RC1 LNet: Added LNI 192.168.203.146@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa134edf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 20906 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 task: ffff880131b51110 ti: ffff88008c304000 task.ti: ffff88008c304000 RIP: 0010:[<ffffffffa134edf2>] [<ffffffffa134edf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008c307848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008c307890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8801348de890 R13: 0000000000000000 R14: ffff880091466000 R15: ffff8800a0e390c0 FS: 0000000000000000(0000) GS:ffff88013e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001368fa000 CR4: 00000000000006e0 Call Trace: [<ffffffffa1373e80>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137425b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e1730>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ed401>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1269a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1260b3d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12690f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1266ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05c9782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01c0e35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01c4066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05f3be6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0677ed4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06803ba>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02008fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa062c3e0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa062e099>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa062d4c0>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 16150:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438563/real 1699438563] req@ffff880093249900 x1781990429829696/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438569 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 16806:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 20895:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438600/real 1699438600] req@ffff88009e15a300 x1781990507418304/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438606 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21545:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 25390:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438632/real 1699438632] req@ffff8800a1771e00 x1781990545166720/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438638 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 26051:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 31355:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438672/real 1699438672] req@ffff88009e158000 x1781990580818880/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438678 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32033:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 3848:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438708/real 1699438708] req@ffff88009b874b00 x1781990620664704/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438714 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 4510:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 8581:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1699438740/real 1699438740] req@ffff88009e15f800 x1781990659461504/t0(0) o251->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1699438746 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 9255:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.204.160@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_36_ga9e47f3 LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa130cdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 1 PID: 19007 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800ba6fd550 ti: ffff8800bfc44000 task.ti: ffff8800bfc44000 RIP: 0010:[<ffffffffa130cdf2>] [<ffffffffa130cdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800bfc47848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800bfc47890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b5129890 R13: 0000000000000000 R14: ffff8800aa88f000 R15: ffff8800ab4539c0 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000006e0 Call Trace: [<ffffffffa1331e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa133222b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ba6f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11c63c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa122353f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1229a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1220acd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa12290f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1226ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05b0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0171e35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0175066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05da986>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa065ec74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa066715a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01c688e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0613180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0614e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0614260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 15700:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425490/real 1687425490] req@ffff8800bb88d580 x1769393806649984/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425496 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 16354:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 20426:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425527/real 1687425527] req@ffff8800aec27200 x1769393886335872/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425533 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21064:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 mount.lustre (23578) used greatest stack depth: 9448 bytes left Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 24878:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425560/real 1687425560] req@ffff8800aec276c0 x1769393926181312/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425566 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 25540:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 30824:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425601/real 1687425601] req@ffff8800bd5b3900 x1769393962881984/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425607 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 31500:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 3223:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425639/real 1687425639] req@ffff8800aec24280 x1769394003776384/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425645 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 3878:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 7811:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1687425672/real 1687425672] req@ffff8800bd5b6880 x1769394043621824/t0(0) o251->MGC192.168.203.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1687425678 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg301-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 8477:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.101@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg301-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3 LNet: Added LNI 192.168.203.101@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg301-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed5df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 1 PID: 6893 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff880115e02220 ti: ffff880084268000 task.ti: ffff880084268000 RIP: 0010:[<ffffffffa0ed5df2>] [<ffffffffa0ed5df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008426b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008426b890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800ad2f2a90 R13: 0000000000000000 R14: ffff8800c970f000 R15: ffff8800b7948240 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000011738c000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0efae50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efb22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d836f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d8f3c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dec53f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df2a00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de9acd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df20f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0defff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05e2782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01dbe35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01df066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa060c986>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0690c74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069915a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa021b88e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0645180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0646e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0646260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 28180:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1686614353 with bad export cookie 3854249869916708303 LustreError: 166-1: MGC192.168.203.171@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28180:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg371-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32592:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.171@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg371-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.3_RC1_1_gdef45fb LNet: Added LNI 192.168.203.171@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg371-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg371-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa131fdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 14061 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800c87ae660 ti: ffff8800b97b0000 task.ti: ffff8800b97b0000 RIP: 0010:[<ffffffffa131fdf2>] [<ffffffffa131fdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b97b3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b97b3890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bf952a90 R13: 0000000000000000 R14: ffff88010e3ee000 R15: ffff88010bd50480 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d384e000 CR4: 00000000000006f0 Call Trace: [<ffffffffa1344e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa134522b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11cd6b0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11d9381>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa123653f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa123ca00>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1233acd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa123c0f0>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1239ff1>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05c1782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01bae35>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01be066>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05eb986>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa066fc74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa067815a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01fa88e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0624180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0625e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0625260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 14595:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989449/real 1685989449] req@ffff8800b15597c0 x1767888014813248/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1685989455 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 14595:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989455/real 1685989455] req@ffff8800b89e63c0 x1767888014813568/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989461 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 15252:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 19305:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989492/real 1685989492] req@ffff8800aa03b900 x1767888095548096/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989498 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 19947:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 23780:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989524/real 1685989524] req@ffff8800b155d580 x1767888134345088/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989530 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 24434:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 29674:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989561/real 1685989561] req@ffff8800b155f200 x1767888168948416/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1685989567 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 29674:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989567/real 1685989567] req@ffff8800b85de880 x1767888168948736/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989573 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 30335:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 2063:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989605/real 1685989605] req@ffff8800b89e1c80 x1767888212988800/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989611 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 2711:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 6706:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1685989638/real 1685989638] req@ffff8800b89e17c0 x1767888251785664/t0(0) o251->MGC192.168.201.120@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1685989644 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg220-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 7364:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.120@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg220-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.2_91_g21c66f4 LNet: Added LNI 192.168.201.120@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg220-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg220-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1352df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 1 PID: 11522 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800c6836660 ti: ffff8800b5244000 task.ti: ffff8800b5244000 RIP: 0010:[<ffffffffa1352df2>] [<ffffffffa1352df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b5247848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b5247890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b7dc3290 R13: 0000000000000000 R14: ffff88010ed4c000 R15: ffff8800ae686f00 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000006e0 Call Trace: [<ffffffffa1377e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137822b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e75f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f32c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa126955f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126fa20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1266b5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126f110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126d011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05d0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01c9dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01cd006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05faa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa067f014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa068953a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa020982e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0633180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0634e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0634260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: server umount lustre-OST0000 complete Lustre: 11938:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766873/real 1668766873] req@ffff880117175f00 x1749828831816256/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1668766879 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: 11938:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766879/real 1668766879] req@ffff8800b19ccc00 x1749828831816512/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766885 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 12586:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 16676:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766917/real 1668766917] req@ffff8800b19cd580 x1749828915696512/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766923 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 17304:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 21128:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766950/real 1668766950] req@ffff8800b19cf6c0 x1749828955541888/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766956 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21805:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 27068:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668766989/real 1668766989] req@ffff8800b19cbdc0 x1749828991193984/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668766995 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 27712:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 31880:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668767028/real 1668767028] req@ffff8800bfcf7200 x1749829031039872/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668767034 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 32523:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 4049:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1668767060/real 1668767060] req@ffff8800b19cb440 x1749829070885312/t0(0) o251->MGC192.168.203.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1668767066 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 4720:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.144@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed6df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 0 PID: 7786 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800a20c8000 ti: ffff8800a9270000 task.ti: ffff8800a9270000 RIP: 0010:[<ffffffffa0ed6df2>] [<ffffffffa0ed6df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800a9273848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800a9273890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bc953890 R13: 0000000000000000 R14: ffff8800982ec000 R15: ffff8800bc923480 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d34da000 CR4: 00000000000006f0 Call Trace: [<ffffffffa0efbe50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efc22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d845f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d902c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0ded55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df3a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0deab5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df3110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df1011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05fe782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01a7dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01ab006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0628a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06ad014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06b753a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e782e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0661180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0662e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0662260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete LustreError: 31559:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1668335394 with bad export cookie 5395720967403707116 LustreError: 166-1: MGC192.168.201.173@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31559:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg273-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 345:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.173@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg273-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.201.173@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg273-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg273-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 8144 Comm: mdt00_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff88010b20e660 ti: ffff88009ac60000 task.ti: ffff88009ac60000 RIP: 0010:[<ffffffffa0ee8df2>] [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009ac63848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009ac63890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88009f4a9490 R13: 0000000000000000 R14: ffff8800c3ee8000 R15: ffff8800c59f2c00 FS: 0000000000000000(0000) GS:ffff88011e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d63fc000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0f0de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d965f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da22c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dff55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e05a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfcb5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e05110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e03011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01ecdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01f0006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061da06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a2014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ac53a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa022c82e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0656180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0657e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0657260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LustreError: 28742:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1668249073 with bad export cookie 1270199893924624021 LustreError: 166-1: MGC192.168.201.177@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28742:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: DEBUG MARKER: oleg277-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 711:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.177@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg277-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_47_g29aa9f6 LNet: Added LNI 192.168.201.177@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg277-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg277-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 5621 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800cf64a220 ti: ffff880083a2c000 task.ti: ffff880083a2c000 RIP: 0010:[<ffffffffa0ee8df2>] [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880083a2f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880083a2f890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800c4a01e90 R13: 0000000000000000 R14: ffff88009362c000 R15: ffff8800c4a70780 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d6fe4000 CR4: 00000000000006f0 Call Trace: [<ffffffffa0f0de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d965f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da22c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0dff55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e05a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfcb5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e05110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e03011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01a9dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01ad006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061aa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa069f014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a953a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01f982e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0653180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0654e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0654260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 28794:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1668111697 with bad export cookie 12292860998847032263 LustreError: 166-1: MGC192.168.203.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28794:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg346-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 765:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.146@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg346-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_RC1_57_gad04e3f LNet: Added LNI 192.168.203.146@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg346-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg346-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 1 PID: 9329 Comm: mdt00_005 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800b59dc440 ti: ffff8800badd8000 task.ti: ffff8800badd8000 RIP: 0010:[<ffffffffa0edddf2>] [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800baddb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800baddb890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800b4a78490 R13: 0000000000000000 R14: ffff8800b896b000 R15: ffff88008819f6c0 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d5952000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0f02e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0322b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d8b5f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d972c1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0df455f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dfaa20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df1b5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0dfa110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df8011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05ea782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01e3dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01e7006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0614a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0699014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06a353a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa022382e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa064d180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa064ee39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa064e260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 31077:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1666258324 with bad export cookie 464433880997223941 LustreError: 166-1: MGC192.168.201.164@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31077:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: DEBUG MARKER: oleg264-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 617:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.201.164@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg264-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_36_g9d1805c LNet: Added LNI 192.168.201.164@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg264-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg264-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 3 PID: 5556 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800cd014440 ti: ffff8800c9204000 task.ti: ffff8800c9204000 RIP: 0010:[<ffffffffa0ed8df2>] [<ffffffffa0ed8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800c9207848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800c9207890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800c0b4ae90 R13: 0000000000000000 R14: ffff8800c3efe000 R15: ffff8800adb6d540 FS: 0000000000000000(0000) GS:ffff88011e380000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001168e4000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0efde50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0efe22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d86600>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d922d1>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0def55f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df5a20>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0decb5d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df5110>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df3011>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05de782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa019fdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01a3006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0608a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa068d014>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069753a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e782e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0641180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0642e39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0642260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-MDT0000 complete LustreError: 31919:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1666053603 with bad export cookie 11869320381983379453 LustreError: 166-1: MGC192.168.203.172@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 31919:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg372-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 712:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.172@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg372-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.1_RC1_40_g6e98a6e LNet: Added LNI 192.168.203.172@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg372-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg372-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] | Link to test |
| conf-sanity test 53a: check OSS thread count params | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 0 PID: 17588 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800becc0000 ti: ffff8800b13a0000 task.ti: ffff8800b13a0000 RIP: 0010:[<ffffffffa1335df2>] [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b13a3848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b13a3890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a6fd9290 R13: 0000000000000000 R14: ffff8800b77d2000 R15: ffff8800bc80c320 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d56de000 CR4: 00000000000006f0 Call Trace: [<ffffffffa135ae60>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135b23b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11c7640>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11d35a5>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124b3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1252020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12489bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1251710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124f611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05c7782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01c2df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01c6096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa05f1a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0675ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa067b02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa0202a5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa062a3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa062c059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff817e8dce>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa062b480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 21177:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179781/real 1655179781] req@ffff8800aa8eb900 x1735581722749824/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179787 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 21746:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: lustre-OST0000: deleting orphan objects from 0x0:35 to 0x0:65 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 25227:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179820/real 1655179820] req@ffff8800b2098000 x1735581811872448/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179826 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 25802:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:97 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 29040:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179855/real 1655179855] req@ffff8800a73c2140 x1735581852766656/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179861 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 29646:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:67 to 0x0:129 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 1980:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179902/real 1655179902] req@ffff8800b209df00 x1735581891564672/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179908 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 2569:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:131 to 0x0:161 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 6161:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179942/real 1655179942] req@ffff8800b209c740 x1735581938750336/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179948 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 6758:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: lustre-OST0000: deleting orphan objects from 0x0:163 to 0x0:193 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: server umount lustre-OST0000 complete Lustre: 10157:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1655179977/real 1655179977] req@ffff8800b209ed40 x1735581980692928/t0(0) o251->MGC192.168.203.163@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1655179983 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 10763:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_141_g0b3cf7b LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ vdc: vdc1 vdc9 vde: vde1 vde9 vdf: vdf1 vdf9 vde: vde1 vde9 vdf: vdf1 vdf9 Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1334f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw floppy virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 0 PID: 24332 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802e16e0010 ti: ffff8802f2784000 task.ti: ffff8802f2784000 RIP: 0010:[<ffffffffa1334f12>] [<ffffffffa1334f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802f2787848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802f2787890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88026ee8c978 R13: 0000000000000000 R14: ffff88029691b3f8 R15: ffff880091dd69e8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa1359ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135a3cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e0650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ec5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa124a66f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1250b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa1247c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1250220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124e121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0618782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0228df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa022c096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0642a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c6ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cc02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02aca5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-24.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_157_gd043fd3 LNet: Added LNI 192.168.123.26@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f1bf12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2b1613067 PUD 2b1612067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm drm_panel_orientation_quirks ata_piix serio_raw floppy i2c_core libata virtio_blk [last unloaded: libcfs] CPU: 13 PID: 20461 Comm: mdt06_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff88029b259280 ti: ffff8802b1314000 task.ti: ffff8802b1314000 RIP: 0010:[<ffffffffa0f1bf12>] [<ffffffffa0f1bf12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b1317848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b1317890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88009af86a98 R13: 0000000000000000 R14: ffff8800931ea2a8 R15: ffff8802addfa9e8 FS: 0000000000000000(0000) GS:ffff880331d40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002cc612000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f40ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f413cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0dc7650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dd35ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e3166f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e37b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e2ec6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e37220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e35121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06c8782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025edf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0262096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f2a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0776ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077c02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa035ca5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa072d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa072c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-104.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eecf12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2ddeaa067 PUD 31eea4067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw i2c_core virtio_blk libata [last unloaded: libcfs] CPU: 6 PID: 15291 Comm: mdt03_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802d11b9280 ti: ffff8802958b8000 task.ti: ffff8802958b8000 RIP: 0010:[<ffffffffa0eecf12>] [<ffffffffa0eecf12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802958bb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802958bb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802cb6883e8 R13: 0000000000000000 R14: ffff8802ae8322a8 R15: ffff8802cc311b40 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002a1648000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f11ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f123cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d98650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da45ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e0266f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e08b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dffc6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e08220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e06121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ca782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0256df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa025a096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f4a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0778ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077e02a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa035ea5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072d3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa072f059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa072e480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-54.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee2f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2c5dd3067 PUD 2a54fc067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console pcspkr virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk serio_raw i2c_core floppy libata [last unloaded: libcfs] CPU: 11 PID: 26282 Comm: mdt05_002 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802f2bc5c40 ti: ffff88031b2e8000 task.ti: ffff88031b2e8000 RIP: 0010:[<ffffffffa0ee2f12>] [<ffffffffa0ee2f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031b2eb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031b2eb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802db260dd8 R13: 0000000000000000 R14: ffff880323eba2a8 R15: ffff8800989fb890 FS: 0000000000000000(0000) GS:ffff880331cc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002b9684000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f07ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f083cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d8e650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d9a5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0df866f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dfeb30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df5c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0dfe220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0dfc121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06cf782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f9a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077dca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078302a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa033fa5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07323a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0734059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0733480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-9.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ed3df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 2 PID: 5694 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800ca602220 ti: ffff88008757c000 task.ti: ffff88008757c000 RIP: 0010:[<ffffffffa0ed3df2>] [<ffffffffa0ed3df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008757f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008757f890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bc794090 R13: 0000000000000000 R14: ffff8800c98e0000 R15: ffff8800b0e1a708 FS: 0000000000000000(0000) GS:ffff88011e300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000cf466000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0ef8e70>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0ef924b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d7f4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d8b455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0de93bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0df0020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de69bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0def710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0ded611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05db782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0199dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa019d006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0605a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06898b4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa068ec2a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e68ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa063e3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0640059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa063f480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 1 previous similar message Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-OST0000 complete LustreError: 21117:0:(client.c:1256:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff880116b7ed40 x1735093064329984/t0(0) o41->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 28887:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1654713787 with bad export cookie 13087907309105548108 LustreError: 166-1: MGC192.168.203.163@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28887:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 2 previous similar messages Lustre: DEBUG MARKER: oleg363-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 874:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.163@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg363-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_43_ge43f033 LNet: Added LNI 192.168.203.163@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg363-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg363-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs pcspkr i2c_piix4 i2c_core binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix serio_raw libata [last unloaded: libcfs] CPU: 0 PID: 7338 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800b8a36660 ti: ffff8800bb554000 task.ti: ffff8800bb554000 RIP: 0010:[<ffffffffa0edddf2>] [<ffffffffa0edddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800bb557848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800bb557890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800bcf38290 R13: 0000000000000000 R14: ffff8800ba239000 R15: ffff8800c90ae898 FS: 0000000000000000(0000) GS:ffff88011e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000cf614000 CR4: 00000000000006f0 Call Trace: [<ffffffffa0f02e60>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0323b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d894f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d95455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0df33bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dfa020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df09bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0df9710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0df7611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05e1782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01acdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01b0006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa060ba06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa068fc94>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069500a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01ec8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06443a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0646059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa0645480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 1 previous similar message Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 1 previous similar message Lustre: Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. LustreError: Skipped 1 previous similar message LustreError: 28670:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1654682054 with bad export cookie 7127315674021537718 LustreError: 166-1: MGC192.168.203.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28670:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: oleg302-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 630:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.102@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg302-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_55_g8fc89ea LNet: Added LNI 192.168.203.102@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg302-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg302-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef4f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] CPU: 10 PID: 26406 Comm: mdt05_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802bdfc5c40 ti: ffff8802df55c000 task.ti: ffff8802df55c000 RIP: 0010:[<ffffffffa0ef4f12>] [<ffffffffa0ef4f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802df55f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802df55f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802aece97c8 R13: 0000000000000000 R14: ffff88009c21a2a8 R15: ffff880324496e18 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000b9012000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f19ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1a3cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da0650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dac5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e0a66f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e10b30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e07c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e10220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0e121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06b3782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06dda06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0761ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa076702a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa0347a5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07163a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0718059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0717480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-9.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-9.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef2f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw i2c_core libata [last unloaded: libcfs] CPU: 14 PID: 16624 Comm: mdt07_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800b48f1280 ti: ffff8802b9084000 task.ti: ffff8802b9084000 RIP: 0010:[<ffffffffa0ef2f12>] [<ffffffffa0ef2f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b9087848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b9087890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880098f2c2d8 R13: 0000000000000000 R14: ffff88009d4822a8 R15: ffff8802c4afb678 FS: 0000000000000000(0000) GS:ffff880331d80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031afbe000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f17ff0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f183cb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9e650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0daa5ba>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e0866f>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0eb30>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e05c6d>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0e220>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e0c121>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06c5782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025edf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0262096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06efa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0773ca4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa077902a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa0359a5e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07283a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa072a059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0729480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-109.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1339df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm drm_panel_orientation_quirks libata serio_raw i2c_core virtio_blk floppy [last unloaded: libcfs] CPU: 12 PID: 6149 Comm: mdt06_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800a8e524f0 ti: ffff880079988000 task.ti: ffff880079988000 RIP: 0010:[<ffffffffa1339df2>] [<ffffffffa1339df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88007998b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88007998b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802b56e21b8 R13: 0000000000000000 R14: ffff88029ac80008 R15: ffff8802c220a9e8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000b91c6000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e54f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124f3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1256020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa124c9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1255710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1253611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0619782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa022ddd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0231006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0643a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c77c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06d1e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa026d8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067c3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067e059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067d480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-24.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.26@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-24.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1351df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 83fb3067 PUD 77389067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata floppy serio_raw i2c_core virtio_blk [last unloaded: libcfs] CPU: 8 PID: 14284 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff88027283a4f0 ti: ffff8803250c4000 task.ti: ffff8803250c4000 RIP: 0010:[<ffffffffa1351df2>] [<ffffffffa1351df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8803250c7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8803250c7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88026660e748 R13: 0000000000000000 R14: ffff8803251122a8 R15: ffff8802db357460 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000009a780000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1376e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137722b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11fd4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1209455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12673bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa126e020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12649bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa126d710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126b611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0613782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0235dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0239006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa063da06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c17c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06cbe6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02b08ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06763a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0678059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0677480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-84.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.86@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: Setting parameter lustre-MDT0002.mdt.identity_upcall in log lustre-MDT0002 Lustre: srv-lustre-MDT0002: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0002: new disk, initializing Lustre: lustre-MDT0002: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2:mdt Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2:mdt] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:3e8:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f08df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk libata i2c_core floppy serio_raw [last unloaded: libcfs] CPU: 0 PID: 22762 Comm: mdt00_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802da2b1280 ti: ffff880290e30000 task.ti: ffff880290e30000 RIP: 0010:[<ffffffffa0f08df2>] [<ffffffffa0f08df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880290e33848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880290e33890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88031d191128 R13: 0000000000000000 R14: ffff8802933ea2a8 R15: ffff880318f2faa8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f2de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f2e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0db44f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dc0455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e1e3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e25020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e1b9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e24710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e22611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ba782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa026add5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa026e006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06e4a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07687c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0772e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03498ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa071d3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa071f059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa071e480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-44.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1354df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 27f374067 PUD 277046067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] CPU: 12 PID: 6840 Comm: mdt06_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802d9803760 ti: ffff8800b4a68000 task.ti: ffff8800b4a68000 RIP: 0010:[<ffffffffa1354df2>] [<ffffffffa1354df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8800b4a6b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8800b4a6b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802c4616de8 R13: 0000000000000000 R14: ffff8802d33bb3f8 R15: ffff8802793ea9e8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002df98a000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1379e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa137a22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e64f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f2455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa126a3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1271020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12679bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1270710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa126e611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0618782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0231dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0235006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0642a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c67c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06d0e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02d98ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa067d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-19.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.21@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD a83ae067 PUD 93d38067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks virtio_blk floppy serio_raw i2c_core libata [last unloaded: libcfs] CPU: 10 PID: 23879 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802c29ddc40 ti: ffff88031ebf8000 task.ti: ffff88031ebf8000 RIP: 0010:[<ffffffffa0ee8df2>] [<ffffffffa0ee8df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031ebfb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031ebfb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802f4f65d58 R13: 0000000000000000 R14: ffff8802b5d8a2a8 R15: ffff8802d05bdb40 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000abc34000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f0de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d944f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da0455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0dfe3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e05020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfb9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e04710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e02611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0742782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0262dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0266006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa076ca06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07f07c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07fae6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa04038ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07a53a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa07a7059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa07a6480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-49.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-49.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-49.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-49.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1338df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2c5547067 PUD 2c5548067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix virtio_blk serio_raw i2c_core libata floppy [last unloaded: libcfs] CPU: 2 PID: 7040 Comm: mdt01_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff88031efcdc40 ti: ffff8802a7134000 task.ti: ffff8802a7134000 RIP: 0010:[<ffffffffa1338df2>] [<ffffffffa1338df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802a7137848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802a7137890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8803258b8a88 R13: 0000000000000000 R14: ffff8802df8622a8 R15: ffff8802d677e9e8 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ce9c0000 CR4: 00000000000007e0 Call Trace: [<ffffffffa135de50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135e22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e44f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f0455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124e3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1255020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa124b9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1254710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1252611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f5782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0233dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0237006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa061fa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a37c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ade6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02738ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06583a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa065a059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0659480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-89.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.91@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-89.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-89.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-89.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1362df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2a2976067 PUD 2b1818067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk libata floppy i2c_core [last unloaded: libcfs] CPU: 8 PID: 9217 Comm: mdt04_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802c7ba3760 ti: ffff8802bd76c000 task.ti: ffff8802bd76c000 RIP: 0010:[<ffffffffa1362df2>] [<ffffffffa1362df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bd76f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bd76f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88028b8b38e8 R13: 0000000000000000 R14: ffff88029b0e1158 R15: ffff8802976de9e8 FS: 0000000000000000(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ce764000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1387e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138822b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11f44f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1200455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12783bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127f020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12759bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa127e710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa127c611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa060d782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa024edd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0252006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0637a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06bb7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06c5e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02a18ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06703a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffff810bb2a0>] ? wake_up_atomic_t+0x30/0x30 [<ffffffffa0672059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0671480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-14.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.16@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 27443d067 PUD 27bafa067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw i2c_core floppy virtio_blk libata [last unloaded: libcfs] CPU: 0 PID: 24398 Comm: mdt00_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802acc88010 ti: ffff880294e6c000 task.ti: ffff880294e6c000 RIP: 0010:[<ffffffffa1335df2>] [<ffffffffa1335df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880294e6f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880294e6f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802df471e68 R13: 0000000000000000 R14: ffff8802ee925698 R15: ffff8802ef903460 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000029bf4a000 CR4: 00000000000007f0 Call Trace: [<ffffffffa135ae50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa135b22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11e14f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11ed455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa124b3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1252020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12489bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1251710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa124f611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05de782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0225dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0229006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0608a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa068c7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0696e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02938ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06413a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0643059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0642480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-84.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.86@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: Setting parameter lustre-MDT0002.mdt.identity_upcall in log lustre-MDT0002 Lustre: srv-lustre-MDT0002: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0002: new disk, initializing Lustre: lustre-MDT0002: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2:mdt Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2:mdt] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:3e8:ost] Lustre: DEBUG MARKER: centos-84.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ef9df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_balloon virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw virtio_blk i2c_core libata [last unloaded: libcfs] CPU: 6 PID: 29334 Comm: mdt03_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802dbcb0010 ti: ffff8802b2ad8000 task.ti: ffff8802b2ad8000 RIP: 0010:[<ffffffffa0ef9df2>] [<ffffffffa0ef9df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b2adb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b2adb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802e2d89478 R13: 0000000000000000 R14: ffff8802eeb822a8 R15: ffff8800b5dcf248 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f1ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da54f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0db1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e0f3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e16020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e0c9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e15710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e13611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa070e782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0276dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa027a006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0738a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07bc7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07c6e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03cf8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07713a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0773059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0772480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-29.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-29.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-29.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-29.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console pcspkr i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw virtio_blk libata i2c_core [last unloaded: libcfs] CPU: 4 PID: 16563 Comm: mdt02_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802ecf48010 ti: ffff8802b8e70000 task.ti: ffff8802b8e70000 RIP: 0010:[<ffffffffa1360df2>] [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b8e73848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b8e73890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802af293c38 R13: 0000000000000000 R14: ffff8802f03eb3f8 R15: ffff8800987269e8 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1385e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138622b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11f24f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11fe455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12763bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127d020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12739bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa127c710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa127a611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0606782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0253dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0257006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0630a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06b47c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06bee6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02938ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06693a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa066b059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa066a480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-39.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.41@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-39.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-39.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-39.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1329df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix i2c_core serio_raw virtio_blk floppy libata [last unloaded: libcfs] CPU: 0 PID: 13834 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802cce3c9d0 ti: ffff88008df50000 task.ti: ffff88008df50000 RIP: 0010:[<ffffffffa1329df2>] [<ffffffffa1329df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88008df53848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88008df53890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a8b242d8 R13: 0000000000000000 R14: ffff880094c09158 R15: ffff88009de3e9e8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000003195d4000 CR4: 00000000000007f0 Call Trace: [<ffffffffa134ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa134f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11d54f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11e1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa123f3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa1246020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa123c9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1245710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1243611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa061b782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0645a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06c97c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06d3e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02d78ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa067e3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0680059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa067f480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-94.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_54_g30cc1dc LNet: Added LNI 192.168.123.96@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_balloon pcspkr virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk floppy serio_raw libata i2c_core [last unloaded: libcfs] CPU: 6 PID: 19911 Comm: mdt03_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802bca3dc40 ti: ffff8802ce64c000 task.ti: ffff8802ce64c000 RIP: 0010:[<ffffffffa0eebdf2>] [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ce64f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ce64f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800ad43d368 R13: 0000000000000000 R14: ffff88008fd622a8 R15: ffff8800b42b4a80 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000003196d0000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f10e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1122b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d974f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da3455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e013bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e08020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfe9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e07710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e05611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06cc782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f6a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077a7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0784e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa038d8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa072f3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0731059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0730480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-94.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f17df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2ee5f3067 PUD 2b23ea067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console virtio_balloon i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic drm_panel_orientation_quirks virtio_blk pata_acpi floppy serio_raw ata_piix i2c_core libata [last unloaded: libcfs] CPU: 4 PID: 16326 Comm: mdt02_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880322ab3760 ti: ffff8802e2fa4000 task.ti: ffff8802e2fa4000 RIP: 0010:[<ffffffffa0f17df2>] [<ffffffffa0f17df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802e2fa7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802e2fa7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880296df0dd8 R13: 0000000000000000 R14: ffff8802af2fa2a8 R15: ffff8802da8f3890 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002ca692000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f3ce50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f3d22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0dc34f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dcf455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e2d3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e34020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e2a9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e33710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e31611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0721782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0261dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0265006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa074ba06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07cf7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07d9e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03808ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07843a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0786059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa0785480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-59.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-59.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee9df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix i2c_core serio_raw virtio_blk libata floppy [last unloaded: libcfs] CPU: 12 PID: 14557 Comm: mdt06_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880091d949d0 ti: ffff88009c3bc000 task.ti: ffff88009c3bc000 RIP: 0010:[<ffffffffa0ee9df2>] [<ffffffffa0ee9df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009c3bf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009c3bf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88029e0717c8 R13: 0000000000000000 R14: ffff8800a426a2a8 R15: ffff8802dc2125b8 FS: 0000000000000000(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f0ee50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0f22b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d954f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da1455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0dff3bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e06020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfc9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e05710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e03611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06be782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025ddd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0261006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06e8a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa076c7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0776e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa037f8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07213a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0723059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0722480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-44.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-44.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm drm_panel_orientation_quirks floppy virtio_blk serio_raw libata i2c_core [last unloaded: libcfs] CPU: 10 PID: 28640 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802edb39280 ti: ffff8802d9fdc000 task.ti: ffff8802d9fdc000 RIP: 0010:[<ffffffffa0eebdf2>] [<ffffffffa0eebdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802d9fdf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802d9fdf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a1ab60a8 R13: 0000000000000000 R14: ffff8802da88a2a8 R15: ffff8802b7f4b030 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f10e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1122b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d974f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da3455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e013bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e08020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0dfe9bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e07710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e05611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06faa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077e7c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0788e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03758ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07333a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0735059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0734480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-54.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eefdf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks ata_piix virtio_blk i2c_core serio_raw floppy libata [last unloaded: libcfs] CPU: 0 PID: 3001 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880327053760 ti: ffff88031e328000 task.ti: ffff88031e328000 RIP: 0010:[<ffffffffa0eefdf2>] [<ffffffffa0eefdf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88031e32b848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88031e32b890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802acea0a88 R13: 0000000000000000 R14: ffff880327be22a8 R15: ffff8800aae8f248 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000ba862000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f14e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1522b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9b4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da7455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e053bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0c020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e029bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0b710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e09611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d8782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa026cdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0270006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0702a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07867c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0790e6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03828ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa073b3a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa073d059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa073c480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-119.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0efddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 2f05fc067 PUD 2f05fd067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 virtio_console pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk libata i2c_core floppy [last unloaded: libcfs] CPU: 4 PID: 6377 Comm: mdt02_003 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802ac765c40 ti: ffff88009d0a4000 task.ti: ffff88009d0a4000 RIP: 0010:[<ffffffffa0efddf2>] [<ffffffffa0efddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009d0a7848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009d0a7890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802e850b8e8 R13: 0000000000000000 R14: ffff8800a5a222a8 R15: ffff88009102baa8 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002f054e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f22e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f2322b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0da94f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0db5455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e133bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e1a020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e109bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e19710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e17611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06d6782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa025cdd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0260006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0700a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07847c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078ee6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa036a8ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07393a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa073b059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffffa073a480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-14.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-MDT0002: Not available for connect from 0@lo (not set up) Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-14.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 296b35067 PUD 2b7113067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic ttm pata_acpi drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw i2c_core libata [last unloaded: libcfs] CPU: 1 PID: 29272 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880092b549d0 ti: ffff880323a6c000 task.ti: ffff880323a6c000 RIP: 0010:[<ffffffffa1360df2>] [<ffffffffa1360df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff880323a6f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff880323a6f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff88029710e3f8 R13: 0000000000000000 R14: ffff880091e422a8 R15: ffff8802faf6a9e8 FS: 0000000000000000(0000) GS:ffff880331a40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000002c69d4000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1385e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa138622b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa120c4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa1218455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa12763bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa127d020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12739bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa127c710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa127a611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0662782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0232dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0236006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa068ca06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07107c4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa071ae6a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03238ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa06c53a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa06c7059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa06c6480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-114.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_58_g7d7a06f2 LNet: Added LNI 192.168.123.116@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-114.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-114.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-114.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeddf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ttm drm ata_generic pata_acpi drm_panel_orientation_quirks ata_piix serio_raw virtio_blk floppy i2c_core libata [last unloaded: libcfs] CPU: 10 PID: 28742 Comm: mdt05_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800940f8010 ti: ffff88009ef44000 task.ti: ffff88009ef44000 RIP: 0010:[<ffffffffa0eeddf2>] [<ffffffffa0eeddf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88009ef47848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88009ef47890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff880092b338e8 R13: 0000000000000000 R14: ffff88009f0222a8 R15: ffff8800a5c83aa8 FS: 0000000000000000(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f12e70>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1324b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d994f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da5455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e033bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0a020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e009bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e09710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e07611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0703782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa0260dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0264006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa072da06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07b13e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07bba8a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03c48ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07663a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0768059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0767480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-54.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-54.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 54b: test llverfs and partial verify of filesystem | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ee3df2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic dm_flakey dm_mod crc_t10dif crct10dif_generic crct10dif_common rpcsec_gss_krb5 squashfs i2c_piix4 i2c_core pcspkr binfmt_misc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi serio_raw ata_piix libata [last unloaded: libcfs] CPU: 1 PID: 5311 Comm: mdt00_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff880116869110 ti: ffff88010c410000 task.ti: ffff88010c410000 RIP: 0010:[<ffffffffa0ee3df2>] [<ffffffffa0ee3df2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff88010c413848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88010c413890 R08: 0000000000000000 R09: 000000000000fffe R10: 0000000000000000 R11: 0000000000000001 R12: ffff88008d599e90 R13: 0000000000000000 R14: ffff88010e645000 R15: ffff8800bbb33180 FS: 0000000000000000(0000) GS:ffff88011e280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000000d6c00000 CR4: 00000000000006e0 Call Trace: [<ffffffffa0f08e50>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f0922b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d91570>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d9d091>] mdd_layout_change+0x1461/0x1b90 [mdd] [<ffffffffa0df93bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e00020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0df69bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0dff710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0dfd611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05f6782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa01a8dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa01ac006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0620a06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa06a45d4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa06ae7fa>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa01e87fe>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0659180>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa065ae39>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffffa065a260>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817f3e5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 LustreError: Skipped 1 previous similar message Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: server umount lustre-OST0000 complete LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) Lustre: Skipped 2 previous similar messages LustreError: 20743:0:(client.c:1256:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff8800913d3900 x1734252909587776/t0(0) o41->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' Lustre: server umount lustre-MDT0000 complete LustreError: 28502:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1653912545 with bad export cookie 5663760698040137429 LustreError: 166-1: MGC192.168.203.177@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail LustreError: 28502:0:(ldlm_lockd.c:2500:ldlm_cancel_handler()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: oleg377-server.virtnet: executing unload_modules_local Key type lgssc unregistered LNet: 458:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit LNet: Removed LNI 192.168.203.177@tcp LNet: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 alg: No test for adler32 (adler32-zlib) Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_hostid Lustre: DEBUG MARKER: oleg377-server.virtnet: executing load_modules_local Lustre: Lustre: Build Version: 2.15.50_14_g692e820 LNet: Added LNI 192.168.203.177@tcp [8/256/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0001: new disk, initializing Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:2710:ost Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:2710:ost] Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: oleg377-server.virtnet: executing set_default_debug -1 all 8 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:3e8:ost Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:3e8:ost] Lustre: DEBUG MARKER: oleg377-client.virtnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeedf2>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr i2c_piix4 virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks i2c_core ata_piix serio_raw virtio_blk floppy libata [last unloaded: libcfs] CPU: 0 PID: 2719 Comm: mdt00_001 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8800abbb1280 ti: ffff8802ee9bc000 task.ti: ffff8802ee9bc000 RIP: 0010:[<ffffffffa0eeedf2>] [<ffffffffa0eeedf2>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ee9bf848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ee9bf890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802de2ce0a8 R13: 0000000000000000 R14: ffff8800aa8322a8 R15: ffff880327b63aa8 FS: 0000000000000000(0000) GS:ffff880331a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007f0 Call Trace: [<ffffffffa0f13e70>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f1424b>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9a4f0>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da6455>] mdd_layout_change+0x1455/0x1b80 [mdd] [<ffffffffa0e043bf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0b020>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e019bd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e0a710>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e08611>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06e0782>] ldlm_lock_enqueue+0x3c2/0xb40 [ptlrpc] [<ffffffffa02a7dd5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa02ab006>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa070aa06>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa078e3e4>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0798a8a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03868ce>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa07433a0>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0745059>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0744480>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-119.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop7): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop7): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 3 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0002-mdc-*.mds_server_uuid LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-4): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:2710:ost Lustre: Skipped 2 previous similar messages Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:2710:ost] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa133ef12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon pcspkr virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm ata_piix drm virtio_blk libata drm_panel_orientation_quirks serio_raw i2c_core floppy [last unloaded: libcfs] CPU: 4 PID: 16891 Comm: mdt02_004 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802bc3ddc40 ti: ffff8802d2050000 task.ti: ffff8802d2050000 RIP: 0010:[<ffffffffa133ef12>] [<ffffffffa133ef12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802d2053848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802d2053890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802afb4a1b8 R13: 0000000000000000 R14: ffff8802ea118008 R15: ffff8802f58ba9e8 FS: 0000000000000000(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa1363fe0>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa13643bb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa11ea650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa11f65ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa12543cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa125a890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa12519cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa1259f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa1257e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa05e2841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa022edf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0232096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa060caa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0690d74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa069b24a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa02768ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0645470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0647129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0646550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-104.localnet: executing set_hostid Lustre: Lustre: Build Version: 2.15.50_141_gb58179d LNet: Added LNI 192.168.123.106@tcp [8/512/0/180] LNet: Accept secure, port 988 Key type lgssc registered Lustre: Echo OBD driver; http://www.lustre.org/ Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: lustre-MDT0000: new disk, initializing Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid Lustre: lustre-OST2710: new disk, initializing Lustre: srv-lustre-OST2710: No data found on store. Initialize space: rc = -61 Lustre: Skipped 1 previous similar message Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid Lustre: lustre-OST03e8: new disk, initializing Lustre: srv-lustre-OST03e8: No data found on store. Initialize space: rc = -61 Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 Lustre: DEBUG MARKER: centos-104.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0ecef12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_console i2c_piix4 virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks virtio_blk serio_raw floppy i2c_core libata [last unloaded: libcfs] CPU: 2 PID: 9416 Comm: mdt01_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff880320e95c40 ti: ffff8802bde60000 task.ti: ffff8802bde60000 RIP: 0010:[<ffffffffa0ecef12>] [<ffffffffa0ecef12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802bde63848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802bde63890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800a2255018 R13: 0000000000000000 R14: ffff8802989fa2a8 R15: ffff8802ed83a188 FS: 0000000000000000(0000) GS:ffff880331a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001c10000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0ef4010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0ef43eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d7a650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0d865ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0de43cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0dea890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0de19cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0de9f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0de7e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0725841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa027adf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa027e096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa074faa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa07d3d74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa07de24a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03e68ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0788470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa078a129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0789550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-119.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-119.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f0bf12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 9246f067 PUD 93748067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks ata_piix virtio_blk i2c_core serio_raw libata floppy [last unloaded: libcfs] CPU: 15 PID: 31596 Comm: mdt07_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802dcdfdc40 ti: ffff8802b6274000 task.ti: ffff8802b6274000 RIP: 0010:[<ffffffffa0f0bf12>] [<ffffffffa0f0bf12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802b6277848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802b6277890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802cac9bc38 R13: 0000000000000000 R14: ffff8802cd13a2a8 R15: ffff8802dc3b4eb0 FS: 0000000000000000(0000) GS:ffff880331dc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000bafb8000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f31010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f313eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0db7650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0dc35ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e213cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e27890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e1e9cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e26f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e24e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ce841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa025fdf5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0263096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f8aa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077cd74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078724a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa033e8ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0731470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0733129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0732550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-94.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-94.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0f22f12>] lod_fill_mirrors+0x192/0x620 [lod] PGD a5f7d067 PUD adaa1067 PMD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic drm pata_acpi drm_panel_orientation_quirks serio_raw virtio_blk ata_piix floppy libata i2c_core [last unloaded: libcfs] CPU: 6 PID: 20679 Comm: mdt03_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802a86b9280 ti: ffff8802cf5f8000 task.ti: ffff8802cf5f8000 RIP: 0010:[<ffffffffa0f22f12>] [<ffffffffa0f22f12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802cf5fb848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802cf5fb890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802d216a508 R13: 0000000000000000 R14: ffff8802e8d7a2a8 R15: ffff8802b997df70 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 00000000aba9a000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f48010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f483eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0b5c650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0b685ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e383cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e3e890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e359cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e3df80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e3be81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa0be4841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa0261df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa0265096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa0c0eaa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa0c92d74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa0c9d24a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa035b8ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0c47470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0c49129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0c48550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-109.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST03e8-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey Lustre: DEBUG MARKER: centos-109.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |
| conf-sanity test 56a: check big OST indexes and out-of-index-order start | BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffffa0eeef12>] lod_fill_mirrors+0x192/0x620 [lod] PGD 0 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ata_generic pata_acpi ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] CPU: 6 PID: 3711 Comm: mdt03_001 Kdump: loaded Tainted: P B OE ------------ 3.10.0-7.9-debug #2 Hardware name: Red Hat KVM, BIOS 1.15.0-1.module_el8.6.0+1087+b42c8331 04/01/2014 task: ffff8802d2ecdc40 ti: ffff8802ed82c000 task.ti: ffff8802ed82c000 RIP: 0010:[<ffffffffa0eeef12>] [<ffffffffa0eeef12>] lod_fill_mirrors+0x192/0x620 [lod] RSP: 0018:ffff8802ed82f848 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff8802ed82f890 R08: 0000000000000000 R09: 73726f7272696d5f R10: 0000000000000000 R11: 0000000000000001 R12: ffff8802d2b863f8 R13: 0000000000000000 R14: ffff8802e5a7a2a8 R15: ffff8800a8133678 FS: 0000000000000000(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000031d89e000 CR4: 00000000000007e0 Call Trace: [<ffffffffa0f14010>] lod_striped_create+0x3a0/0x5d0 [lod] [<ffffffffa0f143eb>] lod_layout_change+0x1ab/0x2a0 [lod] [<ffffffffa0d9a650>] dt_layout_change+0x20/0xc0 [mdd] [<ffffffffa0da65ca>] mdd_layout_change+0x145a/0x1b80 [mdd] [<ffffffffa0e043cf>] mdt_layout_change+0x2ff/0x490 [mdt] [<ffffffffa0e0a890>] mdt_intent_layout+0x910/0xe60 [mdt] [<ffffffffa0e019cd>] mdt_intent_opc+0x1dd/0xc10 [mdt] [<ffffffffa0e09f80>] ? mdt_intent_open+0x480/0x480 [mdt] [<ffffffffa0e07e81>] mdt_intent_policy+0x1a1/0x360 [mdt] [<ffffffffa06ce841>] ldlm_lock_enqueue+0x481/0xbc0 [ptlrpc] [<ffffffffa0267df5>] ? cfs_hash_rw_unlock+0x15/0x20 [libcfs] [<ffffffffa026b096>] ? cfs_hash_add+0xa6/0x180 [libcfs] [<ffffffffa06f8aa6>] ldlm_handle_enqueue0+0x8c6/0x1780 [ptlrpc] [<ffffffffa077cd74>] tgt_enqueue+0x64/0x240 [ptlrpc] [<ffffffffa078724a>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<ffffffffa03628ee>] ? libcfs_nid2str_r+0xfe/0x130 [lnet] [<ffffffffa0731470>] ptlrpc_server_handle_request+0x250/0xc30 [ptlrpc] [<ffffffffa0733129>] ptlrpc_main+0xbd9/0x15f0 [ptlrpc] [<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90 [<ffffffff817e329e>] ? _raw_spin_unlock_irq+0xe/0x30 [<ffffffffa0732550>] ? ptlrpc_wait_event+0x620/0x620 [ptlrpc] [<ffffffff810ba114>] kthread+0xe4/0xf0 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 [<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140 | Lustre: DEBUG MARKER: centos-19.localnet: executing set_hostid LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (loop5): file extents enabled, maximum tree depth=5 LDISKFS-fs (loop5): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-1): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: lustre-OST2710-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Lustre: DEBUG MARKER: centos-19.localnet: executing wait_import_state_mount FULL osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid Lustre: Mounted lustre-client | Link to test |