| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| racer test 1: racer on clients: centos-110.localnet DURATION=2700 | LustreError: 9112:0:(llog_osd.c:626:llog_osd_write_rec()) LBUG Pid: 9112, comm: mdt_rdpg03_000 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] libcfs_call_trace+0x90/0xf0 [libcfs] [<0>] lbug_with_loc+0x4c/0xa0 [libcfs] [<0>] llog_osd_write_rec+0x1e5d/0x1e90 [obdclass] [<0>] llog_write_rec+0x290/0x590 [obdclass] [<0>] llog_cat_add_rec+0x1d1/0xa40 [obdclass] [<0>] llog_add+0x17f/0x1f0 [obdclass] [<0>] sub_updates_write+0x317/0xe8b [ptlrpc] [<0>] top_trans_stop+0x429/0xf30 [ptlrpc] [<0>] lod_trans_stop+0x6a/0x350 [lod] [<0>] mdd_trans_stop+0x28/0x16e [mdd] [<0>] mdd_attr_set+0x697/0xdd0 [mdd] [<0>] mdt_mfd_close+0x76e/0xbb0 [mdt] [<0>] mdt_close_internal+0xb4/0x240 [mdt] [<0>] mdt_close+0x28c/0x970 [mdt] [<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc] [<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe | LustreError: 9087:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/13 failed: rc = -114 LustreError: 15620:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x280000403:0x3:0x0] mdc close failed: rc = -116 Lustre: 9085:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 514 < left 671, rollback = 2 Lustre: 9085:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/0, destroy: 0/0/0 Lustre: 9085:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0 Lustre: 9085:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0 Lustre: 9085:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 10/203/1, delete: 0/0/0 Lustre: 9085:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 LustreError: 9072:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/5 failed: rc = -16 LustreError: 9072:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 16128:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x280000404:0x2:0x0] mdc close failed: rc = -13 Lustre: 16498:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 503 < left 610, rollback = 2 Lustre: 16498:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 7 previous similar messages Lustre: 16498:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 6/24/8, destroy: 0/0/0 Lustre: 16498:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 16498:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 8/610/0 Lustre: 16498:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 16498:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 11/192/0, punch: 0/0/0, quota 1/3/0 Lustre: 16498:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 16498:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 19/350/4, delete: 1/1/0 Lustre: 16498:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 16498:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 1/1/0 Lustre: 16498:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 7 previous similar messages LustreError: 16386:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/5 failed: rc = -2 Lustre: 9081:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 506 < left 582, rollback = 2 Lustre: 9081:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 11 previous similar messages Lustre: 9081:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/7, destroy: 0/0/0 Lustre: 9081:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 11 previous similar messages Lustre: 9081:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 6/582/0 Lustre: 9081:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 11 previous similar messages Lustre: 9081:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0 Lustre: 9081:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 11 previous similar messages Lustre: 9081:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/2, delete: 0/0/0 Lustre: 9081:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 11 previous similar messages Lustre: 9081:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0 Lustre: 9081:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 11 previous similar messages Lustre: 16006:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 512 < left 788, rollback = 2 Lustre: 16006:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 5 previous similar messages Lustre: 16006:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/1, destroy: 0/0/0 Lustre: 16006:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 5 previous similar messages Lustre: 16006:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 10/788/0 Lustre: 16006:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 5 previous similar messages Lustre: 16006:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0 Lustre: 16006:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 5 previous similar messages Lustre: 16006:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 12/215/2, delete: 1/1/0 Lustre: 16006:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 5 previous similar messages Lustre: 16006:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 1/1/0 Lustre: 16006:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 5 previous similar messages LustreError: 9090:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/16 failed: rc = -114 LustreError: 9083:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '15' migration was interrupted, run 'lfs migrate -m 0 -c 3 -H crush 15' to finish migration. LustreError: 15854:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0001: migrate [0x240000403:0x1:0x0]/14 failed: rc = -114 LustreError: 15854:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 1 previous similar message Lustre: 18216:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 508 < left 938, rollback = 2 Lustre: 18216:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 50 previous similar messages Lustre: 18216:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/4, destroy: 0/0/0 Lustre: 18216:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 50 previous similar messages Lustre: 18216:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 10/938/0 Lustre: 18216:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 50 previous similar messages Lustre: 18216:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0 Lustre: 18216:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 50 previous similar messages Lustre: 18216:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/3, delete: 0/0/0 Lustre: 18216:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 50 previous similar messages Lustre: 18216:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0 Lustre: 18216:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 50 previous similar messages Lustre: 9100:0:(mdt_recovery.c:149:mdt_req_from_lrd()) @@@ restoring transno req@ffff8802989e2dc0 x1782000926321984/t4294968813(0) o101->933bfd55-8c20-4a83-a73c-428bcadb1faf@0@lo:263/0 lens 376/864 e 0 to 0 dl 1699448638 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0 LustreError: 16833:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200000403:0x1:0x0]/3 failed: rc = -16 LustreError: 16833:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 7 previous similar messages LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) header@ffff880290d38008[0x4, 2, [0x380000401:0x9:0x0] hash]{ LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880290d38088[0] LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880295caf8c8id: 0x380000401:9 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448617 atime: 1699448617 ctime: 1699448617 blocks: 0 LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) } header@ffff880290d38008 LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) header@ffff8802860af248[0x4, 1, [0x240000404:0x35:0x0] hash]{ LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802860af2c8(0) inode: ffff88028dda8108 162129603832315957/37748740 100644 1 0 ffff8802860af2c8 [0x240000404:0x35:0x0] LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880286976388entries: 1, valid, lsm{ffff88029605fb18 0x0BD10BD0 1 0}: LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) header@ffff880290d38008[0x4, 2, [0x380000401:0x9:0x0] hash]{ LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880290d38088[0] LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880295caf8c8id: 0x380000401:9 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448617 atime: 1699448617 ctime: 1699448617 blocks: 0 LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) } header@ffff880290d38008 LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802860af248 LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 18439:0:(lov_object.c:186:lov_init_sub()) header@ffff880088a73aa8[0x0, 1, [0x200000404:0x40:0x0] hash] LustreError: 18439:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 18439:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x40:0x0]: rc = -5 LustreError: 18439:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 Lustre: 9084:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 510 < left 1756, rollback = 2 Lustre: 9084:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 47 previous similar messages Lustre: 9084:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/2, destroy: 1/4/0 Lustre: 9084:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 47 previous similar messages Lustre: 9084:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 19/1756/0 Lustre: 9084:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 47 previous similar messages Lustre: 9084:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 7/105/0 Lustre: 9084:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 47 previous similar messages Lustre: 9084:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 14/287/2, delete: 2/5/1 Lustre: 9084:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 47 previous similar messages Lustre: 9084:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 2/2/0 Lustre: 9084:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 47 previous similar messages LustreError: 9084:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0001: [0x240000403:0x90:0x0] migrate mdt count mismatch 2 != 1 LustreError: 19877:0:(llite_lib.c:1836:ll_update_lsm_md()) lustre: [0x240000403:0xb5:0x0] dir layout mismatch: LustreError: 19877:0:(lustre_lmv.h:178:lmv_stripe_object_dump()) dump LMV: refs 215092432 magic=0x2 count=2 index=1 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool= LustreError: 19877:0:(lustre_lmv.h:185:lmv_stripe_object_dump()) stripe[0] [0x240000400:0x8:0x0] LustreError: 19877:0:(lustre_lmv.h:178:lmv_stripe_object_dump()) dump LMV: refs 215092432 magic=0x1 count=3 index=1 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=2 migrate_hash=fnv_1a_64:2 pool= LustreError: 21243:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x200000403:0x10:0x0] mdc close failed: rc = -116 LustreError: 9082:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/6 failed: rc = -16 LustreError: 9082:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 15 previous similar messages Lustre: 16833:0:(mdt_recovery.c:149:mdt_req_from_lrd()) @@@ restoring transno req@ffff8802cc3ece40 x1782000927694784/t4294970643(0) o101->caee2dfc-acdf-4df3-88cc-a51025f1266a@0@lo:281/0 lens 376/840 e 0 to 0 dl 1699448656 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0 Lustre: 17654:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 514 < left 565, rollback = 2 Lustre: 17654:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 194 previous similar messages Lustre: 17654:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0 Lustre: 17654:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 194 previous similar messages Lustre: 17654:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 7/565/0 Lustre: 17654:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 194 previous similar messages Lustre: 17654:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/21/0, punch: 0/0/0, quota 6/80/0 Lustre: 17654:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 194 previous similar messages Lustre: 17654:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 1/36/0, delete: 2/2/1 Lustre: 17654:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 194 previous similar messages Lustre: 17654:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 1/1/0 Lustre: 17654:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 194 previous similar messages Lustre: 16506:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0xfd:0x0] with magic=0xbd60bd0 Lustre: 19082:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0xee:0x0] with magic=0xbd60bd0 Lustre: 19082:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 9 previous similar messages LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{ LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d638b00[0] LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d638a80 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff880088a70a80[0x4, 1, [0x280000403:0x8d:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088a70b00(0) inode: ffff88028defe988 180144002325020813/41943044 100644 1 0 ffff880088a70b00 [0x280000403:0x8d:0x0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01a7f8entries: 4, valid, lsm{ffff880289eaa9b8 0x0BD60BD0 1 4}: LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d638b00[0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d638a80 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088a70a80 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff88028b43fcc0[0x0, 1, [0x240000404:0x1a1:0x0] hash] LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 26244:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x1a1:0x0]: rc = -5 LustreError: 26244:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{ LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0] LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d63b460 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff8802783e7890[0x4, 1, [0x280000404:0xfc:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802783e7910(0) inode: ffff88028dfee008 180144002341798140/41943044 100644 1 0 ffff8802783e7910 [0x280000404:0xfc:0x0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880326f325c0entries: 4, valid, lsm{ffff88028ed8c008 0x0BD60BD0 1 4}: LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d63b460 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802783e7890 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff88008fa5e188[0x0, 1, [0x240000404:0x213:0x0] hash] LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own. Lustre: 9079:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x13d:0x0] with magic=0xbd60bd0 Lustre: 9079:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 23 previous similar messages LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{ LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d126208[0] LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d126188 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a42b890[0x4, 1, [0x200000404:0x5:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a42b910(0) inode: ffff88028dc33088 144115205322833925/33554436 101155 2 0 ffff88008a42b910 [0x200000404:0x5:0x0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880297f23310entries: 2, valid, lsm{ffff88028d71d5f8 0x0BD60BD0 1 5}: LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x8000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d126208[0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d126188 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 3, 0x1, 0, 0x10, 3, 2097152 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125928[0x4, 1, [0x300000402:0x9:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d1259a8[65536] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895cda8id: 0x300000402:9 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 4008 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125928 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125f70[0x4, 1, [0x340000402:0xb:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125ff0[65537] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828405c8id: 0x340000402:11 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125f70 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125b40[0x4, 1, [0x2c0000402:0xb:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125bc0[65538] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282840e68id: 0x2c0000402:11 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125b40 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a42b890 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff88008fa5fcc0[0x0, 1, [0x240000404:0x1f0:0x0] hash] LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own. Lustre: 10981:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{ LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028908fd40[0] LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028908fcc0 LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43c008[0x4, 1, [0x200000404:0x26:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43c088(0) inode: ffff880090ba9408 144115205322833958/33554436 101755 2 0 ffff88028b43c088 [0x200000404:0x26:0x0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880289b9cd58entries: 1, valid, lsm{ffff88008d498f38 0x0BD10BD0 1 0}: LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{ LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028908fd40[0] LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028908fcc0 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43c008 LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff880287627cc0[0x0, 1, [0x240000404:0x216:0x0] hash] LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 26244:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x216:0x0]: rc = -5 LustreError: 26244:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 26244:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 26244:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 2 previous similar messages Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{ LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d126208[0] LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d126188 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a42b890[0x4, 1, [0x200000404:0x5:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a42b910(0) inode: ffff88028dc33088 144115205322833925/33554436 101155 2 0 ffff88008a42b910 [0x200000404:0x5:0x0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880297f23310entries: 2, valid, lsm{ffff88028d71d5f8 0x0BD60BD0 1 5}: LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x8000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d126208[0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d126188 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 3, 0x1, 0, 0x10, 3, 2097152 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125928[0x4, 1, [0x300000402:0x9:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d1259a8[65536] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895cda8id: 0x300000402:9 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 4008 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125928 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125f70[0x4, 1, [0x340000402:0xb:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125ff0[65537] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828405c8id: 0x340000402:11 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125f70 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125b40[0x4, 1, [0x2c0000402:0xb:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125bc0[65538] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282840e68id: 0x2c0000402:11 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125b40 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a42b890 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff880287625928[0x0, 1, [0x240000404:0x1f0:0x0] hash] LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 27619:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x1f0:0x0]: rc = -5 LustreError: 27619:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{ LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880293fb1148[0] LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0 LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) } header@ffff880293fb10c8 LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) header@ffff880293ee0438[0x4, 1, [0x200000404:0x8d:0x0] hash]{ LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880293ee04b8(0) inode: ffff880084c39d88 144115205322834061/33554436 100644 1 0 ffff880293ee04b8 [0x200000404:0x8d:0x0] LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01b9b8entries: 1, valid, lsm{ffff88028e50fcc8 0x0BD10BD0 1 0}: LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{ LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880293fb1148[0] LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0 LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293fb10c8 LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293ee0438 LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 28280:0:(lov_object.c:186:lov_init_sub()) header@ffff880296178650[0x0, 1, [0x240000403:0x13c:0x0] hash] LustreError: 28280:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{ LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d638b00[0] LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d638a80 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff880088a70a80[0x4, 1, [0x280000403:0x8d:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088a70b00(0) inode: ffff88028defe988 180144002325020813/41943044 100644 1 0 ffff880088a70b00 [0x280000403:0x8d:0x0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01a7f8entries: 4, valid, lsm{ffff880289eaa9b8 0x0BD60BD0 1 4}: LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d638b00[0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d638a80 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088a70a80 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff88029617a9e8[0x0, 1, [0x240000404:0x1a1:0x0] hash] LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{ LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0] LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d63b460 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff8802783e7890[0x4, 1, [0x280000404:0xfc:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802783e7910(0) inode: ffff88028dfee008 180144002341798140/41943044 100644 1 0 ffff8802783e7910 [0x280000404:0xfc:0x0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880326f325c0entries: 4, valid, lsm{ffff88028ed8c008 0x0BD60BD0 1 4}: LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d63b460 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802783e7890 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff88029617a5b8[0x0, 1, [0x240000404:0x213:0x0] hash] LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{ LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d126208[0] LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d126188 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a42b890[0x4, 1, [0x200000404:0x5:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a42b910(0) inode: ffff88028dc33088 144115205322833925/33554436 101155 2 0 ffff88008a42b910 [0x200000404:0x5:0x0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880297f23310entries: 2, valid, lsm{ffff88028d71d5f8 0x0BD60BD0 1 5}: LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x8000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d126208[0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d126188 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 3, 0x1, 0, 0x10, 3, 2097152 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125928[0x4, 1, [0x300000402:0x9:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d1259a8[65536] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895cda8id: 0x300000402:9 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 4008 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125928 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125f70[0x4, 1, [0x340000402:0xb:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125ff0[65537] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828405c8id: 0x340000402:11 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125f70 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125b40[0x4, 1, [0x2c0000402:0xb:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125bc0[65538] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282840e68id: 0x2c0000402:11 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125b40 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a42b890 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff88027a2c4220[0x0, 1, [0x240000404:0x1f0:0x0] hash] LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{ LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028908fd40[0] LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028908fcc0 LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43c008[0x4, 1, [0x200000404:0x26:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43c088(0) inode: ffff880090ba9408 144115205322833958/33554436 101755 2 0 ffff88028b43c088 [0x200000404:0x26:0x0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880289b9cd58entries: 1, valid, lsm{ffff88008d498f38 0x0BD10BD0 1 0}: LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{ LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028908fd40[0] LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028908fcc0 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43c008 LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff880287624a80[0x0, 1, [0x240000404:0x216:0x0] hash] LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 27619:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x216:0x0]: rc = -5 LustreError: 27619:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 4 previous similar messages LustreError: 27619:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 27619:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 4 previous similar messages LustreError: 29138:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x200000404:0x14e:0x0] mdc close failed: rc = -116 Lustre: 16855:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x1a1:0x0] with magic=0xbd60bd0 Lustre: 16855:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 13 previous similar messages Lustre: 19400:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 514 < left 618, rollback = 7 Lustre: 19400:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message LustreError: 29723:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x240000403:0x1c8:0x0] mdc close failed: rc = -116 LustreError: 29723:0:(file.c:246:ll_close_inode_openhandle()) Skipped 1 previous similar message 14[29966]: segfault at 8 ip 00007f50e349b7e8 sp 00007fffdddba8d0 error 4 in ld-2.17.so[7f50e3490000+22000] LustreError: 27696:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/19 failed: rc = -114 LustreError: 27696:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 15 previous similar messages LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{ LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880293fb1148[0] LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0 LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) } header@ffff880293fb10c8 LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) header@ffff880293ee0438[0x4, 1, [0x200000404:0x8d:0x0] hash]{ LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880293ee04b8(0) inode: ffff880084c39d88 144115205322834061/33554436 100644 1 0 ffff880293ee04b8 [0x200000404:0x8d:0x0] LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01b9b8entries: 1, valid, lsm{ffff88028e50fcc8 0x0BD10BD0 1 0}: LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{ LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880293fb1148[0] LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0 LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293fb10c8 LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293ee0438 LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 30062:0:(lov_object.c:186:lov_init_sub()) header@ffff880287625f70[0x0, 1, [0x240000403:0x13c:0x0] hash] LustreError: 30062:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 30062:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000403:0x13c:0x0]: rc = -5 LustreError: 30062:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 Lustre: 21222:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 534, rollback = 7 Lustre: 21222:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message LustreError: 31331:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x240000403:0x26a:0x0] mdc close failed: rc = -116 LustreError: 31331:0:(file.c:246:ll_close_inode_openhandle()) Skipped 6 previous similar messages Lustre: dir [0x200000404:0x1db:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: 29795:0:(mdt_recovery.c:149:mdt_req_from_lrd()) @@@ restoring transno req@ffff880283d147c0 x1782000930991424/t4294971292(0) o101->caee2dfc-acdf-4df3-88cc-a51025f1266a@0@lo:322/0 lens 376/816 e 0 to 0 dl 1699448697 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0 Lustre: 16682:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 499 < left 671, rollback = 2 Lustre: 16682:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 108 previous similar messages Lustre: 16682:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/12, destroy: 0/0/0 Lustre: 16682:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 116 previous similar messages Lustre: 16682:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0 Lustre: 16682:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 116 previous similar messages Lustre: 16682:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0 Lustre: 16682:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 116 previous similar messages Lustre: 16682:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/4, delete: 0/0/0 Lustre: 16682:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 116 previous similar messages Lustre: 16682:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0 Lustre: 16682:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 116 previous similar messages Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message Lustre: mdt02_000: service thread pid 9078 was inactive for 40.075 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: Pid: 9078, comm: mdt02_000 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt] [<0>] mdt_object_lock_try+0xa0/0x250 [mdt] [<0>] mdt_getattr_name_lock+0x17f9/0x2b30 [mdt] [<0>] mdt_intent_getattr+0x2c5/0x4b0 [mdt] [<0>] mdt_intent_opc+0x1c2/0xc40 [mdt] [<0>] mdt_intent_policy+0xfa/0x460 [mdt] [<0>] ldlm_lock_enqueue+0x3b1/0xbb0 [ptlrpc] [<0>] ldlm_handle_enqueue+0x34d/0x17b0 [ptlrpc] [<0>] tgt_enqueue+0x68/0x240 [ptlrpc] [<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc] [<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Pid: 9083, comm: mdt03_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt] [<0>] mdt_object_lock+0x88/0x1c0 [mdt] [<0>] mdt_object_find_lock+0x54/0x170 [mdt] [<0>] mdt_reint_setxattr+0x169/0x1110 [mdt] [<0>] mdt_reint_rec+0x87/0x240 [mdt] [<0>] mdt_reint_internal+0x74c/0xba0 [mdt] [<0>] mdt_reint+0x67/0x150 [mdt] [<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc] [<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Pid: 9091, comm: mdt05_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt] [<0>] mdt_object_lock_try+0xa0/0x250 [mdt] [<0>] mdt_getattr_name_lock+0x17f9/0x2b30 [mdt] [<0>] mdt_intent_getattr+0x2c5/0x4b0 [mdt] [<0>] mdt_intent_opc+0x1c2/0xc40 [mdt] [<0>] mdt_intent_policy+0xfa/0x460 [mdt] [<0>] ldlm_lock_enqueue+0x3b1/0xbb0 [ptlrpc] [<0>] ldlm_handle_enqueue+0x34d/0x17b0 [ptlrpc] [<0>] tgt_enqueue+0x68/0x240 [ptlrpc] [<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc] [<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Lustre: mdt05_009: service thread pid 18596 was inactive for 40.545 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. LustreError: 302:0:(lov_object.c:184:lov_init_sub()) header@ffff88027dff6188[0x4, 2, [0x300000400:0x30:0x0] hash]{ LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027dff6208[0] LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8 LustreError: 302:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027dff6188 LustreError: 302:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43f460[0x4, 1, [0x280000404:0x210:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43f4e0(0) inode: ffff8802775f9408 180144002341798416/41943044 100644 1 0 ffff88028b43f4e0 [0x280000404:0x210:0x0] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc4478entries: 1, valid, lsm{ffff88008df86018 0x0BD10BD0 1 0}: LustreError: 302:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 3, 1048576 } LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff6188[0x4, 2, [0x300000400:0x30:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6208[0] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff6188 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff69e8[0x4, 1, [0x340000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6a68[1] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff69e8 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff8802883c8a80[0x4, 1, [0x380000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802883c8b00[2] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802883c8a80 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43f460 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 302:0:(lov_object.c:186:lov_init_sub()) header@ffff88008711e7d0[0x0, 1, [0x200000403:0x1b2:0x0] hash] LustreError: 302:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 302:0:(lov_object.c:184:lov_init_sub()) header@ffff88027dff69e8[0x4, 2, [0x340000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027dff6a68[1] LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027dff69e8 LustreError: 302:0:(lov_object.c:184:lov_init_sub()) stripe 1 is already owned. LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43f460[0x4, 1, [0x280000404:0x210:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43f4e0(0) inode: ffff8802775f9408 180144002341798416/41943044 100644 1 0 ffff88028b43f4e0 [0x280000404:0x210:0x0] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc4478entries: 1, valid, lsm{ffff88008df86018 0x0BD10BD0 1 0}: LustreError: 302:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 3, 1048576 } LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff6188[0x4, 1, [0x300000400:0x30:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6208[0] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff6188 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff69e8[0x4, 2, [0x340000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6a68[1] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff69e8 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff8802883c8a80[0x4, 1, [0x380000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802883c8b00[2] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802883c8a80 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43f460 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 302:0:(lov_object.c:186:lov_init_sub()) header@ffff88008711e7d0[0x0, 1, [0x200000403:0x1b2:0x0] hash] LustreError: 302:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 302:0:(lov_object.c:184:lov_init_sub()) header@ffff8802883c8a80[0x4, 2, [0x380000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802883c8b00[2] LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802883c8a80 LustreError: 302:0:(lov_object.c:184:lov_init_sub()) stripe 2 is already owned. LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43f460[0x4, 1, [0x280000404:0x210:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43f4e0(0) inode: ffff8802775f9408 180144002341798416/41943044 100644 1 0 ffff88028b43f4e0 [0x280000404:0x210:0x0] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc4478entries: 1, valid, lsm{ffff88008df86018 0x0BD10BD0 1 0}: LustreError: 302:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 3, 1048576 } LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff6188[0x4, 1, [0x300000400:0x30:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6208[0] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff6188 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff69e8[0x4, 1, [0x340000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6a68[1] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff69e8 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff8802883c8a80[0x4, 2, [0x380000400:0x32:0x0] hash]{ LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802883c8b00[2] LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802883c8a80 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43f460 LustreError: 302:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 302:0:(lov_object.c:186:lov_init_sub()) header@ffff88008711e7d0[0x0, 1, [0x200000403:0x1b2:0x0] hash] LustreError: 302:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 302:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000403:0x1b2:0x0]: rc = -5 LustreError: 302:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 1040:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x240000404:0x373:0x0] mdc close failed: rc = -116 Lustre: dir [0x240000404:0x358:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 9119:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0x358:0x0]: rc = -2 Lustre: 20409:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20409:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) header@ffff88008417a188[0x4, 2, [0x2c0000400:0x3b:0x0] hash]{ LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88008417a208[0] LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880278ab9148id: 0x2c0000400:59 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448717 blocks: 72 LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) } header@ffff88008417a188 LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) header@ffff880282830650[0x4, 1, [0x280000403:0x216:0x0] hash]{ LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802828306d0(0) inode: ffff880084da4388 180144002325021206/41943044 100700 1 0 ffff8802828306d0 [0x280000403:0x216:0x0] LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efc4240entries: 1, valid, lsm{ffff88027ae9a888 0x0BD10BD0 1 0}: LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) header@ffff88008417a188[0x4, 2, [0x2c0000400:0x3b:0x0] hash]{ LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88008417a208[0] LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880278ab9148id: 0x2c0000400:59 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448717 blocks: 72 LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008417a188 LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) } header@ffff880282830650 LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 4080:0:(lov_object.c:186:lov_init_sub()) header@ffff880085868eb0[0x0, 1, [0x240000404:0x430:0x0] hash] LustreError: 4080:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9121:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000403:0x1d2:0x0]: rc = -2 LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{ LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88029765b0b0[0] LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16 LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) } header@ffff88029765b030 LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) header@ffff88008711df70[0x4, 1, [0x200000404:0x24e:0x0] hash]{ LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008711dff0(0) inode: ffff880084da8a88 144115205322834510/33554436 100755 3 0 ffff88008711dff0 [0x200000404:0x24e:0x0] LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efc7780entries: 1, valid, lsm{ffff88028700e378 0x0BD10BD0 1 0}: LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{ LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88029765b0b0[0] LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16 LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) } header@ffff88029765b030 LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008711df70 LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 4487:0:(lov_object.c:186:lov_init_sub()) header@ffff880279c729e8[0x4, 1, [0x280000403:0x250:0x0] hash] LustreError: 4487:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 4487:0:(lov_object.c:1337:lov_layout_change()) lustre-clilov-ffff88008fa0e678: cannot apply new layout on [0x280000403:0x250:0x0] : rc = -5 LustreError: 4487:0:(vvp_io.c:1876:vvp_io_init()) lustre: refresh file layout [0x280000403:0x250:0x0] error -5. Lustre: 16367:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x2b5:0x0] with magic=0xbd60bd0 Lustre: 16367:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 3 previous similar messages LustreError: 29805:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200000404:0x27b:0x0]/11 failed: rc = -2 LustreError: 29805:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 65 previous similar messages Lustre: 10970:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 Lustre: 10970:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 3 previous similar messages Lustre: dir [0x200000404:0x2e3:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{ LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88029765b0b0[0] LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16 LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) } header@ffff88029765b030 LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) header@ffff88008711df70[0x4, 1, [0x200000404:0x24e:0x0] hash]{ LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008711dff0(0) inode: ffff880084da8a88 144115205322834510/33554436 100755 3 0 ffff88008711dff0 [0x200000404:0x24e:0x0] LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efc7780entries: 1, valid, lsm{ffff88028700e378 0x0BD10BD0 1 0}: LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{ LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88029765b0b0[0] LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16 LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) } header@ffff88029765b030 LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008711df70 LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 6589:0:(lov_object.c:186:lov_init_sub()) header@ffff880279c729e8[0x4, 1, [0x280000403:0x250:0x0] hash] LustreError: 6589:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 6589:0:(lov_object.c:1337:lov_layout_change()) lustre-clilov-ffff88008fa0e678: cannot apply new layout on [0x280000403:0x250:0x0] : rc = -5 LustreError: 6589:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000403:0x250:0x0]: rc = -5 LustreError: 6589:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 1 previous similar message LustreError: 6589:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 6589:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 1 previous similar message Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000403:0x271:0x0] with magic=0xbd60bd0 Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 7 previous similar messages Lustre: 9100:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 503 < left 671, rollback = 2 Lustre: 9100:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 462 previous similar messages Lustre: 9100:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/8, destroy: 0/0/0 Lustre: 9100:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 470 previous similar messages Lustre: 9100:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0 Lustre: 9100:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 470 previous similar messages Lustre: 9100:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 1/3/0 Lustre: 9100:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 470 previous similar messages Lustre: 9100:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/4, delete: 0/0/0 Lustre: 9100:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 470 previous similar messages Lustre: 9100:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0 Lustre: 9100:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 470 previous similar messages LustreError: 7371:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x200000404:0x2fc:0x0] mdc close failed: rc = -116 LustreError: 7371:0:(file.c:246:ll_close_inode_openhandle()) Skipped 8 previous similar messages LustreError: 16690:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000403:0x224:0x0]: rc = -2 Lustre: dir [0x280000404:0x305:0x0] stripe 1 readdir failed: -2, directory is partially accessed! LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{ LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0] LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}: LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00 LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 6755:0:(lov_object.c:186:lov_init_sub()) header@ffff880080451b40[0x0, 1, [0x200000403:0x32b:0x0] hash] LustreError: 6755:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{ LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0] LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}: LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00 LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 7198:0:(lov_object.c:186:lov_init_sub()) header@ffff88027cf4f890[0x0, 1, [0x200000403:0x32b:0x0] hash] LustreError: 7198:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{ LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0] LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}: LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00 LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8493:0:(lov_object.c:186:lov_init_sub()) header@ffff880275d95d58[0x0, 1, [0x200000403:0x32b:0x0] hash] LustreError: 8493:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 8956:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0002_UUID lock: ffff88029214de00/0x6a008282726167e6 lrc: 3/0,0 mode: PR/PR res: [0x280000403:0x1a2:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x6a008282726167bc expref: 248 pid: 16506 timeout: 310 lvb_type: 0 LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{ LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802907cddd8[0] LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0 LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802907cdd58 LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 11-0: lustre-MDT0002-mdc-ffff88009074b7e8: operation ldlm_enqueue to node 0@lo failed: rc = -107 Lustre: lustre-MDT0002-mdc-ffff88009074b7e8: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 167-0: lustre-MDT0002-mdc-ffff88009074b7e8: This client was evicted by lustre-MDT0002; in progress operations using this service will fail. LustreError: 27089:0:(vvp_io.c:1876:vvp_io_init()) lustre: refresh file layout [0x280000403:0x1a2:0x0] error -108. LustreError: 26878:0:(file.c:5373:ll_inode_revalidate_fini()) lustre: revalidate FID [0x280000403:0x1a2:0x0] error: rc = -108 LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dca94f8[0x4, 1, [0x240000403:0x48e:0x0] hash]{ LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dca9578(0) inode: ffff88026cd53088 162129603815539854/37748740 107514 2 0 ffff88027dca9578 [0x240000403:0x48e:0x0] LustreError: 8194:0:(mdc_request.c:1465:mdc_read_page()) lustre-MDT0002-mdc-ffff88009074b7e8: [0x280000401:0x14:0x0] lock enqueue fails: rc = -5 LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802793227f8entries: 1, valid, lsm{ffff8802820281b8 0x0BD10BD0 1 0}: LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{ LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802907cddd8[0] LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0 Lustre: lustre-MDT0002-mdc-ffff88009074b7e8: Connection restored to (at 0@lo) LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802907cdd58 LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dca94f8 LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 6677:0:(lov_object.c:186:lov_init_sub()) header@ffff88027acb0a80[0x0, 1, [0x280000403:0x2ab:0x0] hash] LustreError: 6677:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9083:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '0' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 0' to finish migration. 16[8823]: segfault at 8 ip 00007f8b288237e8 sp 00007ffcaa09f790 error 4 in ld-2.17.so[7f8b28818000+22000] LustreError: 27680:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '1' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 1' to finish migration. LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{ LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802907cddd8[0] LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0 LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802907cdd58 LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dca94f8[0x4, 1, [0x240000403:0x48e:0x0] hash]{ LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dca9578(0) inode: ffff88026cd53088 162129603815539854/37748740 107514 2 0 ffff88027dca9578 [0x240000403:0x48e:0x0] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802793227f8entries: 1, valid, lsm{ffff8802820281b8 0x0BD10BD0 1 0}: LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{ LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802907cddd8[0] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802907cdd58 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dca94f8 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) header@ffff880087c13460[0x0, 1, [0x280000403:0x2ab:0x0] hash] LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. 4[9398]: segfault at 8 ip 00007f42328a67e8 sp 00007fff260698c0 error 4 in ld-2.17.so[7f423289b000+22000] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{ LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}: LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{ LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) header@ffff880087c125b8[0x0, 1, [0x200000403:0x32b:0x0] hash] LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{ LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e884b8[0] LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1 LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e88438 LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff88027cf4e9e8[0x4, 1, [0x280000403:0x234:0x0] hash]{ LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027cf4ea68(0) inode: ffff88026cdf8108 180144002325021236/41943044 100755 1 0 ffff88027cf4ea68 [0x280000403:0x234:0x0] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802821cb548entries: 1, valid, lsm{ffff88027b30e1c8 0x0BD10BD0 1 0}: LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{ LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e884b8[0] LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e88438 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027cf4e9e8 LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) header@ffff880087c110c8[0x0, 1, [0x200000403:0x32e:0x0] hash] LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) try to own. Lustre: 10969:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 540, rollback = 7 Lustre: 10969:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 19 previous similar messages LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{ LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802907cddd8[0] LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0 LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802907cdd58 LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dca94f8[0x4, 1, [0x240000403:0x48e:0x0] hash]{ LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dca9578(0) inode: ffff88026cd53088 162129603815539854/37748740 107514 2 0 ffff88027dca9578 [0x240000403:0x48e:0x0] LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802793227f8entries: 1, valid, lsm{ffff8802820281b8 0x0BD10BD0 1 0}: LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{ LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802907cddd8[0] LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0 LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802907cdd58 LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dca94f8 LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8376:0:(lov_object.c:186:lov_init_sub()) header@ffff8800804510c8[0x0, 1, [0x280000403:0x2ab:0x0] hash] LustreError: 8376:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 20324:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '0' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 0' to finish migration. LustreError: 20324:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) Skipped 1 previous similar message LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f794f8[0x4, 2, [0x300000400:0x5a:0x0] hash]{ LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f79578[0] LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880082308e68id: 0x300000400:90 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448773 atime: 1699448773 ctime: 1699448773 blocks: 0 LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f794f8 LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dcab030[0x4, 1, [0x280000405:0x2e:0x0] hash]{ LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dcab0b0(0) inode: ffff88026cdc0108 180144002358575150/41943044 100644 1 0 ffff88027dcab0b0 [0x280000405:0x2e:0x0] LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280a19aa8entries: 2, valid, lsm{ffff88026efb2a38 0x0BD60BD0 1 2}: LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x40000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 65536 } LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f794f8[0x4, 2, [0x300000400:0x5a:0x0] hash]{ LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f79578[0] LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880082308e68id: 0x300000400:90 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448773 atime: 1699448773 ctime: 1699448773 blocks: 0 LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f794f8 LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) [0x40000, 0xffffffffffffffff): { 0x0BD10BD0, 2, 0x1, 65535, 0x0, 1, 2097152 } LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dcab030 LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 11129:0:(lov_object.c:186:lov_init_sub()) header@ffff88027cf4f248[0x0, 1, [0x240000404:0x5b9:0x0] hash] LustreError: 11129:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{ LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e884b8[0] LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1 LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e88438 LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) header@ffff88027cf4e9e8[0x4, 1, [0x280000403:0x234:0x0] hash]{ LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027cf4ea68(0) inode: ffff88026cdf8108 180144002325021236/41943044 100755 1 0 ffff88027cf4ea68 [0x280000403:0x234:0x0] LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802821cb548entries: 1, valid, lsm{ffff88027b30e1c8 0x0BD10BD0 1 0}: LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{ LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e884b8[0] LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1 LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e88438 LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027cf4e9e8 LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 11688:0:(lov_object.c:186:lov_init_sub()) header@ffff8802889a94f8[0x0, 1, [0x200000403:0x32e:0x0] hash] LustreError: 11688:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{ LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e884b8[0] LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1 LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e88438 LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) header@ffff88027cf4e9e8[0x4, 1, [0x280000403:0x234:0x0] hash]{ LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027cf4ea68(0) inode: ffff88026cdf8108 180144002325021236/41943044 100755 1 0 ffff88027cf4ea68 [0x280000403:0x234:0x0] LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802821cb548entries: 1, valid, lsm{ffff88027b30e1c8 0x0BD10BD0 1 0}: LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{ LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e884b8[0] LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1 LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e88438 LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027cf4e9e8 LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 11609:0:(lov_object.c:186:lov_init_sub()) header@ffff8802889a9b40[0x0, 1, [0x200000403:0x32e:0x0] hash] LustreError: 11609:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{ LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f7ae98[0] LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72 LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f7ae18 LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) header@ffff880083424eb0[0x4, 1, [0x200000404:0x336:0x0] hash]{ LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880083424f30(0) inode: ffff880084e11408 144115205322834742/33554436 100700 1 0 ffff880083424f30 [0x200000404:0x336:0x0] LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008409e150entries: 1, valid, lsm{ffff880295c1e018 0x0BD10BD0 1 0}: LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{ LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f7ae98[0] LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72 LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f7ae18 LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) } header@ffff880083424eb0 LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 11168:0:(lov_object.c:186:lov_init_sub()) header@ffff880083425d58[0x0, 1, [0x240000403:0x5da:0x0] hash] LustreError: 11168:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9087:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0000: '1' migration was interrupted, run 'lfs migrate -m 2 -c 3 -H crush 1' to finish migration. LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{ LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f7ae98[0] LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72 LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f7ae18 LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) header@ffff880083424eb0[0x4, 1, [0x200000404:0x336:0x0] hash]{ LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880083424f30(0) inode: ffff880084e11408 144115205322834742/33554436 100700 1 0 ffff880083424f30 [0x200000404:0x336:0x0] LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008409e150entries: 1, valid, lsm{ffff880295c1e018 0x0BD10BD0 1 0}: LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{ LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f7ae98[0] LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72 LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f7ae18 LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) } header@ffff880083424eb0 LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 12370:0:(lov_object.c:186:lov_init_sub()) header@ffff880280b44868[0x0, 1, [0x240000403:0x5da:0x0] hash] LustreError: 12370:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{ LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f7ae98[0] LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72 LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f7ae18 LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) header@ffff880083424eb0[0x4, 1, [0x200000404:0x336:0x0] hash]{ LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880083424f30(0) inode: ffff880084e11408 144115205322834742/33554436 100700 1 0 ffff880083424f30 [0x200000404:0x336:0x0] LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008409e150entries: 1, valid, lsm{ffff880295c1e018 0x0BD10BD0 1 0}: LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{ LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f7ae98[0] LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72 LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f7ae18 LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) } header@ffff880083424eb0 LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 12597:0:(lov_object.c:186:lov_init_sub()) header@ffff88027cf4d2e0[0x0, 1, [0x240000403:0x5da:0x0] hash] LustreError: 12597:0:(lov_object.c:186:lov_init_sub()) try to own. Lustre: 9093:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000404:0x3a2:0x0] with magic=0xbd60bd0 Lustre: 9093:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 15 previous similar messages LustreError: 12913:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x240000404:0x6ba:0x0] mdc close failed: rc = -116 LustreError: 12913:0:(file.c:246:ll_close_inode_openhandle()) Skipped 27 previous similar messages LustreError: 3642:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/14 failed: rc = -114 LustreError: 3642:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 41 previous similar messages Lustre: 10973:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 618, rollback = 7 Lustre: 10973:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 5 previous similar messages 19[27046]: segfault at 8 ip 00007fda6aecb7e8 sp 00007fffca60a630 error 4 in ld-2.17.so[7fda6aec0000+22000] Lustre: 19400:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0 Lustre: 19400:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 650 previous similar messages Lustre: 19400:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0 Lustre: 19400:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 650 previous similar messages Lustre: 10973:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 2/564/0, punch: 0/0/0, quota 1/3/0 Lustre: 10973:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 651 previous similar messages Lustre: 10973:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0 Lustre: 10973:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 651 previous similar messages Lustre: 10973:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 Lustre: 10973:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 651 previous similar messages LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188 LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{ LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0] LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}: LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 Lustre: 9098:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 508 < left 671, rollback = 2 Lustre: 9098:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 617 previous similar messages LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188 LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460 LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 28367:0:(lov_object.c:186:lov_init_sub()) header@ffff88028fae9710[0x0, 1, [0x240000404:0x77d:0x0] hash] LustreError: 28367:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 28367:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x77d:0x0]: rc = -5 LustreError: 28367:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 14 previous similar messages LustreError: 28367:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 28367:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 14 previous similar messages LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188 LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{ LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0] LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}: LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188 LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460 LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 29749:0:(lov_object.c:186:lov_init_sub()) header@ffff880280eec868[0x0, 1, [0x240000404:0x77d:0x0] hash] LustreError: 29749:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9088:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '12' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 12' to finish migration. LustreError: 9111:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0x7a1:0x0]: rc = -2 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff88008cc712e0[0x4, 2, [0x300000401:0xd0:0x0] hash]{ LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88008cc71360[0] LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088ad5928id: 0x300000401:208 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448875 atime: 1699448875 ctime: 1699448875 blocks: 0 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff88008cc712e0 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845db890[0x4, 1, [0x240000403:0x777:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802845db910(0) inode: ffff880084fb7308 162129603815540599/37748740 100644 1 0 ffff8802845db910 [0x240000403:0x777:0x0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008bd2eea0entries: 1, valid, lsm{ffff880289e9b608 0x0BD10BD0 1 0}: LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88008cc712e0[0x4, 2, [0x300000401:0xd0:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88008cc71360[0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088ad5928id: 0x300000401:208 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448875 atime: 1699448875 ctime: 1699448875 blocks: 0 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008cc712e0 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845db890 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff8802a00bdf70[0x0, 1, [0x200000404:0x569:0x0] hash] LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188 LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{ LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0] LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}: LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188 LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460 LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 31333:0:(lov_object.c:186:lov_init_sub()) header@ffff88028e6ae9e8[0x0, 1, [0x240000404:0x77d:0x0] hash] LustreError: 31333:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff88027b22b460[0x4, 2, [0x340000400:0x79:0x0] hash]{ LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027b22b4e0[0] LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fea68id: 0x340000400:121 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 26842 mtime: 1699448856 atime: 1699448853 ctime: 1699448856 blocks: 16 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027b22b460 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a4c6e18[0x4, 1, [0x280000404:0x4a7:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028a4c6e98(0) inode: ffff880084f39d88 180144002341799079/41943044 103675 1 0 ffff88028a4c6e98 [0x280000404:0x4a7:0x0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802882570d8entries: 1, valid, lsm{ffff8800870aa1c8 0x0BD10BD0 1 0}: LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88027b22b460[0x4, 2, [0x340000400:0x79:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027b22b4e0[0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fea68id: 0x340000400:121 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 26842 mtime: 1699448856 atime: 1699448853 ctime: 1699448856 blocks: 16 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027b22b460 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a4c6e18 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff8802845d9928[0x0, 1, [0x240000404:0x78e:0x0] hash] LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}: LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff880293afe9e8[0x0, 1, [0x240000404:0x77d:0x0] hash] LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{ LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800bba184b8[0] LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800bba18438 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028faeb030[0x4, 1, [0x280000404:0x506:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028faeb0b0(0) inode: ffff8802671a8108 180144002341799174/41943044 106101 4 0 ffff88028faeb0b0 [0x280000404:0x506:0x0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880275ff2ea0entries: 1, valid, lsm{ffff880285604f38 0x0BD10BD0 1 0}: LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800bba184b8[0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800bba18438 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028faeb030 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff880280eedf70[0x0, 1, [0x200000404:0x586:0x0] hash] LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{ LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800bba184b8[0] LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800bba18438 LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028faeb030[0x4, 1, [0x280000404:0x506:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028faeb0b0(0) inode: ffff8802671a8108 180144002341799174/41943044 106101 4 0 ffff88028faeb0b0 [0x280000404:0x506:0x0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880275ff2ea0entries: 1, valid, lsm{ffff880285604f38 0x0BD10BD0 1 0}: LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{ LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800bba184b8[0] LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800bba18438 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028faeb030 LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff8802845da9e8[0x0, 1, [0x200000404:0x586:0x0] hash] LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 16876:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0001: '14' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 14' to finish migration. Lustre: dir [0x280000405:0x246:0x0] stripe 3 readdir failed: -2, directory is partially accessed! Lustre: Skipped 8 previous similar messages LustreError: 715:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 715:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 715:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 715:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0 LustreError: 715:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 715:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{ LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0] LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}: LustreError: 715:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 715:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 715:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0 LustreError: 715:0:(lov_object.c:185:lov_init_sub()) LustreError: 715:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0 LustreError: 715:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 715:0:(lov_object.c:186:lov_init_sub()) header@ffff88028ef18650[0x0, 1, [0x280000404:0x5f9:0x0] hash] LustreError: 715:0:(lov_object.c:186:lov_init_sub()) try to own. Lustre: 16876:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x84f:0x0] with magic=0xbd60bd0 Lustre: 16876:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 15 previous similar messages LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{ LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800bba184b8[0] LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320 LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800bba18438 LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) header@ffff88028faeb030[0x4, 1, [0x280000404:0x506:0x0] hash]{ LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028faeb0b0(0) inode: ffff8802671a8108 180144002341799174/41943044 106101 4 0 ffff88028faeb0b0 [0x280000404:0x506:0x0] LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880275ff2ea0entries: 1, valid, lsm{ffff880285604f38 0x0BD10BD0 1 0}: LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{ LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800bba184b8[0] LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320 LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800bba18438 LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028faeb030 LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 1213:0:(lov_object.c:186:lov_init_sub()) header@ffff8800807a2c00[0x4, 1, [0x200000404:0x586:0x0] hash] LustreError: 1213:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 1213:0:(lov_object.c:1337:lov_layout_change()) lustre-clilov-ffff88009074b7e8: cannot apply new layout on [0x200000404:0x586:0x0] : rc = -5 LustreError: 1213:0:(vvp_io.c:1876:vvp_io_init()) lustre: refresh file layout [0x200000404:0x586:0x0] error -5. LustreError: 6701:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x609:0x0]: rc = -2 LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{ LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080962a68[0] LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800809629e8 LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18eb0[0x4, 1, [0x200000403:0x55d:0x0] hash]{ LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef18f30(0) inode: ffff880084ff6988 144115205306058077/33554436 100644 1 0 ffff88028ef18f30 [0x200000403:0x55d:0x0] LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880262c01638entries: 4, valid, lsm{ffff88028fb92448 0x0BD60BD0 1 4}: LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 65537, 0x1, 0, 0x10, 1, 65536 } LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{ LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080962a68[0] LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800809629e8 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 65538, 0x1, 65535, 0x0, 2, 2097152 } LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 131075, 0x1, 0, 0x10, 1, 65536 } LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e1b460[0x4, 1, [0x2c0000402:0x6f:0x0] hash]{ LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e1b4e0[131072] LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc0b88id: 0x2c0000402:111 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448924 atime: 1699448924 ctime: 1699448924 blocks: 0 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e1b460 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 131076, 0x1, 65535, 0x0, 2, 2097152 } LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18eb0 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8c98[0x0, 1, [0x280000405:0x34b:0x0] hash] LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{ LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0] LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}: LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8 LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8c98[0x0, 1, [0x240000403:0x7d9:0x0] hash] LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0 LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{ LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0] LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}: LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0 LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0 LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15379:0:(lov_object.c:186:lov_init_sub()) header@ffff88028ef1a3a0[0x0, 1, [0x280000404:0x5f9:0x0] hash] LustreError: 15379:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0 LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{ LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0] LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}: LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0 LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0 LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 3389:0:(lov_object.c:186:lov_init_sub()) header@ffff8802845dae18[0x0, 1, [0x280000404:0x5f9:0x0] hash] LustreError: 3389:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 562:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 562:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 562:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 562:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 562:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 562:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{ LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0] LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}: LustreError: 562:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 562:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 562:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 562:0:(lov_object.c:185:lov_init_sub()) LustreError: 562:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8 LustreError: 562:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 562:0:(lov_object.c:186:lov_init_sub()) header@ffff880271ace188[0x0, 1, [0x240000403:0x7d9:0x0] hash] LustreError: 562:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 562:0:(statahead.c:791:ll_statahead_interpret_work()) lustre: getattr callback for sleep [0x240000403:0x7d9:0x0]: rc = -5 LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{ LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080962a68[0] LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800809629e8 LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18eb0[0x4, 1, [0x200000403:0x55d:0x0] hash]{ LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef18f30(0) inode: ffff880084ff6988 144115205306058077/33554436 100644 1 0 ffff88028ef18f30 [0x200000403:0x55d:0x0] LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880262c01638entries: 4, valid, lsm{ffff88028fb92448 0x0BD60BD0 1 4}: LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 65537, 0x1, 0, 0x10, 1, 65536 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{ LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080962a68[0] LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800809629e8 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 65538, 0x1, 65535, 0x0, 2, 2097152 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 131075, 0x1, 0, 0x10, 1, 65536 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e1b460[0x4, 1, [0x2c0000402:0x6f:0x0] hash]{ LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e1b4e0[131072] LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc0b88id: 0x2c0000402:111 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448924 atime: 1699448924 ctime: 1699448924 blocks: 0 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e1b460 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 131076, 0x1, 65535, 0x0, 2, 2097152 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18eb0 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) header@ffff88028ef18438[0x0, 1, [0x280000405:0x34b:0x0] hash] LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{ LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080962a68[0] LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800809629e8 LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18eb0[0x4, 1, [0x200000403:0x55d:0x0] hash]{ LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef18f30(0) inode: ffff880084ff6988 144115205306058077/33554436 100644 1 0 ffff88028ef18f30 [0x200000403:0x55d:0x0] LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880262c01638entries: 4, valid, lsm{ffff88028fb92448 0x0BD60BD0 1 4}: LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 65537, 0x1, 0, 0x10, 1, 65536 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{ LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080962a68[0] LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800809629e8 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 65538, 0x1, 65535, 0x0, 2, 2097152 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 131075, 0x1, 0, 0x10, 1, 65536 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e1b460[0x4, 1, [0x2c0000402:0x6f:0x0] hash]{ LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e1b4e0[131072] LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc0b88id: 0x2c0000402:111 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448924 atime: 1699448924 ctime: 1699448924 blocks: 0 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e1b460 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 131076, 0x1, 65535, 0x0, 2, 2097152 } LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18eb0 LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) header@ffff880271ace7d0[0x0, 1, [0x280000405:0x34b:0x0] hash] LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 16656:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0000: '19' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 19' to finish migration. LustreError: 16656:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) Skipped 1 previous similar message Lustre: dir [0x200000404:0x64a:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message LustreError: 11035:0:(out_handler.c:886:out_tx_end()) lustre-MDT0000-osd: undo for /home/green/git/lustre-release/lustre/ptlrpc/../../lustre/target/out_handler.c:418: rc = -524 LustreError: 10364:0:(llog_cat.c:737:llog_cat_cancel_arr_rec()) lustre-MDT0000-osp-MDT0002: fail to cancel 1 llog-records: rc = -2 LustreError: 10364:0:(llog_cat.c:773:llog_cat_cancel_records()) lustre-MDT0000-osp-MDT0002: fail to cancel 1 of 1 llog-records: rc = -2 LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{ LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0] LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}: LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8 LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8418:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7cc98[0x0, 1, [0x240000403:0x7d9:0x0] hash] LustreError: 8418:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{ LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0] LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}: LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{ LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0] LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72 LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8 LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8 LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8645:0:(lov_object.c:186:lov_init_sub()) header@ffff8802822d3030[0x0, 1, [0x240000403:0x7d9:0x0] hash] LustreError: 8645:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0 LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{ LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0] LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}: LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{ LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0] LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8 LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0 LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0 LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 9577:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7dd58[0x0, 1, [0x280000404:0x5f9:0x0] hash] LustreError: 9577:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{ LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d16a208[0] LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0 LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d16a188 LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff880280adb890[0x4, 1, [0x280000405:0x2f3:0x0] hash]{ LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280adb910(0) inode: ffff880262899d88 180144002358575859/41943044 100644 1 0 ffff880280adb910 [0x280000405:0x2f3:0x0] LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f36ea0entries: 4, valid, lsm{ffff88028c6c13f8 0x0BD60BD0 1 4}: LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{ LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d16a208[0] LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d16a188 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280adb890 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) header@ffff88028a5fd928[0x0, 1, [0x240000403:0x8ec:0x0] hash] LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{ LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e19360[0] LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0 LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e192e0 LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff880280eecc98[0x4, 1, [0x280000404:0x5c5:0x0] hash]{ LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280eecd18(0) inode: ffff880267324388 180144002341799365/41943044 100644 1 0 ffff880280eecd18 [0x280000404:0x5c5:0x0] LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880279ccf9b8entries: 1, valid, lsm{ffff88028ac695f8 0x0BD10BD0 1 0}: LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{ LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e19360[0] LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e192e0 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280eecc98 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec23a0[0x0, 1, [0x200000403:0x65e:0x0] hash] LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{ LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b6208[0] LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0 LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b6188 LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cee18[0x4, 1, [0x280000405:0x301:0x0] hash]{ LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cee98(0) inode: ffff8802673b2708 180144002358575873/41943044 100644 1 0 ffff88027f6cee98 [0x280000405:0x301:0x0] LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e6139b8entries: 4, valid, lsm{ffff88008770a0a8 0x0BD60BD0 1 4}: LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{ LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b6208[0] LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b6188 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 } LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cee18 LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec1710[0x0, 1, [0x200000403:0x632:0x0] hash] LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 6701:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000405:0x455:0x0]: rc = -2 LustreError: 9524:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x280000405:0x455:0x0] mdc close failed: rc = -2 LustreError: 9524:0:(file.c:246:ll_close_inode_openhandle()) Skipped 22 previous similar messages LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{ LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d16a208[0] LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0 LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d16a188 LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff880280adb890[0x4, 1, [0x280000405:0x2f3:0x0] hash]{ LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280adb910(0) inode: ffff880262899d88 180144002358575859/41943044 100644 1 0 ffff880280adb910 [0x280000405:0x2f3:0x0] LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f36ea0entries: 4, valid, lsm{ffff88028c6c13f8 0x0BD60BD0 1 4}: LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{ LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d16a208[0] LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d16a188 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280adb890 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7ec00[0x0, 1, [0x240000403:0x8ec:0x0] hash] LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{ LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e19360[0] LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0 LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e192e0 LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff880280eecc98[0x4, 1, [0x280000404:0x5c5:0x0] hash]{ LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280eecd18(0) inode: ffff880267324388 180144002341799365/41943044 100644 1 0 ffff880280eecd18 [0x280000404:0x5c5:0x0] LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880279ccf9b8entries: 1, valid, lsm{ffff88028ac695f8 0x0BD10BD0 1 0}: LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{ LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e19360[0] LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e192e0 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280eecc98 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) header@ffff88026fa6b678[0x0, 1, [0x200000403:0x65e:0x0] hash] LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{ LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b6208[0] LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0 LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b6188 LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cee18[0x4, 1, [0x280000405:0x301:0x0] hash]{ LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cee98(0) inode: ffff8802673b2708 180144002358575873/41943044 100644 1 0 ffff88027f6cee98 [0x280000405:0x301:0x0] LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e6139b8entries: 4, valid, lsm{ffff88008770a0a8 0x0BD60BD0 1 4}: LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{ LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b6208[0] LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b6188 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 } LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cee18 LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec14f8[0x0, 1, [0x200000403:0x632:0x0] hash] LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{ LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028a77a208[0] LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72 LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028a77a188 LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a5fd0c8[0x4, 1, [0x280000405:0x46f:0x0] hash]{ LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028a5fd148(0) inode: ffff8802629b7308 180144002358576239/41943044 100755 1 0 ffff88028a5fd148 [0x280000405:0x46f:0x0] LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880292ca3bf0entries: 1, valid, lsm{ffff88028761abe8 0x0BD10BD0 1 0}: LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{ LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028a77a208[0] LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72 LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a77a188 LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a5fd0c8 LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 13292:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7e3a0[0x0, 1, [0x240000404:0xb40:0x0] hash] LustreError: 13292:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890 LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{ LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0] LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}: LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890 LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008 LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 14549:0:(lov_object.c:186:lov_init_sub()) header@ffff88028fb33248[0x0, 1, [0x280000405:0x427:0x0] hash] LustreError: 14549:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{ LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) header@ffff880284e514f8[0x4, 2, [0x2c0000400:0xc1:0x0] hash]{ LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880284e51578[0] LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880279fb7308id: 0x2c0000400:193 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) } header@ffff880284e514f8 LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff8800807a3cc0[0x4, 1, [0x280000404:0x74e:0x0] hash]{ LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8800807a3d40(0) inode: ffff880262a4b088 180144002341799758/41943044 100644 1 0 ffff8800807a3d40 [0x280000404:0x74e:0x0] LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e613548entries: 1, valid, lsm{ffff8800823f70f8 0x0BD10BD0 1 0}: LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff880284e514f8[0x4, 2, [0x2c0000400:0xc1:0x0] hash]{ LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880284e51578[0] LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880279fb7308id: 0x2c0000400:193 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff880284e514f8 LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800807a3cc0 LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7e3a0[0x0, 1, [0x200000404:0x73a:0x0] hash] LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88026d1c1578[0] LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72 LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) } header@ffff88026d1c14f8 LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) header@ffff880280ad8650[0x4, 1, [0x280000405:0x57e:0x0] hash]{ LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280ad86d0(0) inode: ffff880262b00108 180144002358576510/41943044 100755 1 0 ffff880280ad86d0 [0x280000405:0x57e:0x0] LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800877f1f18entries: 1, valid, lsm{ffff880296950a28 0x0BD10BD0 1 0}: LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{ LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88026d1c1578[0] LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72 LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) } header@ffff88026d1c14f8 LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280ad8650 LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 14531:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7cc98[0x0, 1, [0x240000404:0xb97:0x0] hash] LustreError: 14531:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{ LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028a77a208[0] LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72 LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028a77a188 LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a5fd0c8[0x4, 1, [0x280000405:0x46f:0x0] hash]{ LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028a5fd148(0) inode: ffff8802629b7308 180144002358576239/41943044 100755 1 0 ffff88028a5fd148 [0x280000405:0x46f:0x0] LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880292ca3bf0entries: 1, valid, lsm{ffff88028761abe8 0x0BD10BD0 1 0}: LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{ LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028a77a208[0] LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72 LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a77a188 LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a5fd0c8 LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8a80[0x0, 1, [0x240000404:0xb40:0x0] hash] LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890 LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{ LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0] LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}: LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890 LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008 LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15691:0:(lov_object.c:186:lov_init_sub()) header@ffff880088a40008[0x0, 1, [0x280000405:0x427:0x0] hash] LustreError: 15691:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890 LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{ LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0] LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}: LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890 LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008 LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15587:0:(lov_object.c:186:lov_init_sub()) header@ffff880088a43890[0x0, 1, [0x280000405:0x427:0x0] hash] LustreError: 15587:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{ LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d16a208[0] LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0 LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d16a188 LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff880280adb890[0x4, 1, [0x280000405:0x2f3:0x0] hash]{ LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280adb910(0) inode: ffff880262899d88 180144002358575859/41943044 100644 1 0 ffff880280adb910 [0x280000405:0x2f3:0x0] LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f36ea0entries: 4, valid, lsm{ffff88028c6c13f8 0x0BD60BD0 1 4}: LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{ LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d16a208[0] LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d16a188 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 } LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890 LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{ LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0] LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}: LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{ LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0] LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7c220[0x0, 1, [0x280000405:0x427:0x0] hash] LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960868[0x4, 2, [0x300000401:0xda:0x0] hash]{ LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800809608e8[0] LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802779b1fa8id: 0x300000401:218 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448892 atime: 1699448892 ctime: 1699448892 blocks: 0 LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960868 LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bc008[0x4, 1, [0x240000404:0x7dd:0x0] hash]{ LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bc088(0) inode: ffff880267308a88 162129603832317917/37748740 100644 1 0 ffff8802a00bc088 [0x240000404:0x7dd:0x0] LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34d58entries: 1, valid, lsm{ffff880275d5d298 0x0BD10BD0 1 0}: LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960868[0x4, 2, [0x300000401:0xda:0x0] hash]{ LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800809608e8[0] LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802779b1fa8id: 0x300000401:218 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448892 atime: 1699448892 ctime: 1699448892 blocks: 0 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960868 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bc008 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) header@ffff88026ee99f70[0x0, 1, [0x280000405:0x36a:0x0] hash] LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280adb890 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec0868[0x0, 1, [0x240000403:0x8ec:0x0] hash] LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{ LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e19360[0] LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0 LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e192e0 LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff880280eecc98[0x4, 1, [0x280000404:0x5c5:0x0] hash]{ LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280eecd18(0) inode: ffff880267324388 180144002341799365/41943044 100644 1 0 ffff880280eecd18 [0x280000404:0x5c5:0x0] LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880279ccf9b8entries: 1, valid, lsm{ffff88028ac695f8 0x0BD10BD0 1 0}: LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{ LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e19360[0] LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e192e0 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280eecc98 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) header@ffff88026ee9b248[0x0, 1, [0x200000403:0x65e:0x0] hash] LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{ LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b6208[0] LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0 LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b6188 LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cee18[0x4, 1, [0x280000405:0x301:0x0] hash]{ LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cee98(0) inode: ffff8802673b2708 180144002358575873/41943044 100644 1 0 ffff88027f6cee98 [0x280000405:0x301:0x0] LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e6139b8entries: 4, valid, lsm{ffff88008770a0a8 0x0BD60BD0 1 4}: LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{ LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b6208[0] LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b6188 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 } LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cee18 LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) header@ffff880271acf678[0x0, 1, [0x200000403:0x632:0x0] hash] LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{ LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88026d1c1578[0] LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72 LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) } header@ffff88026d1c14f8 LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) header@ffff880280ad8650[0x4, 1, [0x280000405:0x57e:0x0] hash]{ LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280ad86d0(0) inode: ffff880262b00108 180144002358576510/41943044 100755 1 0 ffff880280ad86d0 [0x280000405:0x57e:0x0] LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800877f1f18entries: 1, valid, lsm{ffff880296950a28 0x0BD10BD0 1 0}: LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{ LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88026d1c1578[0] LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72 LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) } header@ffff88026d1c14f8 LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280ad8650 LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 16216:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8c98[0x0, 1, [0x240000404:0xb97:0x0] hash] LustreError: 16216:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) header@ffff88026eebf678[0x4, 2, [0x340000401:0xd8:0x0] hash]{ LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88026eebf6f8[0] LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088ad6788id: 0x340000401:216 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448925 atime: 1699448925 ctime: 1699448925 blocks: 0 LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) } header@ffff88026eebf678 LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18650[0x4, 1, [0x240000403:0x7c6:0x0] hash]{ LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef186d0(0) inode: ffff8802673bba08 162129603815540678/37748740 100644 1 0 ffff88028ef186d0 [0x240000403:0x7c6:0x0] LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080734008entries: 1, valid, lsm{ffff880286b4fcc8 0x0BD10BD0 1 0}: LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff88026eebf678[0x4, 2, [0x340000401:0xd8:0x0] hash]{ LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88026eebf6f8[0] LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088ad6788id: 0x340000401:216 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448925 atime: 1699448925 ctime: 1699448925 blocks: 0 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff88026eebf678 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18650 LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) header@ffff88028da1f030[0x0, 1, [0x280000405:0x37c:0x0] hash] LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) try to own. LustreError: 9113:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x6da:0x0]: rc = -2 LustreError: 9113:0:(mdd_object.c:3621:mdd_close()) Skipped 1 previous similar message Lustre: dir [0x240000404:0xc1f:0x0] stripe 3 readdir failed: -2, directory is partially accessed! Lustre: Skipped 7 previous similar messages LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) header@ffff88027f589928[0x4, 2, [0x380000400:0xfa:0x0] hash]{ LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027f5899a8[0] LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027e13fba8id: 0x380000400:250 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448999 blocks: 72 LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027f589928 LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned. LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) header@ffff880271c729e8[0x4, 1, [0x280000405:0x647:0x0] hash]{ LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880271c72a68(0) inode: ffff88025ec76988 180144002358576711/41943044 100700 1 0 ffff880271c72a68 [0x280000405:0x647:0x0] LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efba7f8entries: 1, valid, lsm{ffff88008a5a3b18 0x0BD10BD0 1 0}: LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 } LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f589928[0x4, 2, [0x380000400:0xfa:0x0] hash]{ LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027f5899a8[0] LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027e13fba8id: 0x380000400:250 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448999 blocks: 72 LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f589928 LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) } header@ffff880271c729e8 LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) owned. LustreError: 18442:0:(lov_object.c:186:lov_init_sub()) header@ffff8802822d3cc0[0x0, 1, [0x200000404:0x7bc:0x0] hash] LustreError: 18442:0:(lov_object.c:186:lov_init_sub()) try to own. ptlrpc_watchdog_fire: 1 callbacks suppressed Lustre: ll_ost01_002: service thread pid 10907 was inactive for 40.094 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: Lustre: Skipped 2 previous similar messages Pid: 10907, comm: ll_ost01_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] tgt_extent_lock+0xea/0x2a0 [ptlrpc] [<0>] ofd_getattr_hdl+0x385/0x750 [ofd] [<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc] [<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Lustre: 10976:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 Lustre: 10976:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 63 previous similar messages Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000404:0x97f:0x0] with magic=0xbd60bd0 Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 21 previous similar messages LustreError: 9112:0:(llog_osd.c:624:llog_osd_write_rec()) lustre-MDT0000-osp-MDT0002: index 1259 already set in llog bitmap [0x200000402:0x2:0x0] | Link to test |
| racer test 1: racer on clients: centos-0.localnet DURATION=2700 | LustreError: 19981:0:(llog_osd.c:630:llog_osd_write_rec()) LBUG Pid: 19981, comm: mdt_rdpg06_001 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] libcfs_call_trace+0x90/0xf0 [libcfs] [<0>] lbug_with_loc+0x4c/0xa0 [libcfs] [<0>] llog_osd_write_rec+0x1b54/0x1b60 [obdclass] [<0>] llog_write_rec+0x290/0x590 [obdclass] [<0>] llog_cat_add_rec+0x1e1/0x990 [obdclass] [<0>] llog_add+0x17f/0x1f0 [obdclass] [<0>] sub_updates_write+0x303/0xe3e [ptlrpc] [<0>] top_trans_stop+0x49a/0xfb0 [ptlrpc] [<0>] lod_trans_stop+0x25c/0x340 [lod] [<0>] mdd_trans_stop+0x28/0x16e [mdd] [<0>] mdd_attr_set+0x897/0x1200 [mdd] [<0>] mdt_mfd_close+0x76e/0xbb0 [mdt] [<0>] mdt_close_internal+0xb4/0x240 [mdt] [<0>] mdt_close+0x291/0x9a0 [mdt] [<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc] [<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe | Lustre: 19949:0:(mdt_recovery.c:150:mdt_req_from_lrd()) @@@ restoring transno req@ffff88009bf05b40 x1756624771794368/t4294968090(0) o101->61984978-aa7d-4c6a-adf8-385000fbd621@0@lo:398/0 lens 400/816 e 0 to 0 dl 1675248003 ref 1 fl Interpret:H/2/0 rc 0/0 job:'cat.0' LustreError: 27984:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000404:0x9e:0x0] mdc close failed: rc = -116 LustreError: 27984:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000404:0x9e:0x0] mdc close failed: rc = -116 Lustre: 21934:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 511 < left 1133, rollback = 2 Lustre: 21934:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/1, destroy: 1/4/0 Lustre: 21934:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 12/1133/0 Lustre: 21934:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 7/105/0 Lustre: 21934:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 13/251/2, delete: 2/5/1 Lustre: 21934:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 1/1/0 LustreError: 19977:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x1:0x0]: rc = -2 LustreError: 28983:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x240000404:0x25:0x0] mdc close failed: rc = -116 LustreError: 28983:0:(file.c:242:ll_close_inode_openhandle()) Skipped 2 previous similar messages LustreError: 28785:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880322108958: inode [0x280000404:0x9e:0x0] mdc close failed: rc = -116 LustreError: 28785:0:(file.c:242:ll_close_inode_openhandle()) Skipped 2 previous similar messages Lustre: 22802:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 512 < left 805, rollback = 2 Lustre: 22802:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 14 previous similar messages Lustre: 22802:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 2/8/0, destroy: 1/4/0 Lustre: 22802:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 22802:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 10/805/0 Lustre: 22802:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 22802:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 1/10/0, punch: 0/0/0, quota 1/3/0 Lustre: 22802:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 22802:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 12/194/2, delete: 3/6/1 Lustre: 22802:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 22802:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 3/3/0 Lustre: 22802:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 23877:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 506 < left 671, rollback = 2 Lustre: 23877:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 14 previous similar messages Lustre: 23877:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/5, destroy: 0/0/0 Lustre: 23877:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 23877:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0 Lustre: 23877:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 23877:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0 Lustre: 23877:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 23877:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/4, delete: 0/0/0 Lustre: 23877:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 23877:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 Lustre: 23877:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 14 previous similar messages Lustre: 19954:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 502 < left 955, rollback = 2 Lustre: 19954:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 7 previous similar messages Lustre: 19954:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/8, destroy: 1/4/0 Lustre: 19954:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 19954:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 10/955/0 Lustre: 19954:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 19954:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 1/3/0 Lustre: 19954:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 19954:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 14/287/4, delete: 2/5/1 Lustre: 19954:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 7 previous similar messages Lustre: 19954:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 1/1/0 Lustre: 19954:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 7 previous similar messages LustreError: 29192:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0xf7:0x0]: rc = -2 LustreError: 32063:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880322108958: inode [0x240000403:0xf7:0x0] mdc close failed: rc = -2 LustreError: 32063:0:(file.c:242:ll_close_inode_openhandle()) Skipped 1 previous similar message LustreError: 29526:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '17' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 17' to finish migration. LustreError: 19979:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0xae:0x0]: rc = -2 Lustre: 19956:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 509 < left 582, rollback = 2 Lustre: 19956:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 78 previous similar messages Lustre: 19956:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/3, destroy: 0/0/0 Lustre: 19956:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 78 previous similar messages Lustre: 19956:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 6/582/0 Lustre: 19956:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 78 previous similar messages Lustre: 19956:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0 Lustre: 19956:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 78 previous similar messages Lustre: 19956:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/3, delete: 0/0/0 Lustre: 19956:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 78 previous similar messages Lustre: 19956:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 Lustre: 19956:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 78 previous similar messages Lustre: 20791:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 LustreError: 1933:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000403:0x1b5:0x0] mdc close failed: rc = -116 LustreError: 1933:0:(file.c:242:ll_close_inode_openhandle()) Skipped 2 previous similar messages Lustre: dir [0x200000404:0x3e:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: 29526:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 504 < left 671, rollback = 2 Lustre: 29526:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 62 previous similar messages Lustre: 29526:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/8, destroy: 0/0/0 Lustre: 29526:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 64 previous similar messages Lustre: 29526:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0 Lustre: 29526:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 64 previous similar messages Lustre: 29526:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0 Lustre: 29526:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 64 previous similar messages Lustre: 29526:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/3, delete: 0/0/0 Lustre: 29526:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 64 previous similar messages Lustre: 29526:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 Lustre: 29526:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 64 previous similar messages Lustre: dir [0x280000403:0x1cf:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: 19947:0:(mdt_recovery.c:150:mdt_req_from_lrd()) @@@ restoring transno req@ffff8802c155a740 x1756624777229056/t4294972655(0) o101->12ddc59d-106f-448a-9d1b-b5d57dd1e845@0@lo:499/0 lens 376/816 e 0 to 0 dl 1675248104 ref 1 fl Interpret:H/2/0 rc 0/0 job:'dd.0' Lustre: dir [0x200000403:0x63:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: 23449:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 Lustre: 23449:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 1 previous similar message Lustre: 23874:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 618, rollback = 7 Lustre: 23874:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 3 previous similar messages LustreError: 6447:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x48:0x0]: rc = -2 LustreError: 7180:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x200000404:0x48:0x0] mdc close failed: rc = -2 LustreError: 7180:0:(file.c:242:ll_close_inode_openhandle()) Skipped 1 previous similar message Lustre: 20773:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20773:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 3 previous similar messages Lustre: 19960:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 499 < left 877, rollback = 2 Lustre: 19960:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 128 previous similar messages Lustre: 19960:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/12, destroy: 0/0/0 Lustre: 19960:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 138 previous similar messages Lustre: 19960:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 11/877/0 Lustre: 19960:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 138 previous similar messages Lustre: 19960:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0 Lustre: 19960:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 138 previous similar messages Lustre: 19960:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 12/215/4, delete: 1/1/0 Lustre: 19960:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 138 previous similar messages Lustre: 19960:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 1/1/0 Lustre: 19960:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 138 previous similar messages Lustre: 20769:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20769:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 1 previous similar message Lustre: 19948:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x232:0x0] with magic=0xbd60bd0 Lustre: 25133:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x223:0x0] with magic=0xbd60bd0 Lustre: 25133:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 5 previous similar messages Lustre: 19965:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x247:0x0] with magic=0xbd60bd0 Lustre: 19965:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 7 previous similar messages LustreError: 19973:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000404:0x286:0x0]: rc = -2 LustreError: 19973:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message 4[12413]: segfault at 8 ip 00007fddfd3907e8 sp 00007ffff7d6c7e0 error 4 in ld-2.17.so[7fddfd385000+22000] Lustre: 718:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000404:0x29a:0x0] with magic=0xbd60bd0 Lustre: 718:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 1 previous similar message Lustre: 13413:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 Lustre: 13413:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 1 previous similar message 9[14785]: segfault at 8 ip 00007f1d2e93c7e8 sp 00007ffdb5053d20 error 4 in ld-2.17.so[7f1d2e931000+22000] LustreError: 24027:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0x36b:0x0]: rc = -2 LustreError: 24027:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message LustreError: 19948:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '14' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 14' to finish migration. Lustre: 26533:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 501 < left 1116, rollback = 2 Lustre: 26533:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 363 previous similar messages Lustre: 26533:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/10, destroy: 0/0/0 Lustre: 26533:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 369 previous similar messages Lustre: 26533:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 12/1116/0 Lustre: 26533:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 369 previous similar messages Lustre: 26533:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0 Lustre: 26533:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 369 previous similar messages Lustre: 26533:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/4, delete: 0/0/0 Lustre: 26533:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 369 previous similar messages Lustre: 26533:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 Lustre: 26533:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 369 previous similar messages LustreError: 13527:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0001: '6' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 6' to finish migration. LustreError: 22826:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '14' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 14' to finish migration. Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x1a9:0x0] with magic=0xbd60bd0 Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 1 previous similar message LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{ LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) ....lovsub@ffff8802b044e208[0] LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72 LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) } header@ffff8802b044e188 LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) stripe 0 is already owned. LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) header@ffff880098dc4220[0x4, 1, [0x240000403:0x1c6:0x0] hash]{ LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....vvp@ffff880098dc42a0(0) inode: ffff88023bf641d8 162129603815539142/37748740 100755 1 0 ffff880098dc42a0 [0x240000403:0x1c6:0x0] LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....lov@ffff8802c2753780entries: 1, valid, lsm{ffff88008b434a28 0x0BD10BD0 1 0}: LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 1048576 } LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{ LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....lovsub@ffff8802b044e208[0] LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72 LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) } header@ffff8802b044e188 LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) } header@ffff880098dc4220 LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) owned. LustreError: 19016:0:(lov_object.c:192:lov_init_sub()) header@ffff8802ad033aa8[0x0, 1, [0x200000404:0x179:0x0] hash] LustreError: 19016:0:(lov_object.c:192:lov_init_sub()) try to own. LustreError: 19016:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x179:0x0]: rc = -5 LustreError: 19016:0:(llite_lib.c:3299:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 19480:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880322108958: inode [0x280000403:0x403:0x0] mdc close failed: rc = -116 LustreError: 19480:0:(file.c:242:ll_close_inode_openhandle()) Skipped 12 previous similar messages Lustre: 20790:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20790:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 5 previous similar messages Lustre: dir [0x200000403:0x1a1:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages LustreError: 25390:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0000: '17' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 17' to finish migration. Lustre: ll_ost05_003: service thread pid 23694 was inactive for 62.227 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: Pid: 23694, comm: ll_ost05_003 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] tgt_extent_lock+0xea/0x2a0 [ptlrpc] [<0>] ofd_getattr_hdl+0x385/0x750 [ofd] [<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc] [<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe LustreError: 19970:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0x379:0x0]: rc = -2 LustreError: 19970:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{ LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) ....lovsub@ffff8802b044e208[0] LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72 LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) } header@ffff8802b044e188 LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) stripe 0 is already owned. LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) header@ffff880098dc4220[0x4, 1, [0x240000403:0x1c6:0x0] hash]{ LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....vvp@ffff880098dc42a0(0) inode: ffff88023bf641d8 162129603815539142/37748740 100755 1 0 ffff880098dc42a0 [0x240000403:0x1c6:0x0] LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....lov@ffff8802c2753780entries: 1, valid, lsm{ffff88008b434a28 0x0BD10BD0 1 0}: LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 1048576 } LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{ LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....lovsub@ffff8802b044e208[0] LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72 LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) } header@ffff8802b044e188 LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) } header@ffff880098dc4220 LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) owned. LustreError: 23192:0:(lov_object.c:192:lov_init_sub()) header@ffff8800994bc650[0x0, 1, [0x200000404:0x179:0x0] hash] LustreError: 23192:0:(lov_object.c:192:lov_init_sub()) try to own. LustreError: 23192:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x179:0x0]: rc = -5 LustreError: 23192:0:(llite_lib.c:3299:ll_prep_inode()) new_inode -fatal: rc -5 LustreError: 23008:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0001: '17' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 17' to finish migration. LustreError: 23008:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 1 previous similar message Lustre: dir [0x200000404:0x20f:0x0] stripe 1 readdir failed: -2, directory is partially accessed! 2[26717]: segfault at 8 ip 00007f93b1f157e8 sp 00007ffcaa4ff550 error 4 in ld-2.17.so[7f93b1f0a000+22000] 3[26973]: segfault at 8 ip 00007f73cb2427e8 sp 00007ffe34d64370 error 4 in ld-2.17.so[7f73cb237000+22000] Lustre: dir [0x200000404:0x2a3:0x0] stripe 1 readdir failed: -2, directory is partially accessed! 4[28052]: segfault at 8 ip 00007fa990fac7e8 sp 00007fff53409370 error 4 in ld-2.17.so[7fa990fa1000+22000] LustreError: 19951:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '12' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 12' to finish migration. Lustre: 20787:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20787:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 11 previous similar messages LustreError: 4218:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x35c:0x0]: rc = -2 LustreError: 4218:0:(mdd_object.c:3458:mdd_close()) Skipped 2 previous similar messages Lustre: dir [0x240000403:0x628:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages LustreError: 19937:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: filter-lustre-OST0000_UUID lock: ffff8802c72ed2c0/0x8c82fea2730d0c21 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 0@lo remote: 0x8c82fea2730d0c0c expref: 30 pid: 2182 timeout: 77798 lvb_type: 0 LustreError: 11-0: lustre-OST0000-osc-ffff880322108958: operation ldlm_enqueue to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-ffff880322108958: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 167-0: lustre-OST0000-osc-ffff880322108958: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x280000404:0x9bc:0x0]// may get corrupted (rc -108) Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x280000404:0xa19:0x0]// may get corrupted (rc -108) LustreError: 1050:0:(ldlm_resource.c:1124:ldlm_resource_complain()) lustre-OST0000-osc-ffff880322108958: namespace resource [0x2c0000400:0xd3:0x0].0x0 (ffff8802de1a0cc0) refcount nonzero (1) after lock cleanup; forcing cleanup. Lustre: mdt07_003: service thread pid 23101 was inactive for 62.068 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: Pid: 23101, comm: mdt07_003 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] mdt_object_local_lock+0x52f/0xba0 [mdt] [<0>] mdt_object_lock_internal+0x70/0x390 [mdt] [<0>] mdt_reint_object_lock+0x2c/0x60 [mdt] [<0>] mdt_reint_striped_lock+0x89/0x6a0 [mdt] [<0>] mdt_attr_set+0xb0/0x910 [mdt] [<0>] mdt_reint_setattr+0x71a/0x1240 [mdt] [<0>] mdt_reint_rec+0x87/0x240 [mdt] [<0>] mdt_reint_internal+0x76c/0xba0 [mdt] [<0>] mdt_reint+0x67/0x150 [mdt] [<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc] [<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Pid: 28540, comm: mdt01_010 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] mdt_object_local_lock+0x52f/0xba0 [mdt] [<0>] mdt_object_lock_internal+0x70/0x390 [mdt] [<0>] mdt_reint_object_lock+0x2c/0x60 [mdt] [<0>] mdt_reint_striped_lock+0x89/0x6a0 [mdt] [<0>] mdt_attr_set+0xb0/0x910 [mdt] [<0>] mdt_reint_setattr+0x71a/0x1240 [mdt] [<0>] mdt_reint_rec+0x87/0x240 [mdt] [<0>] mdt_reint_internal+0x76c/0xba0 [mdt] [<0>] mdt_reint+0x67/0x150 [mdt] [<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc] [<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Lustre: mdt06_003: service thread pid 22833 was inactive for 62.211 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. Lustre: lustre-OST0000: haven't heard from client 61984978-aa7d-4c6a-adf8-385000fbd621 (at 0@lo) in 32 seconds. I think it's dead, and I am evicting it. exp ffff8802a2e16678, cur 1675248188 expire 1675248158 last 1675248156 LustreError: 19937:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0001_UUID lock: ffff88008ce00040/0x8c82fea2731808b5 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x286:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x8c82fea2731808a7 expref: 249 pid: 30655 timeout: 77837 lvb_type: 0 LustreError: 11-0: lustre-MDT0001-mdc-ffff880098fe92a8: operation mds_close to node 0@lo failed: rc = -107 Lustre: lustre-MDT0001-mdc-ffff880098fe92a8: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 167-0: lustre-MDT0001-mdc-ffff880098fe92a8: This client was evicted by lustre-MDT0001; in progress operations using this service will fail. LustreError: 31838:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x240000403:0x660:0x0] mdc close failed: rc = -108 LustreError: 31838:0:(file.c:242:ll_close_inode_openhandle()) Skipped 14 previous similar messages LustreError: 32652:0:(llite_lib.c:3299:ll_prep_inode()) new_inode -fatal: rc -2 LustreError: 30669:0:(mdc_request.c:1474:mdc_read_page()) lustre-MDT0001-mdc-ffff880098fe92a8: [0x240000403:0x5f5:0x0] lock enqueue fails: rc = -108 LustreError: 32425:0:(llite_lib.c:1959:ll_md_setattr()) md_setattr fails: rc = -108 Lustre: lustre-MDT0001-mdc-ffff880098fe92a8: Connection restored to (at 0@lo) Lustre: mdt05_011: service thread pid 8426 was inactive for 40.062 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. Lustre: 25090:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 501 < left 1027, rollback = 2 Lustre: 25090:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 782 previous similar messages Lustre: 25090:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/10, destroy: 0/0/0 Lustre: 25090:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 800 previous similar messages Lustre: 25090:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 11/1027/0 Lustre: 25090:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 800 previous similar messages Lustre: 25090:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0 Lustre: 25090:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 800 previous similar messages Lustre: 25090:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/4, delete: 0/0/0 Lustre: 25090:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 800 previous similar messages Lustre: 25090:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 Lustre: 25090:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 800 previous similar messages Lustre: lustre-OST0000-osc-ffff880322108958: Connection restored to (at 0@lo) LustreError: 11-0: lustre-OST0000-osc-ffff880322108958: operation ldlm_enqueue to node 0@lo failed: rc = -107 Lustre: lustre-OST0000-osc-ffff880322108958: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 167-0: lustre-OST0000-osc-ffff880322108958: This client was evicted by lustre-OST0000; in progress operations using this service will fail. 2[9003]: segfault at 0 ip (null) sp 00007ffdf03b5d38 error 14 in 2[400000+6000] Lustre: dir [0x280000404:0x822:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: 19948:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000403:0x909:0x0] with magic=0xbd60bd0 Lustre: 19948:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 29 previous similar messages Lustre: 20782:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7 Lustre: 20782:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 15 previous similar messages LustreError: 22791:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '9' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 9' to finish migration. LustreError: 19979:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0xa15:0x0]: rc = -2 LustreError: 19979:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message Lustre: dir [0x200000404:0x53c:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 14 previous similar messages LustreError: 29526:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0001: '11' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 11' to finish migration. LustreError: 29526:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 9 previous similar messages Lustre: 22872:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x414:0x0] with magic=0xbd60bd0 Lustre: 22872:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 23 previous similar messages 13[30765]: segfault at 8 ip 00007fe3727e97e8 sp 00007ffceba597d0 error 4 in ld-2.17.so[7fe3727de000+22000] 19[30568]: segfault at 8 ip 00007fdad6ff77e8 sp 00007ffc2d500850 error 4 in ld-2.17.so[7fdad6fec000+22000] Lustre: 16533:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0 Lustre: 16533:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 850 previous similar messages Lustre: 16533:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0 Lustre: 16533:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 850 previous similar messages Lustre: 16533:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 4/150/0 Lustre: 16533:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 850 previous similar messages Lustre: 16533:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0 Lustre: 16533:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 850 previous similar messages Lustre: 16533:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 Lustre: 16533:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 850 previous similar messages Lustre: 19958:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x6da:0x0] with magic=0xbd60bd0 Lustre: 19958:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 13 previous similar messages Lustre: 20809:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 508 < left 728, rollback = 2 Lustre: 20809:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 762 previous similar messages LustreError: 394:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000403:0xd60:0x0] mdc close failed: rc = -2 LustreError: 394:0:(file.c:242:ll_close_inode_openhandle()) Skipped 28 previous similar messages 18[2874]: segfault at 8 ip 00007f2cc9b6b7e8 sp 00007ffc4eed7730 error 4 in ld-2.17.so[7f2cc9b60000+22000] Lustre: 23112:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 515 < left 558, rollback = 7 Lustre: 23112:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 105 previous similar messages 9[7688]: segfault at 8 ip 00007f72b4e9d7e8 sp 00007ffd8cc0b8d0 error 4 in ld-2.17.so[7f72b4e92000+22000] 9[14105]: segfault at 8 ip 00007fd1bd9767e8 sp 00007ffdadd01120 error 4 in ld-2.17.so[7fd1bd96b000+22000] Lustre: 24476:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x4c5:0x0] with magic=0xbd60bd0 Lustre: 24476:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 9 previous similar messages LustreError: 19968:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000403:0x544:0x0]: rc = -2 LustreError: 19968:0:(mdd_object.c:3458:mdd_close()) Skipped 7 previous similar messages Lustre: dir [0x280000403:0x12ec:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 13 previous similar messages LustreError: 19944:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0000: '15' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 15' to finish migration. LustreError: 19944:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 12 previous similar messages 7[25140]: segfault at 8 ip 00007fc4177427e8 sp 00007ffd09483e30 error 4 in ld-2.17.so[7fc417737000+22000] 17[27974]: segfault at 0 ip 0000000000403e5f sp 00007ffeb7244300 error 6 in 17[400000+6000] LustreError: 20269:0:(llog_cat.c:739:llog_cat_cancel_arr_rec()) lustre-MDT0002-osp-MDT0001: fail to cancel 1 llog-records: rc = -116 LustreError: 20269:0:(llog_cat.c:775:llog_cat_cancel_records()) lustre-MDT0002-osp-MDT0001: fail to cancel 1 of 1 llog-records: rc = -116 LustreError: 20269:0:(llog_cat.c:739:llog_cat_cancel_arr_rec()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 llog-records: rc = -116 LustreError: 20269:0:(llog_cat.c:739:llog_cat_cancel_arr_rec()) Skipped 4 previous similar messages LustreError: 20269:0:(llog_cat.c:775:llog_cat_cancel_records()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 of 1 llog-records: rc = -116 LustreError: 20269:0:(llog_cat.c:775:llog_cat_cancel_records()) Skipped 4 previous similar messages 11[12593]: segfault at 8 ip 00007f1cf34127e8 sp 00007ffc57803f00 error 4 in ld-2.17.so[7f1cf3407000+22000] 10[25812]: segfault at 0 ip (null) sp 00007fff0649a048 error 14 in 10[400000+6000] LustreError: 2798:0:(mdd_object.c:403:mdd_xattr_get()) lustre-MDD0000: object [0x200000404:0xdb4:0x0] not found: rc = -2 ptlrpc_watchdog_fire: 2 callbacks suppressed Lustre: ll_ost04_009: service thread pid 9443 was inactive for 62.132 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: Lustre: Skipped 1 previous similar message Pid: 9443, comm: ll_ost04_009 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022 Call Trace: [<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc] [<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc] [<0>] tgt_extent_lock+0xea/0x2a0 [ptlrpc] [<0>] ofd_getattr_hdl+0x385/0x750 [ofd] [<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc] [<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc] [<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc] [<0>] kthread+0xe4/0xf0 [<0>] ret_from_fork_nospec_begin+0x7/0x21 [<0>] 0xfffffffffffffffe Lustre: 24327:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0 Lustre: 24327:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 2646 previous similar messages Lustre: 24327:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0 Lustre: 24327:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 2646 previous similar messages Lustre: 24327:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 4/150/0 Lustre: 24327:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 2646 previous similar messages Lustre: 24327:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0 Lustre: 24327:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 2646 previous similar messages Lustre: 24327:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 Lustre: 24327:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 2646 previous similar messages Lustre: 19958:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 502 < left 1086, rollback = 2 Lustre: 19958:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 2502 previous similar messages 2[32653]: segfault at 8 ip 00007fbff35f97e8 sp 00007ffd09143420 error 4 in ld-2.17.so[7fbff35ee000+22000] LustreError: 412:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x200000404:0xe57:0x0] mdc close failed: rc = -2 LustreError: 412:0:(file.c:242:ll_close_inode_openhandle()) Skipped 55 previous similar messages LustreError: 28524:0:(mdd_object.c:403:mdd_xattr_get()) lustre-MDD0000: object [0x200000404:0xe57:0x0] not found: rc = -2 Lustre: 10327:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7 Lustre: 10327:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 121 previous similar messages 2[4969]: segfault at 0 ip (null) sp 00007ffdd00f78f8 error 14 in 10[400000+6000] 13[4733]: segfault at 8 ip 00007fb21aa6e7e8 sp 00007fffdd04f080 error 4 in ld-2.17.so[7fb21aa63000+22000] LustreError: 19937:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 0@lo ns: filter-lustre-OST0001_UUID lock: ffff8800adb4e940/0x8c82fea2735f3e60 lrc: 3/0,0 mode: PW/PW res: [0x300000400:0x24d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 0@lo remote: 0x8c82fea2735f3e59 expref: 18 pid: 9113 timeout: 78253 lvb_type: 0 LustreError: 11-0: lustre-OST0001-osc-ffff880322108958: operation ldlm_enqueue to node 0@lo failed: rc = -107 Lustre: lustre-OST0001-osc-ffff880322108958: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete LustreError: 167-0: lustre-OST0001-osc-ffff880322108958: This client was evicted by lustre-OST0001; in progress operations using this service will fail. Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x280000403:0x1c89:0x0]// may get corrupted (rc -108) Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x240000405:0x1388:0x0]/ may get corrupted (rc -108) LustreError: 5481:0:(ldlm_resource.c:1124:ldlm_resource_complain()) lustre-OST0001-osc-ffff880322108958: namespace resource [0x300000400:0x24d:0x0].0x0 (ffff8802b5338040) refcount nonzero (1) after lock cleanup; forcing cleanup. LustreError: 5481:0:(ldlm_resource.c:1124:ldlm_resource_complain()) Skipped 2 previous similar messages Lustre: lustre-OST0001-osc-ffff880322108958: Connection restored to (at 0@lo) Lustre: Skipped 1 previous similar message Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000405:0x111a:0x0] with magic=0xbd60bd0 Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 19 previous similar messages 13[6072]: segfault at 8 ip 00007fac24b7e7e8 sp 00007fffd295cef0 error 4 in ld-2.17.so[7fac24b73000+22000] 8[7687]: segfault at 8 ip 00007f9ab74657e8 sp 00007fffa6f9a6a0 error 4 in ld-2.17.so[7f9ab745a000+22000] 8[7915]: segfault at 8 ip 00007f02a50407e8 sp 00007ffc44ab61a0 error 4 in ld-2.17.so[7f02a5035000+22000] LustreError: 22748:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '7' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 7' to finish migration. LustreError: 22748:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 31 previous similar messages LustreError: 19978:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0x2394:0x0]: rc = -2 LustreError: 19978:0:(mdd_object.c:3458:mdd_close()) Skipped 19 previous similar messages Lustre: dir [0x240000405:0x1618:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 67 previous similar messages 3[10077]: segfault at 8 ip 00007fc645a9c7e8 sp 00007ffd5cb08330 error 4 in ld-2.17.so[7fc645a91000+22000] LustreError: 17523:0:(llite_nfs.c:338:ll_dir_get_parent_fid()) lustre: failure inode [0x200000404:0x10bf:0x0] get parent: rc = -116 19[19380]: segfault at 8 ip 00007f79677047e8 sp 00007ffe8a169b90 error 4 in ld-2.17.so[7f79676f9000+22000] LustreError: 23560:0:(llite_lib.c:1959:ll_md_setattr()) md_setattr fails: rc = -5 LustreError: 19981:0:(llog_osd.c:628:llog_osd_write_rec()) lustre-MDT0001-osp-MDT0002: index 2522 already set in llog bitmap [0x240000401:0x2:0x0] | Link to test |