LustreError: 18588:0:(llog_osd.c:630:llog_osd_write_rec()) LBUG
CPU: 7 PID: 18588 Comm: mdt_rdpg03_000 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
Call Trace:
[<ffffffff817d93f8>] dump_stack+0x19/0x1b
[<ffffffffa018aa9d>] lbug_with_loc+0x4d/0xb0 [libcfs]
[<ffffffff817df60e>] ? mutex_unlock+0xe/0x10
[<ffffffffa03708b9>] llog_osd_write_rec+0x1ba9/0x1c70 [obdclass]
[<ffffffffa0359800>] llog_write_rec+0x290/0x590 [obdclass]
[<ffffffffa0360339>] llog_cat_add_rec+0x1d9/0xa50 [obdclass]
[<ffffffffa03566bf>] llog_add+0x17f/0x1f0 [obdclass]
[<ffffffffa06fb6fd>] sub_updates_write+0x31a/0xeb4 [ptlrpc]
[<ffffffffa06fb3d2>] ? prepare_writing_updates.isra.10+0x144/0x155 [ptlrpc]
[<ffffffffa06da2f9>] top_trans_stop+0x439/0xf40 [ptlrpc]
[<ffffffffa1379282>] lod_trans_stop+0x72/0x360 [lod]
[<ffffffffa123f908>] mdd_trans_stop+0x28/0x16e [mdd]
[<ffffffffa12305ac>] mdd_attr_set+0x67c/0xdd0 [mdd]
[<ffffffffa0ae62e9>] ? osd_read_unlock+0x39/0x40 [osd_zfs]
[<ffffffffa1396600>] ? lod_write_locked+0x50/0xc0 [lod]
[<ffffffffa12c046e>] mdt_mfd_close+0x76e/0xbb0 [mdt]
[<ffffffffa12c6594>] mdt_close_internal+0xb4/0x240 [mdt]
[<ffffffffa12c69ac>] mdt_close+0x28c/0x980 [mdt]
[<ffffffffa06c170e>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<ffffffffa0609ce1>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<ffffffffa060bace>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<ffffffff810dbb51>] ? put_prev_entity+0x31/0x400
[<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90
[<ffffffffa060ae50>] ? ptlrpc_wait_event+0x630/0x630 [ptlrpc]
[<ffffffff810ba114>] kthread+0xe4/0xf0
[<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140
[<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140

ln (29139) used greatest stack depth: 10016 bytes left
LustreError: 18574:0:(mdt_handler.c:776:mdt_pack_acl2body()) lustre-MDT0002: unable to read [0x280000403:0x1d8:0x0] ACL: rc = -2
5[21689]: segfault at 8 ip 00007f784097a7e8 sp 00007ffe92cb5c80 error 4 in ld-2.17.so[7f784096f000+22000]
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/6 failed: rc = -114
LustreError: 22170:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x280000403:0x204:0x0] mdc close failed: rc = -116
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/1 failed: rc = -16
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 1 previous similar message
LustreError: 22324:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff88026ec5e678: inode [0x200000403:0x4f5:0x0] mdc close failed: rc = -116
Lustre: 18561:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff880165d2c140 x1812631222565888/t4294973393(0) o101->e3c3af9a-e9ae-4fd8-b595-b2a342385f51@0@lo:706/0 lens 376/864 e 0 to 0 dl 1728660031 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/1 failed: rc = -114
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 3 previous similar messages
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/7 failed: rc = -16
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 1 previous similar message
LustreError: 5469:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x280000404:0x3cc:0x0] mdc close failed: rc = -116
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/7 failed: rc = -16
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 3 previous similar messages
LustreError: 5469:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x280000404:0x3cc:0x0] mdc close failed: rc = -116
LustreError: 5469:0:(file.c:262:ll_close_inode_openhandle()) Skipped 3 previous similar messages
LustreError: 2426:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x200000403:0x5d1:0x0] mdc close failed: rc = -116
LustreError: 18623:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x2:0x0]/3 failed: rc = -16
LustreError: 18623:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 4 previous similar messages
LustreError: 2420:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x200000403:0x7a5:0x0] mdc close failed: rc = -116
LustreError: 2420:0:(file.c:262:ll_close_inode_openhandle()) Skipped 3 previous similar messages
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0001: migrate [0x240000403:0x1:0x0]/18 failed: rc = -16
LustreError: 18641:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 1 previous similar message
LustreError: 21252:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x240000404:0x630:0x0] mdc close failed: rc = -116
16[26672]: segfault at 0 ip (null) sp 00007ffdc7a21028 error 14 in 16[400000+6000]
16[26690]: segfault at 0 ip (null) sp 00007ffc5182cdd8 error 14 in 16[400000+6000]
LustreError: 18622:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/19 failed: rc = -16
LustreError: 18622:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 14 previous similar messages
LustreError: 3124:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x200000404:0xa59:0x0] mdc close failed: rc = -116
LustreError: 3124:0:(file.c:262:ll_close_inode_openhandle()) Skipped 14 previous similar messages
LustreError: 18636:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0000: '7' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 7' to finish migration: rc = -1
LustreError: 28677:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x796:0x0]: rc = -5
LustreError: 28677:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 12564:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0xb08:0x0]: rc = -5
LustreError: 12564:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 5132:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0xb08:0x0]: rc = -5
LustreError: 5132:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 5132:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 5132:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 1 previous similar message
Lustre: 8396:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff880125084140 x1812631241765632/t4294978230(0) o101->1f7f26d1-5499-45b9-85da-c0995980e369@0@lo:167/0 lens 376/840 e 0 to 0 dl 1728660247 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
LustreError: 16899:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x796:0x0]: rc = -5
LustreError: 16899:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 4933:0:(llite_lib.c:1867:ll_update_lsm_md()) lustre: [0x200000404:0xaa2:0x0] dir layout mismatch:
LustreError: 4933:0:(lustre_lmv.h:160:lmv_stripe_object_dump()) dump LMV: refs 215092432 magic=0x3 count=2 index=0 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool=
LustreError: 4933:0:(lustre_lmv.h:167:lmv_stripe_object_dump()) stripe[0] [0x200000400:0xe:0x0]
LustreError: 4933:0:(lustre_lmv.h:160:lmv_stripe_object_dump()) dump LMV: refs 215092432 magic=0x1 count=4 index=0 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=2 migrate_hash=crush:3 pool=
Lustre: lustre-MDT0001: trigger partial OI scrub for RPC inconsistency, checking FID [0x240000403:0x7a8:0x0]/0xa): rc = 0
Lustre: dir [0x200000404:0xaa2:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
LustreError: 15861:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/18 failed: rc = -114
LustreError: 15861:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 35 previous similar messages
LustreError: 19434:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x280000404:0x774:0x0] mdc close failed: rc = -116
LustreError: 19434:0:(file.c:262:ll_close_inode_openhandle()) Skipped 10 previous similar messages
LustreError: 14924:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0xb99:0x0]: rc = -5
LustreError: 14924:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 17933:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0001: '1' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 1' to finish migration: rc = -1
LustreError: 18623:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0000: '13' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 13' to finish migration: rc = -1
LustreError: 12846:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0001: '15' migration was interrupted, run 'lfs migrate -m 2 -c 2 -H crush 15' to finish migration: rc = -1
LustreError: 14924:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0xb85:0x0]: rc = -5
LustreError: 14924:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 2 previous similar messages
LustreError: 14924:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 14924:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 2 previous similar messages
2[28438]: segfault at 8 ip 00007f2cba5897e8 sp 00007ffe445a5850 error 4 in ld-2.17.so[7f2cba57e000+22000]
10[32700]: segfault at 8 ip 00007f3c278007e8 sp 00007ffe4046a600 error 4 in ld-2.17.so[7f3c277f5000+22000]
Lustre: 23072:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x842:0x0] with magic=0xbd60bd0
LustreError: 28126:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x796:0x0]: rc = -5
LustreError: 28126:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 18627:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0001: '13' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 13' to finish migration: rc = -1
LustreError: 24028:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff88026ec5e678: cannot apply new layout on [0x200000404:0xb08:0x0] : rc = -5
LustreError: 24028:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x200000404:0xb08:0x0] error -5.
6[7854]: segfault at 8 ip 00007f49027f27e8 sp 00007ffea5888360 error 4 in ld-2.17.so[7f49027e7000+22000]
Lustre: 3306:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x84c:0x0] with magic=0xbd60bd0
Lustre: 3306:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 1 previous similar message
Lustre: 2756:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0xce5:0x0] with magic=0xbd60bd0
Lustre: 2756:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 1 previous similar message
Lustre: dir [0x240000404:0x985:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
Lustre: dir [0x240000403:0x9a0:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: mdt02_011: service thread pid 12844 was inactive for 40.049 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Pid: 12844, comm: mdt02_011 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x913/0xd50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x890 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1b3/0x470 [mdt]
[<0>] mdt_object_lock_try+0xa0/0x250 [mdt]
[<0>] mdt_getattr_name_lock+0x1699/0x2bd0 [mdt]
[<0>] mdt_intent_getattr+0x2cc/0x4e0 [mdt]
[<0>] mdt_intent_opc.constprop.73+0x211/0xc50 [mdt]
[<0>] mdt_intent_policy+0x10f/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x34f/0x930 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x507/0x1850 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 18620, comm: mdt_io00_000 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x913/0xd50 [ptlrpc]
[<0>] ldlm_cli_enqueue_fini+0xa24/0xf20 [ptlrpc]
[<0>] ldlm_cli_enqueue+0x461/0xa70 [ptlrpc]
[<0>] osp_md_object_lock+0x151/0x2f0 [osp]
[<0>] lod_object_lock+0xdb/0x7d0 [lod]
[<0>] mdd_object_lock+0x2d/0xd0 [mdd]
[<0>] mdt_remote_object_lock_try+0x14c/0x189 [mdt]
[<0>] mdt_object_lock_internal+0x3c4/0x470 [mdt]
[<0>] mdt_rename_lock+0xc3/0x2d0 [mdt]
[<0>] mdt_reint_migrate+0x891/0x23f0 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x74c/0xbc0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: mdt00_003: service thread pid 28589 was inactive for 72.123 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Lustre: mdt05_008: service thread pid 8301 was inactive for 72.137 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 1 previous similar message
Pid: 28589, comm: mdt00_003 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x913/0xd50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x890 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1b3/0x470 [mdt]
[<0>] mdt_object_lock+0x88/0x1c0 [mdt]
[<0>] mdt_getattr_name_lock+0xf7/0x2bd0 [mdt]
[<0>] mdt_intent_getattr+0x2cc/0x4e0 [mdt]
[<0>] mdt_intent_opc.constprop.73+0x211/0xc50 [mdt]
[<0>] mdt_intent_policy+0x10f/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x34f/0x930 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x507/0x1850 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: mdt_io04_004: service thread pid 15799 was inactive for 72.129 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: mdt07_008: service thread pid 31946 was inactive for 72.151 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 4 previous similar messages
Lustre: mdt_io02_003: service thread pid 8652 was inactive for 72.253 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 3 previous similar messages
Lustre: mdt_io05_004: service thread pid 11910 was inactive for 74.152 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 2 previous similar messages
LustreError: 18514:0:(ldlm_lockd.c:241:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 0@lo ns: mdt-lustre-MDT0001_UUID lock: ffff88028d086940/0x342584dc8651ffb4 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x84c:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x342584dc8651ffa6 expref: 472 pid: 29431 timeout: 6512 lvb_type: 0
LustreError: 472:0:(ldlm_lockd.c:2575:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1728660439 with bad export cookie 3757555546789456917
Lustre: lustre-MDT0001-mdc-ffff880275b20008: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: mdt05_008: service thread pid 8301 completed after 101.599s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt02_011: service thread pid 12844 completed after 99.463s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt00_003: service thread pid 28589 completed after 101.592s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: lustre-MDT0001-mdc-ffff880275b20008: This client was evicted by lustre-MDT0001; in progress operations using this service will fail.
LustreError: 30473:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x240000403:0x9db:0x0] mdc close failed: rc = -108
LustreError: 30473:0:(file.c:262:ll_close_inode_openhandle()) Skipped 5 previous similar messages
Lustre: mdt_io04_004: service thread pid 15799 completed after 100.852s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt07_008: service thread pid 31946 completed after 99.345s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: lustre-MDT0001-mdc-ffff880275b20008: Connection restored to 192.168.123.17@tcp (at 0@lo)
Lustre: mdt_io04_000: service thread pid 18632 completed after 100.823s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 9360:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000404:0xd33:0x0]/1 failed: rc = -114
LustreError: 9360:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 61 previous similar messages
Lustre: mdt_io03_003: service thread pid 9360 completed after 100.811s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io01_001: service thread pid 18624 completed after 100.835s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io03_004: service thread pid 29822 completed after 100.800s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io04_006: service thread pid 17922 completed after 100.347s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io00_000: service thread pid 18620 completed after 99.582s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io03_002: service thread pid 18631 completed after 99.056s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_004: service thread pid 18760 completed after 98.809s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_000: service thread pid 18641 completed after 98.514s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io03_005: service thread pid 9243 completed after 98.074s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io02_003: service thread pid 8652 completed after 97.970s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io05_004: service thread pid 11910 completed after 95.781s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_002: service thread pid 18643 completed after 92.796s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io03_006: service thread pid 17008 completed after 92.700s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_005: service thread pid 17583 completed after 92.513s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io02_001: service thread pid 18627 completed after 92.142s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: dir [0x200000404:0xd55:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
LustreError: 26165:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff88026ec5e678: cannot apply new layout on [0x200000404:0xb08:0x0] : rc = -5
LustreError: 26165:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0xb08:0x0]: rc = -5
LustreError: 26165:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 26165:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 26165:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 1 previous similar message
Lustre: 3306:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0xd53:0x0] with magic=0xbd60bd0
Lustre: 3306:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 3 previous similar messages
Lustre: mdt04_013: service thread pid 31428 was inactive for 40.053 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 4 previous similar messages
Lustre: mdt_io06_000: service thread pid 18638 was inactive for 72.179 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
LustreError: 18514:0:(ldlm_lockd.c:241:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0001_UUID lock: ffff880025d60400/0x342584dc8653f1be lrc: 3/0,0 mode: PW/PW res: [0x240000403:0xa05:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x342584dc8653f1b0 expref: 474 pid: 28194 timeout: 6615 lvb_type: 0
LustreError: 8590:0:(ldlm_lockd.c:1478:ldlm_handle_enqueue()) ### lock on destroyed export ffff8801e6f392a8 ns: mdt-lustre-MDT0001_UUID lock: ffff88008c70fc00/0x342584dc8659d4ee lrc: 3/0,0 mode: PR/PR res: [0x240000403:0xa05:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000000 nid: 0@lo remote: 0x342584dc86543406 expref: 9 pid: 8590 timeout: 0 lvb_type: 0
LustreError: lustre-MDT0001-mdc-ffff88026ec5e678: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-MDT0001-mdc-ffff88026ec5e678: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: mdt04_013: service thread pid 31428 completed after 100.340s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: lustre-MDT0001-mdc-ffff88026ec5e678: This client was evicted by lustre-MDT0001; in progress operations using this service will fail.
LustreError: 23369:0:(ldlm_resource.c:1149:ldlm_resource_complain()) lustre-MDT0001-mdc-ffff88026ec5e678: namespace resource [0x240000403:0x1:0x0].0x0 (ffff8801116e5bc0) refcount nonzero (1) after lock cleanup; forcing cleanup.
Lustre: lustre-MDT0001-mdc-ffff88026ec5e678: Connection restored to 192.168.123.17@tcp (at 0@lo)
Lustre: mdt_io06_000: service thread pid 18638 completed after 88.678s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io06_004: service thread pid 18136 completed after 82.455s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 27832:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000403:0x976:0x0]: rc = -5
LustreError: 27832:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 11 previous similar messages
LustreError: 27832:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 27832:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 11 previous similar messages
Lustre: mdt_io06_003: service thread pid 7896 completed after 81.144s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 18760:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0002: '3' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 3' to finish migration: rc = -1
Lustre: dir [0x280000404:0x9c8:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 8 previous similar messages
LustreError: 18627:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0000: '7' migration was interrupted, run 'lfs migrate -m 2 -c 3 -H crush 7' to finish migration: rc = -1
LustreError: 18627:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) Skipped 1 previous similar message
LustreError: 15799:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0000: '10' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 10' to finish migration: rc = -1
LustreError: 15799:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) Skipped 1 previous similar message
Lustre: lustre-MDT0002: trigger partial OI scrub for RPC inconsistency, checking FID [0x280000403:0xb8c:0x0]/0xa): rc = 0
Lustre: 32450:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x118f:0x0] with magic=0xbd60bd0
Lustre: 32450:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 3 previous similar messages
Lustre: dir [0x200000403:0xf91:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
LustreError: 26468:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff88026ec5e678: cannot apply new layout on [0x280000404:0xb8a:0x0] : rc = -5
LustreError: 26468:0:(lov_object.c:1340:lov_layout_change()) Skipped 1 previous similar message
LustreError: 26468:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x280000404:0xb8a:0x0] error -5.
LustreError: 27687:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000405:0x3c0:0x0]: rc = -5
LustreError: 27687:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 17 previous similar messages
LustreError: 27687:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 27687:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 17 previous similar messages
LustreError: 18631:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/15 failed: rc = -114
LustreError: 18631:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 103 previous similar messages
LustreError: 12354:0:(mdt_handler.c:776:mdt_pack_acl2body()) lustre-MDT0000: unable to read [0x200000403:0x1324:0x0] ACL: rc = -2
LustreError: 12846:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0001: '10' migration was interrupted, run 'lfs migrate -m 0 -c 3 -H crush 10' to finish migration: rc = -1
LustreError: 12846:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) Skipped 2 previous similar messages
Lustre: 5806:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000405:0x3db:0x0] with magic=0xbd60bd0
Lustre: 5806:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 1 previous similar message
LustreError: 8218:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff88026ec5e678: inode [0x200000404:0x1ed9:0x0] mdc close failed: rc = -116
LustreError: 8218:0:(file.c:262:ll_close_inode_openhandle()) Skipped 55 previous similar messages
Lustre: lustre-MDT0000: trigger partial OI scrub for RPC inconsistency, checking FID [0x200000403:0x1304:0x0]/0xa): rc = 0
LustreError: 10830:0:(osd_index.c:201:__osd_xattr_load_by_oid()) lustre-MDT0000: can't get bonus, rc = -2
Lustre: dir [0x240000405:0x355:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 10 previous similar messages
Lustre: 28194:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000405:0x2c5:0x0] with magic=0xbd60bd0
Lustre: 28194:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 5 previous similar messages
Lustre: 29470:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x156c:0x0] with magic=0xbd60bd0
Lustre: 29470:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 23 previous similar messages
LustreError: 18514:0:(ldlm_lockd.c:241:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 0@lo ns: mdt-lustre-MDT0002_UUID lock: ffff8801f3b07c00/0x342584dc866faea4 lrc: 3/0,0 mode: PR/PR res: [0x280000404:0xe40:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x342584dc866fae34 expref: 294 pid: 29675 timeout: 6923 lvb_type: 0
LustreError: 18633:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0002: '14' migration was interrupted, run 'lfs migrate -m 2 -c 3 -H crush 14' to finish migration: rc = -1
LustreError: 18633:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) Skipped 1 previous similar message
LustreError: lustre-MDT0002-mdc-ffff880275b20008: operation mds_close to node 0@lo failed: rc = -107
Lustre: lustre-MDT0002-mdc-ffff880275b20008: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: lustre-MDT0002-mdc-ffff880275b20008: This client was evicted by lustre-MDT0002; in progress operations using this service will fail.
LustreError: 20938:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x280000404:0xe40:0x0] error -108.
LustreError: 21053:0:(file.c:5733:ll_inode_revalidate_fini()) lustre: revalidate FID [0x280000404:0xe40:0x0] error: rc = -108
Lustre: lustre-MDT0002-mdc-ffff880275b20008: Connection restored to 192.168.123.17@tcp (at 0@lo)
Lustre: 31948:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x207c:0x0] with magic=0xbd60bd0
Lustre: 31948:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 1 previous similar message
Lustre: dir [0x240000405:0x53e:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
4[4095]: segfault at 0 ip (null) sp 00007ffc50d0e6f8 error 14 in 13[400000+6000]
LustreError: 15919:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000404:0xddc:0x0]: rc = -5
LustreError: 15919:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 25 previous similar messages
LustreError: 15919:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 15919:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 25 previous similar messages
0[21732]: segfault at 0 ip (null) sp 00007ffc59d66e78 error 14 in 0[400000+6000]
9[22709]: segfault at 8 ip 00007fbd203a27e8 sp 00007ffff3c1de40 error 4 in ld-2.17.so[7fbd20397000+22000]
0[25662]: segfault at 8 ip 00007fcb284d67e8 sp 00007ffed7b39330 error 4 in ld-2.17.so[7fcb284cb000+22000]
14[28897]: segfault at 8 ip 00007ff55c15a7e8 sp 00007ffd9bd8a040 error 4 in ld-2.17.so[7ff55c14f000+22000]
Lustre: dir [0x280000405:0x1a5:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 6 previous similar messages
ptlrpc_watchdog_fire: 22 callbacks suppressed
Lustre: mdt03_003: service thread pid 28543 was inactive for 40.090 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Pid: 28543, comm: mdt03_003 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x913/0xd50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x890 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1b3/0x470 [mdt]
[<0>] mdt_object_lock+0x88/0x1c0 [mdt]
[<0>] mdt_getattr_name_lock+0xf7/0x2bd0 [mdt]
[<0>] mdt_intent_getattr+0x2cc/0x4e0 [mdt]
[<0>] mdt_intent_opc.constprop.73+0x211/0xc50 [mdt]
[<0>] mdt_intent_policy+0x10f/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x34f/0x930 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x507/0x1850 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 7870, comm: mdt03_007 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x913/0xd50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x890 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1b3/0x470 [mdt]
[<0>] mdt_object_lock+0x88/0x1c0 [mdt]
[<0>] mdt_getattr_name_lock+0xf7/0x2bd0 [mdt]
[<0>] mdt_intent_getattr+0x2cc/0x4e0 [mdt]
[<0>] mdt_intent_opc.constprop.73+0x211/0xc50 [mdt]
[<0>] mdt_intent_policy+0x10f/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x34f/0x930 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x507/0x1850 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 28577, comm: mdt07_004 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x913/0xd50 [ptlrpc]
Lustre: mdt03_015: service thread pid 7231 was inactive for 40.044 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 2 previous similar messages
[<0>] ldlm_cli_enqueue_local+0x259/0x890 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1b3/0x470 [mdt]
[<0>] mdt_object_lock+0x88/0x1c0 [mdt]
[<0>] mdt_getattr_name_lock+0xf7/0x2bd0 [mdt]
[<0>] mdt_intent_getattr+0x2cc/0x4e0 [mdt]
[<0>] mdt_intent_opc.constprop.73+0x211/0xc50 [mdt]
[<0>] mdt_intent_policy+0x10f/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x34f/0x930 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x507/0x1850 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x74e/0x1a60 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x281/0xce0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
LustreError: 18514:0:(ldlm_lockd.c:241:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0002_UUID lock: ffff8801f30f25c0/0x342584dc8680e206 lrc: 3/0,0 mode: PR/PR res: [0x280000403:0x1170:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x342584dc8680e1ea expref: 339 pid: 29431 timeout: 7148 lvb_type: 0
Lustre: mdt03_015: service thread pid 7231 completed after 99.449s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 28543:0:(ldlm_lockd.c:1478:ldlm_handle_enqueue()) ### lock on destroyed export ffff8801e6f39bf8 ns: mdt-lustre-MDT0002_UUID lock: ffff880322c53100/0x342584dc8680e253 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 23 type: IBT gid 0 flags: 0x50200400000020 nid: 0@lo remote: 0x342584dc8680e245 expref: 16 pid: 28543 timeout: 0 lvb_type: 0
Lustre: mdt03_003: service thread pid 28543 completed after 100.911s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: lustre-MDT0002-mdc-ffff88026ec5e678: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-MDT0002-mdc-ffff88026ec5e678: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: mdt03_007: service thread pid 7870 completed after 100.703s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt07_010: service thread pid 32342 completed after 99.503s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt07_003: service thread pid 28529 completed after 99.503s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: lustre-MDT0002-mdc-ffff88026ec5e678: This client was evicted by lustre-MDT0002; in progress operations using this service will fail.
Lustre: mdt07_004: service thread pid 28577 completed after 99.528s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 30102:0:(llite_lib.c:2018:ll_md_setattr()) md_setattr fails: rc = -108
LustreError: 30817:0:(file.c:5733:ll_inode_revalidate_fini()) lustre: revalidate FID [0x240000406:0x660:0x0] error: rc = -5
LustreError: 30817:0:(file.c:5733:ll_inode_revalidate_fini()) Skipped 1 previous similar message
Lustre: lustre-MDT0002-mdc-ffff88026ec5e678: Connection restored to 192.168.123.17@tcp (at 0@lo)
LustreError: 15799:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0002: '15' migration was interrupted, run 'lfs migrate -m 1 -c 2 -H crush 15' to finish migration: rc = -1
LustreError: 15799:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) Skipped 9 previous similar messages
Lustre: 8610:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000403:0xef7:0x0] with magic=0xbd60bd0
Lustre: 8610:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 17 previous similar messages
Lustre: mdt01_005: service thread pid 2078 was inactive for 40.011 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 2 previous similar messages
Lustre: lustre-OST0002-osc-ffff880275b20008: disconnect after 20s idle
LustreError: 18514:0:(ldlm_lockd.c:241:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0002_UUID lock: ffff8800194043c0/0x342584dc8686959b lrc: 3/0,0 mode: PW/PW res: [0x280000405:0x524:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x342584dc86868a11 expref: 243 pid: 15513 timeout: 7286 lvb_type: 0
LustreError: 12846:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/9 failed: rc = -16
Lustre: mdt01_005: service thread pid 2078 completed after 100.444s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt02_017: service thread pid 2917 completed after 97.635s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: lustre-MDT0002-mdc-ffff880275b20008: operation mds_reint to node 0@lo failed: rc = -107
LustreError: Skipped 3 previous similar messages
Lustre: lustre-MDT0002-mdc-ffff880275b20008: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 12846:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 137 previous similar messages
LustreError: lustre-MDT0002-mdc-ffff880275b20008: This client was evicted by lustre-MDT0002; in progress operations using this service will fail.
Lustre: dir [0x280000405:0x53a:0x0] stripe 0 readdir failed: -108, directory is partially accessed!
Lustre: Skipped 23 previous similar messages
LustreError: 16284:0:(llite_lib.c:2018:ll_md_setattr()) md_setattr fails: rc = -108
LustreError: 16284:0:(llite_lib.c:2018:ll_md_setattr()) Skipped 1 previous similar message
LustreError: 13072:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x280000405:0x524:0x0] error -108.
LustreError: 13373:0:(mdc_request.c:1454:mdc_read_page()) lustre-MDT0002-mdc-ffff880275b20008: [0x280000403:0x1:0x0] lock enqueue fails: rc = -108
Lustre: lustre-MDT0002-mdc-ffff880275b20008: Connection restored to 192.168.123.17@tcp (at 0@lo)
LustreError: 19280:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 11 [0x200000404:0x24c6:0x0]: rc = -5
LustreError: 27165:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x280000405:0x42e:0x0] mdc close failed: rc = -2
LustreError: 27165:0:(file.c:262:ll_close_inode_openhandle()) Skipped 76 previous similar messages
LustreError: 21791:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 11 [0x240000405:0x9af:0x0]: rc = -5
LustreError: 29324:0:(lustre_lmv.h:499:lmv_is_sane()) unknown layout LMV: magic=0xcd40cd0 count=3 index=2 hash=crush:0x82000003 version=1 migrate_offset=2 migrate_hash=fnv_1a_64:2 pool=
LustreError: 17732:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 13 [0x200000403:0x2696:0x0]: rc = -5
LustreError: 29675:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0002: [0x280000404:0xda3:0x0] migrate mdt count mismatch 1 != 3
Lustre: 18580:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000405:0xc5f:0x0] with magic=0xbd60bd0
Lustre: 18580:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 7 previous similar messages
LustreError: 22712:0:(llite_nfs.c:446:ll_dir_get_parent_fid()) lustre: failure inode [0x280000406:0x594:0x0] get parent: rc = -116
LustreError: 27619:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x2b01:0x0]: rc = -5
LustreError: 27619:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 113 previous similar messages
LustreError: 27619:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 27619:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 113 previous similar messages
Lustre: dir [0x280000407:0x4f0:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 22 previous similar messages
LustreError: 10603:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff88026ec5e678: cannot apply new layout on [0x200000404:0x2bab:0x0] : rc = -5
LustreError: 10603:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x200000404:0x2bab:0x0] error -5.
LustreError: 17076:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff88026ec5e678: cannot apply new layout on [0x200000404:0x2bab:0x0] : rc = -5
LustreError: 2611:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0000: [0x200000403:0x2d42:0x0] migrate mdt count mismatch 3 != 2
LustreError: 29897:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff88026ec5e678: cannot apply new layout on [0x200000404:0x2d56:0x0] : rc = -5
LustreError: 29897:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x200000404:0x2d56:0x0] error -5.
LustreError: 31755:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0001: [0x240000405:0x111f:0x0] migrate mdt count mismatch 2 != 3
LustreError: 8652:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) lustre-MDD0001: '11' migration was interrupted, run 'lfs migrate -m 0 -c 2 -H crush 11' to finish migration: rc = -1
LustreError: 8652:0:(mdd_dir.c:4457:mdd_migrate_cmd_check()) Skipped 32 previous similar messages
LustreError: 15890:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff880275b20008: cannot apply new layout on [0x280000406:0x8bd:0x0] : rc = -5
LustreError: 15890:0:(vvp_io.c:1921:vvp_io_init()) lustre: refresh file layout [0x280000406:0x8bd:0x0] error -5.
LustreError: 488:0:(lov_object.c:1340:lov_layout_change()) lustre-clilov-ffff880275b20008: cannot apply new layout on [0x280000406:0x8bd:0x0] : rc = -5
18[7703]: segfault at 8 ip 00007fa66eb6b7e8 sp 00007ffd32146d30 error 4 in ld-2.17.so[7fa66eb60000+22000]
LustreError: 14182:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 8 [0x200000404:0x2cb7:0x0]: rc = -5
LustreError: 14182:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 5 previous similar messages
LustreError: 11394:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 9 [0x200000404:0x2d27:0x0]: rc = -5
6[12187]: segfault at 8 ip 00007f498d3927e8 sp 00007ffc5af2e2b0 error 4 in ld-2.17.so[7f498d387000+22000]
14[21753]: segfault at 0 ip (null) sp 00007ffcee4f0748 error 14 in 14[400000+6000]
14[22535]: segfault at 0 ip (null) sp 00007ffe5e90d9b8 error 14 in 14[400000+6000]
LustreError: 11394:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 16 [0x240000405:0x127b:0x0]: rc = -5
LustreError: 11394:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 1 previous similar message
LustreError: 18620:0:(mdt_reint.c:2533:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x32cf:0x0]/6 failed: rc = -16
LustreError: 18620:0:(mdt_reint.c:2533:mdt_reint_migrate()) Skipped 392 previous similar messages
LustreError: 21572:0:(file.c:262:ll_close_inode_openhandle()) lustre-clilmv-ffff880275b20008: inode [0x280000406:0xc84:0x0] mdc close failed: rc = -116
LustreError: 21572:0:(file.c:262:ll_close_inode_openhandle()) Skipped 29 previous similar messages
LustreError: 28829:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 18 [0x200000403:0x3359:0x0]: rc = -5
LustreError: 8862:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 18 [0x200000403:0x3359:0x0]: rc = -5
LustreError: 18583:0:(llog_cat.c:591:llog_cat_add_rec()) llog_write_rec -116: lh=ffff88008baf8358
LustreError: 18583:0:(update_trans.c:1040:top_trans_stop()) lustre-MDT0002-osp-MDT0000: write updates failed: rc = -116
Lustre: 31433:0:(lod_lov.c:1418:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x35b7:0x0] with magic=0xbd60bd0
Lustre: 31433:0:(lod_lov.c:1418:lod_parse_striping()) Skipped 65 previous similar messages
LustreError: 18588:0:(llog_osd.c:628:llog_osd_write_rec()) lustre-MDT0002-osp-MDT0000: index 198 already set in llog bitmap [0x280000402:0x3:0x0]

LustreError: 16966:0:(llog_osd.c:630:llog_osd_write_rec()) LBUG
CPU: 4 PID: 16966 Comm: mdt_rdpg02_002 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
Call Trace:
[<ffffffff817d93f8>] dump_stack+0x19/0x1b
[<ffffffffa017aa9d>] lbug_with_loc+0x4d/0xa0 [libcfs]
[<ffffffff817df60e>] ? mutex_unlock+0xe/0x10
[<ffffffffa03950ea>] llog_osd_write_rec+0x1c1a/0x1ca0 [obdclass]
[<ffffffffa0c9b6a9>] ? osd_trunc_unlock_all+0x109/0x150 [osd_ldiskfs]
[<ffffffffa037e7f0>] llog_write_rec+0x290/0x590 [obdclass]
[<ffffffffa0384ec9>] llog_cat_add_rec+0x1d9/0xa50 [obdclass]
[<ffffffffa037b6af>] llog_add+0x17f/0x1f0 [obdclass]
[<ffffffffa071f9d3>] sub_updates_write+0x31a/0xeb4 [ptlrpc]
[<ffffffffa071f6a8>] ? prepare_writing_updates.isra.10+0x144/0x155 [ptlrpc]
[<ffffffffa06ff2f9>] top_trans_stop+0x439/0xf40 [ptlrpc]
[<ffffffffa0eec882>] lod_trans_stop+0x72/0x360 [lod]
[<ffffffffa0db2d58>] mdd_trans_stop+0x28/0x16e [mdd]
[<ffffffffa0da34d7>] mdd_attr_set+0x6b7/0xe70 [mdd]
[<ffffffffa0f0967a>] ? lod_read_unlock+0x2a/0xc0 [lod]
[<ffffffffa0e34a5e>] mdt_mfd_close+0x76e/0xbb0 [mdt]
[<ffffffffa0e3ab84>] mdt_close_internal+0xb4/0x240 [mdt]
[<ffffffffa0e3af9c>] mdt_close+0x28c/0x980 [mdt]
[<ffffffffa06eb24e>] tgt_request_handle+0x74e/0x1a50 [ptlrpc]
[<ffffffffa062e9ac>] ptlrpc_server_handle_request+0x26c/0xcb0 [ptlrpc]
[<ffffffffa063077e>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<ffffffff814119f9>] ? do_raw_spin_unlock+0x49/0x90
[<ffffffffa062fb00>] ? ptlrpc_wait_event+0x630/0x630 [ptlrpc]
[<ffffffff810ba114>] kthread+0xe4/0xf0
[<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140
[<ffffffff817ede5d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff810ba030>] ? kthread_create_on_node+0x140/0x140

LustreError: 9331:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/6 failed: rc = -114
LustreError: 15834:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff880323d70008: inode [0x200000404:0x14:0x0] mdc close failed: rc = -116
LustreError: 9348:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x2:0x0]/18 failed: rc = -114
LustreError: 16479:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff880323d70008: inode [0x200000404:0xd:0x0] mdc close failed: rc = -116
LustreError: 9341:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/7 failed: rc = -114
LustreError: 9341:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 1 previous similar message
LustreError: 16917:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff880323d70008: inode [0x200000404:0x17:0x0] mdc close failed: rc = -116
LustreError: 9327:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/18 failed: rc = -114
LustreError: 9327:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 2 previous similar messages
LustreError: 16925:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff88029d852548: inode [0x280000403:0x6:0x0] mdc close failed: rc = -116
LustreError: 16925:0:(file.c:264:ll_close_inode_openhandle()) Skipped 1 previous similar message
Lustre: 9348:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 512 < left 894, rollback = 2
Lustre: 9348:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/1, destroy: 1/4/0
Lustre: 9348:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 11/894/0
Lustre: 9348:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 1/3/0
Lustre: 9348:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 15/263/2, delete: 3/6/0
Lustre: 9348:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 8/8/0, ref_del: 3/3/0
Lustre: 9314:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 512 < left 671, rollback = 2
Lustre: 9314:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 8 previous similar messages
Lustre: 9314:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/1, destroy: 0/0/0
Lustre: 9314:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 8 previous similar messages
Lustre: 9314:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0
Lustre: 9314:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 8 previous similar messages
Lustre: 9314:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 1/3/0
Lustre: 9314:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 8 previous similar messages
Lustre: 9314:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 10/203/2, delete: 0/0/0
Lustre: 9314:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 8 previous similar messages
Lustre: 9314:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
Lustre: 9314:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 8 previous similar messages
LustreError: 9341:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/10 failed: rc = -114
LustreError: 9341:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 1 previous similar message
Lustre: 9336:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 514 < left 582, rollback = 2
Lustre: 9336:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 7 previous similar messages
Lustre: 9336:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/0, destroy: 0/0/0
Lustre: 9336:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 9336:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 6/582/0
Lustre: 9336:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 9336:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 1/3/0
Lustre: 9336:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 9336:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 11/239/1, delete: 0/0/0
Lustre: 9336:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 9336:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0
Lustre: 9336:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 11113:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 516 < left 618, rollback = 7
LustreError: 19194:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff88029d852548: inode [0x240000403:0x9:0x0] mdc close failed: rc = -116
LustreError: 19194:0:(file.c:264:ll_close_inode_openhandle()) Skipped 1 previous similar message
Lustre: 16486:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff88029b634140 x1801629135627712/t4294969346(0) o101->58d05bf8-ef03-46fb-b03c-f1a0e18a478b@0@lo:503/0 lens 376/840 e 0 to 0 dl 1718167593 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
LustreError: 17790:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/19 failed: rc = -16
LustreError: 17790:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 7 previous similar messages
Lustre: 9330:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 507 < left 955, rollback = 2
Lustre: 9330:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 19 previous similar messages
Lustre: 9330:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/3, destroy: 2/8/0
Lustre: 9330:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 21 previous similar messages
Lustre: 9330:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 10/955/0
Lustre: 9330:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 21 previous similar messages
Lustre: 9330:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 1/3/0
Lustre: 9330:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 21 previous similar messages
Lustre: 9330:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 13/251/3, delete: 7/13/1
Lustre: 9330:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 21 previous similar messages
Lustre: 9330:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 7/7/1
Lustre: 9330:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 21 previous similar messages
Lustre: 11128:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 516 < left 618, rollback = 7
Lustre: 11128:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 1 previous similar message
Lustre: 16729:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff8800904d4140 x1801629135864576/t4294968718(0) o101->58d05bf8-ef03-46fb-b03c-f1a0e18a478b@0@lo:507/0 lens 376/864 e 0 to 0 dl 1718167597 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
Lustre: 11125:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 618, rollback = 7
Lustre: 11125:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 3 previous similar messages
Lustre: 9309:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 506 < left 894, rollback = 2
Lustre: 9309:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 50 previous similar messages
Lustre: 9309:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/4, destroy: 2/8/0
Lustre: 9309:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 56 previous similar messages
Lustre: 9309:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 11/894/0
Lustre: 9309:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 56 previous similar messages
Lustre: 9309:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 7/105/0
Lustre: 9309:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 56 previous similar messages
Lustre: 9309:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 17/335/3, delete: 10/16/1
Lustre: 9309:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 56 previous similar messages
Lustre: 9309:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 10/10/0, ref_del: 10/10/1
Lustre: 9309:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 56 previous similar messages
Lustre: 11116:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 618, rollback = 7
Lustre: 11116:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 1 previous similar message
LustreError: 17622:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0x8:0x0]: rc = -2
LustreError: 18618:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff880323d70008: inode [0x240000403:0x8:0x0] mdc close failed: rc = -2
LustreError: 18426:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: '12' migration was interrupted, run 'lfs migrate -m 0 -c 2 -H crush 12' to finish migration: rc = -1
dd (21995) used greatest stack depth: 9824 bytes left
Lustre: 15724:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 512 < left 745, rollback = 2
Lustre: 15724:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 62 previous similar messages
Lustre: 15724:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 1/4/2, destroy: 0/0/0
Lustre: 15724:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 66 previous similar messages
Lustre: 15724:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 8/745/0
Lustre: 15724:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 66 previous similar messages
Lustre: 15724:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 5/21/0, punch: 0/0/0, quota 1/3/0
Lustre: 15724:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 66 previous similar messages
Lustre: 15724:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 4/105/1, delete: 1/1/0
Lustre: 15724:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 66 previous similar messages
Lustre: 15724:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0
Lustre: 15724:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 66 previous similar messages
LustreError: 9329:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0xa4:0x0]/10 failed: rc = -2
LustreError: 9329:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 18 previous similar messages
Lustre: 11116:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 618, rollback = 7
Lustre: 11116:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 3 previous similar messages
Lustre: 18016:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff880283a10a40 x1801629137237824/t4294969700(0) o101->58d05bf8-ef03-46fb-b03c-f1a0e18a478b@0@lo:523/0 lens 376/816 e 0 to 0 dl 1718167613 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
LustreError: 9246:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0xa3:0x0]: rc = -2
Lustre: 15944:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x1ce:0x0] with magic=0xbd60bd0
Lustre: 11104:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 515 < left 618, rollback = 7
Lustre: 11104:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 5 previous similar messages
Lustre: 9292:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 506 < left 1457, rollback = 2
Lustre: 9292:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 126 previous similar messages
Lustre: 9292:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 1/4/4, destroy: 1/4/0
Lustre: 9292:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 134 previous similar messages
LustreError: 16247:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0xd2:0x0]: rc = -2
LustreError: 23963:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff88029d852548: inode [0x200000404:0xd2:0x0] mdc close failed: rc = -2
LustreError: 23963:0:(file.c:264:ll_close_inode_openhandle()) Skipped 3 previous similar messages
Lustre: 11105:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 11105:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 158 previous similar messages
Lustre: 11105:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 1/3/0
Lustre: 25381:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 12/275/3, delete: 0/0/0
Lustre: 25381:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 158 previous similar messages
Lustre: 25381:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 0/0/0
Lustre: 25381:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 158 previous similar messages
Lustre: 11105:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 178 previous similar messages
13[25529]: segfault at 8 ip 00007f9570f487e8 sp 00007ffc923e9260 error 4 in ld-2.17.so[7f9570f3d000+22000]
9[25286]: segfault at 8 ip 00007f3c991417e8 sp 00007ffeab743190 error 4 in ld-2.17.so[7f3c99136000+22000]
Lustre: dir [0x280000403:0x121:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
LustreError: 9323:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: '16' migration was interrupted, run 'lfs migrate -m 1 -c 2 -H crush 16' to finish migration: rc = -1
LustreError: 17552:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000404:0x147:0x0]: rc = -2
LustreError: 17552:0:(mdd_object.c:3873:mdd_close()) Skipped 1 previous similar message
Lustre: dir [0x240000404:0x46:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
LustreError: 17622:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0x46:0x0]: rc = -2
Lustre: dir [0x200000403:0x234:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
LustreError: 18903:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0000: [0x200000403:0x1c7:0x0] migrate mdt count mismatch 2 != 1
LustreError: 24587:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/1 failed: rc = -16
LustreError: 24587:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 39 previous similar messages
Lustre: 17208:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 618, rollback = 7
Lustre: 17208:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 11 previous similar messages
16[30165]: segfault at 8 ip 00007fb592a227e8 sp 00007ffcad4e4120 error 4 in ld-2.17.so[7fb592a17000+22000]
LustreError: 32632:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x2a6:0x0]: rc = -5
LustreError: 32632:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
19[32043]: segfault at 8 ip 00007ff446c4e7e8 sp 00007fff900b5840 error 4 in ld-2.17.so[7ff446c43000+22000]
LustreError: 16146:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0x2a2:0x0]: rc = -2
Lustre: 25853:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 500 < left 671, rollback = 2
Lustre: 25853:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 398 previous similar messages
Lustre: 25853:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/11, destroy: 0/0/0
Lustre: 25853:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 410 previous similar messages
LustreError: 24587:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: '11' migration was interrupted, run 'lfs migrate -m 2 -c 2 -H crush 11' to finish migration: rc = -1
LustreError: 32350:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x2a6:0x0]: rc = -5
LustreError: 32350:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 32350:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 32350:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 1 previous similar message
LustreError: 32751:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff88029d852548: inode [0x280000404:0x165:0x0] mdc close failed: rc = -116
LustreError: 32751:0:(file.c:264:ll_close_inode_openhandle()) Skipped 9 previous similar messages
Lustre: 20126:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 20126:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 402 previous similar messages
Lustre: 20126:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 1/3/0
Lustre: 20126:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 382 previous similar messages
Lustre: 20126:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 20126:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 402 previous similar messages
Lustre: 20126:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 20126:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 402 previous similar messages
Lustre: 9215:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x162:0x0] with magic=0xbd60bd0
Lustre: 9215:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 1 previous similar message
Lustre: 19272:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0xc1:0x0] with magic=0xbd60bd0
Lustre: 19272:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 5 previous similar messages
9[564]: segfault at 8 ip 00007f4c88cc17e8 sp 00007ffdc0fd5880 error 4 in ld-2.17.so[7f4c88cb6000+22000]
LustreError: 24587:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0000: '7' migration was interrupted, run 'lfs migrate -m 2 -c 3 -H crush 7' to finish migration: rc = -1
LustreError: 1347:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x20c:0x0]: rc = -5
LustreError: 1347:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 1347:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 1347:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 1 previous similar message
Lustre: 15495:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x2c7:0x0] with magic=0xbd60bd0
Lustre: 15495:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 1 previous similar message
Lustre: dir [0x280000403:0x25e:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
16[5219]: segfault at 8 ip 00007fad0b6157e8 sp 00007ffe38cf5670 error 4 in ld-2.17.so[7fad0b60a000+22000]
LustreError: 3700:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000404:0x26e:0x0]: rc = -5
LustreError: 3700:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 2 previous similar messages
LustreError: 3700:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 3700:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 2 previous similar messages
Lustre: 23092:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x23c:0x0] with magic=0xbd60bd0
Lustre: 23092:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 9 previous similar messages
LustreError: 22241:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0000: '2' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 2' to finish migration: rc = -1
LustreError: 22241:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 1 previous similar message
Lustre: 11113:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 618, rollback = 7
Lustre: 11113:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 7 previous similar messages
LustreError: 9254:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0x202:0x0]: rc = -2
LustreError: 22241:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: '11' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 11' to finish migration: rc = -1
LustreError: 22241:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 1 previous similar message
LustreError: 9129:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000403:0x4d6:0x0]: rc = -5
LustreError: 9129:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 9129:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 9129:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 1 previous similar message
Lustre: 20520:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x530:0x0] with magic=0xbd60bd0
Lustre: 20520:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 3 previous similar messages
2[12770]: segfault at 8 ip 00007f23a969e7e8 sp 00007ffd031c2c00 error 4 in ld-2.17.so[7f23a9693000+22000]
LustreError: 13160:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x475:0x0]: rc = -5
LustreError: 13160:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 2 previous similar messages
LustreError: 13160:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 13160:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 2 previous similar messages
LustreError: 9336:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0000: '17' migration was interrupted, run 'lfs migrate -m 2 -c 3 -H crush 17' to finish migration: rc = -1
LustreError: 9336:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 5 previous similar messages
LustreError: 9336:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0000: migrate [0x280000403:0x1:0x0]/17 failed: rc = -1
LustreError: 9336:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 95 previous similar messages
Lustre: 18366:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 18366:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 1050 previous similar messages
LustreError: 21553:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000403:0x645:0x0]: rc = -5
LustreError: 21553:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 14 previous similar messages
LustreError: 21553:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 21553:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 14 previous similar messages
LustreError: 21553:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 10 [0x240000403:0x645:0x0]: rc = -5
Lustre: 9221:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x4d8:0x0] with magic=0xbd60bd0
Lustre: 9221:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 21 previous similar messages
18[25464]: segfault at 57b3bc3b ip 00000000004047c5 sp 00007fff61ebbdf8 error 6 in 18[400000+6000]
12[25701]: segfault at 0 ip (null) sp 00007ffcf2402bd8 error 14 in 12[400000+6000]
LustreError: 18367:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff880323d70008: inode [0x200000404:0x6d7:0x0] mdc close failed: rc = -13
LustreError: 18367:0:(file.c:264:ll_close_inode_openhandle()) Skipped 6 previous similar messages
4[3521]: segfault at 8 ip 00007f4e729367e8 sp 00007fff55826870 error 4 in ld-2.17.so[7f4e7292b000+22000]
16[5975]: segfault at 8 ip 00007f6e45c517e8 sp 00007ffdc7879a30 error 4 in ld-2.17.so[7f6e45c46000+22000]
Lustre: 28308:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 516 < left 618, rollback = 7
Lustre: 28308:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 25 previous similar messages
Lustre: 28308:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 28308:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 1036 previous similar messages
Lustre: 28308:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 4/150/0
Lustre: 28308:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 1036 previous similar messages
Lustre: 28308:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 28308:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 1036 previous similar messages
Lustre: 28308:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 28308:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 1036 previous similar messages
1[13951]: segfault at 8 ip 00007f5ecacbb7e8 sp 00007ffd44ba07f0 error 4 in ld-2.17.so[7f5ecacb0000+22000]
0[16072]: segfault at 8 ip 00007f6e91d587e8 sp 00007ffd1bb74cc0 error 4 in ld-2.17.so[7f6e91d4d000+22000]
10[19522]: segfault at 406000 ip 0000000000406000 sp 00007ffd8a966dc8 error 14 in 10[606000+1000]
10[24850]: segfault at 0 ip 0000000000403cf0 sp 00007ffcb2d75d18 error 6 in 10[400000+6000]
traps: 7[25131] trap invalid opcode ip:4045b2 sp:7fff31461058 error:0 in 7[400000+6000]
19[28676]: segfault at 0 ip (null) sp 00007ffd24ffd4c8 error 14 in 19[400000+6000]
Lustre: 29839:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 509 < left 894, rollback = 2
Lustre: 29839:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 1020 previous similar messages
13[31096]: segfault at 8 ip 00007f0302fb07e8 sp 00007ffeccf10150 error 4 in ld-2.17.so[7f0302fa5000+22000]
9[31227]: segfault at 8 ip 00007f00f76237e8 sp 00007ffef89b1760 error 4 in ld-2.17.so[7f00f7618000+22000]
LustreError: 15714:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0000: '14' migration was interrupted, run 'lfs migrate -m 1 -c 2 -H crush 14' to finish migration: rc = -1
LustreError: 15714:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 1 previous similar message
LustreError: 18386:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x2:0x0]/17 failed: rc = -16
LustreError: 18386:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 40 previous similar messages
17[358]: segfault at 8 ip 00007fc3ed8c77e8 sp 00007ffff0580f40 error 4 in ld-2.17.so[7fc3ed8bc000+22000]
LustreError: 32677:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000404:0xb17:0x0]: rc = -5
LustreError: 32677:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 26 previous similar messages
LustreError: 32677:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 32677:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 26 previous similar messages
Lustre: 28308:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 28308:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 177 previous similar messages
9[4036]: segfault at 406000 ip 0000000000406000 sp 00007ffc90e00178 error 14 in 9[606000+1000]
12[4420]: segfault at 8 ip 00007f611e6147e8 sp 00007ffd3cf61050 error 4 in ld-2.17.so[7f611e609000+22000]
17[4605]: segfault at 8 ip 00007f0ab8b7f7e8 sp 00007ffc77c8cd30 error 4 in ld-2.17.so[7f0ab8b74000+22000]
LustreError: 6314:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff88029d852548: inode [0x240000404:0xad5:0x0] mdc close failed: rc = -116
LustreError: 6314:0:(file.c:264:ll_close_inode_openhandle()) Skipped 20 previous similar messages
Lustre: 18058:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0xb05:0x0] with magic=0xbd60bd0
Lustre: 18058:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 19 previous similar messages
LustreError: 16432:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0xac8:0x0]: rc = -2
LustreError: 16432:0:(mdd_object.c:3873:mdd_close()) Skipped 1 previous similar message
Lustre: 11132:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 510 < left 613, rollback = 7
Lustre: 11132:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 76 previous similar messages
Lustre: 11132:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 11132:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 513 previous similar messages
Lustre: 11132:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 2/618/5, punch: 0/0/0, quota 4/150/0
Lustre: 11132:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 513 previous similar messages
Lustre: 11132:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 11132:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 513 previous similar messages
Lustre: 11132:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 11132:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 513 previous similar messages
13[11705]: segfault at 8 ip 00007f26a7b577e8 sp 00007ffce09a8f00 error 4 in ld-2.17.so[7f26a7b4c000+22000]
LustreError: 25381:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: '8' migration was interrupted, run 'lfs migrate -m 0 -c 3 -H crush 8' to finish migration: rc = -1
LustreError: 25381:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 3 previous similar messages
19[15867]: segfault at 8 ip 00007f48f30017e8 sp 00007fff6f8a2370 error 4 in ld-2.17.so[7f48f2ff6000+22000]
Lustre: dir [0x200000403:0x11de:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
4[16761]: segfault at 8 ip 00007f229c8c77e8 sp 00007ffdd22c2880 error 4 in ld-2.17.so[7f229c8bc000+22000]
11[18139]: segfault at 8 ip 00007f4933d4f7e8 sp 00007ffc229a7830 error 4 in ld-2.17.so[7f4933d44000+22000]
LustreError: 19491:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000403:0xe12:0x0]: rc = -5
LustreError: 19491:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 23 previous similar messages
LustreError: 19491:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 19491:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 23 previous similar messages
Lustre: dir [0x280000404:0xd23:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
3[23476]: segfault at 8 ip 00007ff4d609e7e8 sp 00007fff16cafa20 error 4 in ld-2.17.so[7ff4d6093000+22000]
4[24407]: segfault at 8 ip 00007efe99d1e7e8 sp 00007fffe79e7f90 error 4 in ld-2.17.so[7efe99d13000+22000]
7[27184]: segfault at 8 ip 00007f83420a87e8 sp 00007ffc6a14b210 error 4 in ld-2.17.so[7f834209d000+22000]
5[27128]: segfault at 0 ip (null) sp 00007ffda0903d48 error 14 in 5[400000+6000]
LustreError: 28097:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000404:0xd62:0x0]: rc = -2
LustreError: 28097:0:(mdd_object.c:3873:mdd_close()) Skipped 1 previous similar message
LustreError: 59:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 11 [0x240000403:0xe22:0x0]: rc = -5
LustreError: 59:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 1 previous similar message
9[30321]: segfault at 8 ip 00007ff010b657e8 sp 00007fffca26b370 error 4 in ld-2.17.so[7ff010b5a000+22000]
Lustre: dir [0x240000403:0xdd4:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
LustreError: 15859:0:(lustre_lmv.h:517:lmv_is_sane()) unknown layout LMV: magic=0xcd40cd0 count=2 index=1 hash=crush:0x82000003 version=1 migrate_offset=1 migrate_hash=fnv_1a_64:2 pool=
Lustre: 9329:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 582, rollback = 2
Lustre: 9329:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 1386 previous similar messages
5[377]: segfault at 8 ip 00007ff3cfb997e8 sp 00007ffdfc987ee0 error 4 in ld-2.17.so[7ff3cfb8e000+22000]
LustreError: 127:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 13 [0x280000404:0xf13:0x0]: rc = -5
LustreError: 127:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 1 previous similar message
18[4683]: segfault at 8 ip 00007fc373e407e8 sp 00007ffd41840be0 error 4 in ld-2.17.so[7fc373e35000+22000]
18[4690]: segfault at 8 ip 00007f58831457e8 sp 00007ffda1e387c0 error 4 in ld-2.17.so[7f588313a000+22000]
Lustre: mdt_io05_005: service thread pid 15714 was inactive for 40.016 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Pid: 15714, comm: mdt_io05_005 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xd00 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x870 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt]
[<0>] mdt_object_lock+0x88/0x1c0 [mdt]
[<0>] mdt_rename_source_lock+0xa9/0xd6 [mdt]
[<0>] mdt_reint_migrate+0x1891/0x2420 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x74c/0xbc0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x74e/0x1a50 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x26c/0xcb0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 7498, comm: mdt_io05_007 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Lustre: mdt_io01_002: service thread pid 9314 was inactive for 40.099 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xd00 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x870 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt]
[<0>] mdt_rename_lock+0xc3/0x2d0 [mdt]
[<0>] mdt_reint_migrate+0x891/0x2420 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x74c/0xbc0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x74e/0x1a50 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x26c/0xcb0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 9331, comm: mdt_io04_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xd00 [ptlrpc]
[<0>] ldlm_cli_enqueue_fini+0xa24/0xee0 [ptlrpc]
[<0>] ldlm_cli_enqueue+0x461/0xa70 [ptlrpc]
[<0>] osp_md_object_lock+0x151/0x2f0 [osp]
[<0>] lod_object_lock+0xdb/0x7d0 [lod]
[<0>] mdd_object_lock+0x2d/0xd0 [mdd]
[<0>] mdt_remote_object_lock_try+0x14c/0x189 [mdt]
[<0>] mdt_object_lock_internal+0x377/0x420 [mdt]
[<0>] mdt_rename_lock+0xc3/0x2d0 [mdt]
[<0>] mdt_reint_migrate+0x891/0x2420 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x74c/0xbc0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x74e/0x1a50 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x26c/0xcb0 [ptlrpc]
[<0>] ptlrpc_main+0xc7e/0x1690 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: mdt_io07_005: service thread pid 1814 was inactive for 40.094 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 3 previous similar messages
Lustre: mdt_io05_006: service thread pid 15859 was inactive for 40.096 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 5 previous similar messages
Lustre: mdt07_000: service thread pid 9226 was inactive for 40.089 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 1 previous similar message
LustreError: 9119:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 0@lo ns: mdt-lustre-MDT0001_UUID lock: ffff880278965680/0x5763a546cdd38668 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1163:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x5763a546cdd3843f expref: 833 pid: 9228 timeout: 731 lvb_type: 0
Lustre: mdt_io05_005: service thread pid 15714 completed after 102.251s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt07_010: service thread pid 18362 completed after 97.254s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt07_002: service thread pid 9228 completed after 96.997s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt06_000: service thread pid 9221 completed after 97.210s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt06_012: service thread pid 13033 completed after 97.216s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt01_004: service thread pid 16354 completed after 99.004s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io01_002: service thread pid 9314 completed after 102.102s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 0-0: lustre-MDT0001-mdc-ffff880323d70008: operation mds_close to node 0@lo failed: rc = -107
Lustre: lustre-MDT0001-mdc-ffff880323d70008: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: mdt07_000: service thread pid 9226 completed after 97.496s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 0-0: lustre-MDT0001-mdc-ffff880323d70008: This client was evicted by lustre-MDT0001; in progress operations using this service will fail.
Lustre: mdt07_011: service thread pid 18394 completed after 97.296s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io00_003: service thread pid 18743 completed after 102.118s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt04_009: service thread pid 10306 completed after 97.316s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 5617:0:(llite_lib.c:2028:ll_md_setattr()) md_setattr fails: rc = -108
LustreError: 6384:0:(mdc_request.c:1469:mdc_read_page()) lustre-MDT0001-mdc-ffff880323d70008: [0x240000403:0x1:0x0] lock enqueue fails: rc = -108
Lustre: lustre-MDT0001-mdc-ffff880323d70008: Connection restored to (at 0@lo)
Lustre: mdt_io04_002: service thread pid 9331 completed after 102.351s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io05_007: service thread pid 7498 completed after 102.367s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io03_005: service thread pid 26333 completed after 102.305s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_005: service thread pid 1814 completed after 101.925s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io01_001: service thread pid 9313 completed after 101.952s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io05_004: service thread pid 18426 completed after 101.752s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io05_001: service thread pid 9335 completed after 101.739s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io04_000: service thread pid 9329 completed after 101.742s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io00_000: service thread pid 9309 completed after 101.596s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io05_000: service thread pid 9332 completed after 101.565s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io05_006: service thread pid 15859 completed after 101.116s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io00_001: service thread pid 9310 completed after 98.018s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io04_001: service thread pid 9330 completed after 97.975s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_004: service thread pid 21704 completed after 97.526s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io02_006: service thread pid 28943 completed after 97.097s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io07_001: service thread pid 9345 completed after 96.531s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: dir [0x280000404:0xfde:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 2 previous similar messages
LustreError: 24616:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000403:0x116b:0x0]: rc = -5
LustreError: 24616:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 35 previous similar messages
LustreError: 24616:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 24616:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 35 previous similar messages
LustreError: 24616:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for sleep [0x240000403:0x116b:0x0]: rc = -5
LustreError: 24616:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 2 previous similar messages
Lustre: 9195:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000403:0x105d:0x0] with magic=0xbd60bd0
Lustre: 9195:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 19 previous similar messages
12[14312]: segfault at 406000 ip 0000000000406000 sp 00007fff614b0a98 error 14 in 12[606000+1000]
14[14989]: segfault at 8 ip 00007f125c6ea7e8 sp 00007ffdd5aec0c0 error 4 in ld-2.17.so[7f125c6df000+22000]
LustreError: 25853:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0001: '6' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 6' to finish migration: rc = -1
LustreError: 25853:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 7 previous similar messages
LustreError: 25853:0:(mdt_reint.c:2519:mdt_reint_migrate()) lustre-MDT0001: migrate [0x240000403:0x1:0x0]/6 failed: rc = -1
LustreError: 25853:0:(mdt_reint.c:2519:mdt_reint_migrate()) Skipped 276 previous similar messages
1[21724]: segfault at 8 ip 00007f4ebedba7e8 sp 00007ffd4d6baa40 error 4 in ld-2.17.so[7f4ebedaf000+22000]
18[22412]: segfault at 8 ip 00007ff6aaa207e8 sp 00007ffd35ffe430 error 4 in ld-2.17.so[7ff6aaa15000+22000]
Lustre: 28943:0:(osd_handler.c:1974:osd_trans_dump_creds()) create: 3/12/8, destroy: 1/4/0
Lustre: 28943:0:(osd_handler.c:1974:osd_trans_dump_creds()) Skipped 2396 previous similar messages
traps: 19[24670] trap invalid opcode ip:404e98 sp:7ffe8f34a3b8 error:0 in 19[400000+6000]
Lustre: mdt_io06_004: service thread pid 20219 was inactive for 40.045 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: Skipped 11 previous similar messages
LustreError: 9119:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 0@lo ns: mdt-lustre-MDT0001_UUID lock: ffff880079767c00/0x5763a546cde43708 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x145a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x5763a546cde436ec expref: 878 pid: 26936 timeout: 878 lvb_type: 0
Lustre: mdt00_005: service thread pid 16631 completed after 100.223s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 9205:0:(ldlm_lockd.c:1499:ldlm_handle_enqueue()) ### lock on destroyed export ffff88029bcaca88 ns: mdt-lustre-MDT0001_UUID lock: ffff8802707f1300/0x5763a546cde47694 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 18 type: IBT gid 0 flags: 0x50200000000000 nid: 0@lo remote: 0x5763a546cde4744f expref: 21 pid: 9205 timeout: 0 lvb_type: 0
Lustre: mdt02_010: service thread pid 3273 completed after 100.229s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt03_013: service thread pid 30929 completed after 99.090s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 0-0: lustre-MDT0001-mdc-ffff88029d852548: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-MDT0001-mdc-ffff88029d852548: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: mdt00_009: service thread pid 25284 completed after 99.174s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt00_008: service thread pid 19271 completed after 99.088s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 0-0: lustre-MDT0001-mdc-ffff88029d852548: This client was evicted by lustre-MDT0001; in progress operations using this service will fail.
Lustre: mdt03_010: service thread pid 26402 completed after 99.270s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt02_002: service thread pid 9205 completed after 99.104s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt02_009: service thread pid 14677 completed after 100.392s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 30465:0:(file.c:5683:ll_inode_revalidate_fini()) lustre: revalidate FID [0x240000403:0x1:0x0] error: rc = -5
Lustre: 17398:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 511 < left 1072, rollback = 2
LustreError: 28640:0:(file.c:264:ll_close_inode_openhandle()) lustre-clilmv-ffff88029d852548: inode [0x240000403:0x145a:0x0] mdc close failed: rc = -5
LustreError: 28640:0:(file.c:264:ll_close_inode_openhandle()) Skipped 82 previous similar messages
Lustre: 17398:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 1578 previous similar messages
Lustre: 17398:0:(osd_handler.c:1981:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 13/1072/0
Lustre: 17398:0:(osd_handler.c:1981:osd_trans_dump_creds()) Skipped 2637 previous similar messages
Lustre: 17398:0:(osd_handler.c:1991:osd_trans_dump_creds()) write: 1/10/0, punch: 0/0/0, quota 4/54/0
Lustre: 17398:0:(osd_handler.c:1991:osd_trans_dump_creds()) Skipped 2637 previous similar messages
Lustre: 17398:0:(osd_handler.c:1998:osd_trans_dump_creds()) insert: 12/194/2, delete: 3/6/0
Lustre: 17398:0:(osd_handler.c:1998:osd_trans_dump_creds()) Skipped 2637 previous similar messages
Lustre: 17398:0:(osd_handler.c:2005:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 3/3/0
Lustre: 17398:0:(osd_handler.c:2005:osd_trans_dump_creds()) Skipped 2637 previous similar messages
LustreError: 30665:0:(llite_lib.c:2028:ll_md_setattr()) md_setattr fails: rc = -108
LustreError: 30665:0:(llite_lib.c:2028:ll_md_setattr()) Skipped 3 previous similar messages
LustreError: 29472:0:(mdc_request.c:1469:mdc_read_page()) lustre-MDT0001-mdc-ffff88029d852548: [0x240000402:0x27:0x0] lock enqueue fails: rc = -108
LustreError: 29472:0:(mdc_request.c:1469:mdc_read_page()) Skipped 4 previous similar messages
Lustre: dir [0x280000403:0x1249:0x0] stripe 2 readdir failed: -108, directory is partially accessed!
Lustre: Skipped 3 previous similar messages
Lustre: lustre-MDT0001-mdc-ffff88029d852548: Connection restored to (at 0@lo)
Lustre: mdt_io06_004: service thread pid 20219 completed after 103.443s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: mdt_io06_001: service thread pid 9338 completed after 102.828s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 21705:0:(out_lib.c:1188:out_tx_index_delete_undo()) lustre-MDT0000-osd: Oops, can not rollback index_delete yet: rc = -524
Lustre: 11105:0:(osd_internal.h:1333:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 516 < left 552, rollback = 7
Lustre: 11105:0:(osd_internal.h:1333:osd_trans_exec_op()) Skipped 108 previous similar messages
LustreError: 16631:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 13039:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 13039:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 13 previous similar messages
LustreError: 21124:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 21124:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 6 previous similar messages
LustreError: 9255:0:(mdd_object.c:3873:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x1ba0:0x0]: rc = -2
LustreError: 9255:0:(mdd_object.c:3873:mdd_close()) Skipped 5 previous similar messages
LustreError: 9214:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 9214:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 2 previous similar messages
LustreError: 17017:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 17017:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 2 previous similar messages
LustreError: 26056:0:(out_handler.c:886:out_tx_end()) lustre-MDT0001-osd: undo for /home/green/git/lustre-release/lustre/ptlrpc/../../lustre/target/out_handler.c:418: rc = -524
5[24387]: segfault at 406000 ip 0000000000406000 sp 00007ffc9b197788 error 14 in 5[606000+1000]
LustreError: 17581:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 17581:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 3 previous similar messages
Lustre: 9206:0:(lod_lov.c:1438:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000406:0x1b2:0x0] with magic=0xbd60bd0
Lustre: 9206:0:(lod_lov.c:1438:lod_parse_striping()) Skipped 47 previous similar messages
LustreError: 16231:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 16231:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 65 previous similar messages
LustreError: 880:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 0 [0x280000404:0x1317:0x0]: rc = -5
LustreError: 880:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 1 previous similar message
3[32459]: segfault at 406000 ip 0000000000406000 sp 00007ffee6174e58 error 14 in 3[606000+1000]
LustreError: 24616:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 16 [0x200000404:0x1cf5:0x0]: rc = -5
LustreError: 24616:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 3 previous similar messages
2[3358]: segfault at 8 ip 00007fe04ea077e8 sp 00007ffd9828a0d0 error 4 in ld-2.17.so[7fe04e9fc000+22000]
LustreError: 3909:0:(llite_nfs.c:446:ll_dir_get_parent_fid()) lustre: failure inode [0x200000404:0x195c:0x0] get parent: rc = -2
LustreError: 16655:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 16655:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 87 previous similar messages
16[9012]: segfault at 8 ip 00007f76b6fe67e8 sp 00007fffbff95120 error 4 in ld-2.17.so[7f76b6fdb000+22000]
LustreError: 150:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 8 [0x240000406:0x391:0x0]: rc = -5
LustreError: 150:0:(statahead.c:830:ll_statahead_interpret_work()) Skipped 7 previous similar messages
LustreError: 13488:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000404:0x16aa:0x0]: rc = -5
LustreError: 13488:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 117 previous similar messages
16[13730]: segfault at 8 ip 00007f3b660857e8 sp 00007fff8d7190a0 error 4 in ld-2.17.so[7f3b6607a000+22000]
LustreError: 13488:0:(llite_lib.c:3708:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 13488:0:(llite_lib.c:3708:ll_prep_inode()) Skipped 117 previous similar messages
LustreError: 29839:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: '9' migration was interrupted, run 'lfs migrate -m 0 -c 2 -H crush 9' to finish migration: rc = -1
LustreError: 29839:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) Skipped 19 previous similar messages
LustreError: 11182:0:(out_handler.c:886:out_tx_end()) lustre-MDT0001-osd: undo for /home/green/git/lustre-release/lustre/ptlrpc/../../lustre/target/out_handler.c:441: rc = -524
LustreError: 11182:0:(out_handler.c:886:out_tx_end()) Skipped 3 previous similar messages
2[21044]: segfault at 8 ip 00007f18fddf87e8 sp 00007ffc72bf4030 error 4 in ld-2.17.so[7f18fdded000+22000]
LustreError: 17594:0:(mdt_open.c:1280:mdt_cross_open()) lustre-MDT0002: [0x280000404:0x122a:0x0] doesn't exist!: rc = -14
LustreError: 17594:0:(mdt_open.c:1280:mdt_cross_open()) Skipped 161 previous similar messages
LustreError: 1690:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0000: [0x200000403:0x24d7:0x0] migrate mdt count mismatch 1 != 2
Lustre: dir [0x280000403:0x284b:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 65 previous similar messages
5[27279]: segfault at 0 ip (null) sp 00007ffd4437c3f8 error 14 in 5[400000+6000]
17[29167]: segfault at 8 ip 00007f45acd1f7e8 sp 00007ffcea006770 error 4 in ld-2.17.so[7f45acd14000+22000]
18[509]: segfault at 0 ip 0000000000403cf0 sp 00007ffff891e218 error 6 in 18[400000+6000]
LustreError: 16966:0:(llog_osd.c:628:llog_osd_write_rec()) lustre-MDT0002-osp-MDT0001: index 648 already set in llog bitmap [0x280000401:0x4:0x0]

LustreError: 9112:0:(llog_osd.c:626:llog_osd_write_rec()) LBUG
Pid: 9112, comm: mdt_rdpg03_000 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] libcfs_call_trace+0x90/0xf0 [libcfs]
[<0>] lbug_with_loc+0x4c/0xa0 [libcfs]
[<0>] llog_osd_write_rec+0x1e5d/0x1e90 [obdclass]
[<0>] llog_write_rec+0x290/0x590 [obdclass]
[<0>] llog_cat_add_rec+0x1d1/0xa40 [obdclass]
[<0>] llog_add+0x17f/0x1f0 [obdclass]
[<0>] sub_updates_write+0x317/0xe8b [ptlrpc]
[<0>] top_trans_stop+0x429/0xf30 [ptlrpc]
[<0>] lod_trans_stop+0x6a/0x350 [lod]
[<0>] mdd_trans_stop+0x28/0x16e [mdd]
[<0>] mdd_attr_set+0x697/0xdd0 [mdd]
[<0>] mdt_mfd_close+0x76e/0xbb0 [mdt]
[<0>] mdt_close_internal+0xb4/0x240 [mdt]
[<0>] mdt_close+0x28c/0x970 [mdt]
[<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc]
[<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe

LustreError: 9087:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/13 failed: rc = -114
LustreError: 15620:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x280000403:0x3:0x0] mdc close failed: rc = -116
Lustre: 9085:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 514 < left 671, rollback = 2
Lustre: 9085:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/0, destroy: 0/0/0
Lustre: 9085:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0
Lustre: 9085:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0
Lustre: 9085:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 10/203/1, delete: 0/0/0
Lustre: 9085:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
LustreError: 9072:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/5 failed: rc = -16
LustreError: 9072:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 1 previous similar message
LustreError: 16128:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x280000404:0x2:0x0] mdc close failed: rc = -13
Lustre: 16498:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 503 < left 610, rollback = 2
Lustre: 16498:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 7 previous similar messages
Lustre: 16498:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 6/24/8, destroy: 0/0/0
Lustre: 16498:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 16498:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 8/610/0
Lustre: 16498:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 16498:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 11/192/0, punch: 0/0/0, quota 1/3/0
Lustre: 16498:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 16498:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 19/350/4, delete: 1/1/0
Lustre: 16498:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 16498:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 1/1/0
Lustre: 16498:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 7 previous similar messages
LustreError: 16386:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/5 failed: rc = -2
Lustre: 9081:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 506 < left 582, rollback = 2
Lustre: 9081:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 11 previous similar messages
Lustre: 9081:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/7, destroy: 0/0/0
Lustre: 9081:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 11 previous similar messages
Lustre: 9081:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 6/582/0
Lustre: 9081:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 11 previous similar messages
Lustre: 9081:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0
Lustre: 9081:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 11 previous similar messages
Lustre: 9081:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/2, delete: 0/0/0
Lustre: 9081:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 11 previous similar messages
Lustre: 9081:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0
Lustre: 9081:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 11 previous similar messages
Lustre: 16006:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 512 < left 788, rollback = 2
Lustre: 16006:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 5 previous similar messages
Lustre: 16006:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/1, destroy: 0/0/0
Lustre: 16006:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 5 previous similar messages
Lustre: 16006:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 10/788/0
Lustre: 16006:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 5 previous similar messages
Lustre: 16006:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0
Lustre: 16006:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 5 previous similar messages
Lustre: 16006:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 12/215/2, delete: 1/1/0
Lustre: 16006:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 5 previous similar messages
Lustre: 16006:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 1/1/0
Lustre: 16006:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 5 previous similar messages
LustreError: 9090:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/16 failed: rc = -114
LustreError: 9083:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '15' migration was interrupted, run 'lfs migrate -m 0 -c 3 -H crush 15' to finish migration.
LustreError: 15854:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0001: migrate [0x240000403:0x1:0x0]/14 failed: rc = -114
LustreError: 15854:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 1 previous similar message
Lustre: 18216:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 508 < left 938, rollback = 2
Lustre: 18216:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 50 previous similar messages
Lustre: 18216:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/4, destroy: 0/0/0
Lustre: 18216:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 50 previous similar messages
Lustre: 18216:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 10/938/0
Lustre: 18216:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 50 previous similar messages
Lustre: 18216:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0
Lustre: 18216:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 50 previous similar messages
Lustre: 18216:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/3, delete: 0/0/0
Lustre: 18216:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 50 previous similar messages
Lustre: 18216:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0
Lustre: 18216:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 50 previous similar messages
Lustre: 9100:0:(mdt_recovery.c:149:mdt_req_from_lrd()) @@@ restoring transno req@ffff8802989e2dc0 x1782000926321984/t4294968813(0) o101->933bfd55-8c20-4a83-a73c-428bcadb1faf@0@lo:263/0 lens 376/864 e 0 to 0 dl 1699448638 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
LustreError: 16833:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200000403:0x1:0x0]/3 failed: rc = -16
LustreError: 16833:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 7 previous similar messages
LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) header@ffff880290d38008[0x4, 2, [0x380000401:0x9:0x0] hash]{
LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880290d38088[0]
LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880295caf8c8id: 0x380000401:9 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448617 atime: 1699448617 ctime: 1699448617 blocks: 0
LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) } header@ffff880290d38008
LustreError: 18439:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) header@ffff8802860af248[0x4, 1, [0x240000404:0x35:0x0] hash]{
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802860af2c8(0) inode: ffff88028dda8108 162129603832315957/37748740 100644 1 0 ffff8802860af2c8 [0x240000404:0x35:0x0]
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880286976388entries: 1, valid, lsm{ffff88029605fb18 0x0BD10BD0 1 0}:
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) header@ffff880290d38008[0x4, 2, [0x380000401:0x9:0x0] hash]{
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880290d38088[0]
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880295caf8c8id: 0x380000401:9 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448617 atime: 1699448617 ctime: 1699448617 blocks: 0
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) } header@ffff880290d38008
LustreError: 18439:0:(lov_object.c:185:lov_init_sub())
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802860af248
LustreError: 18439:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 18439:0:(lov_object.c:186:lov_init_sub()) header@ffff880088a73aa8[0x0, 1, [0x200000404:0x40:0x0] hash]
LustreError: 18439:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 18439:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x40:0x0]: rc = -5
LustreError: 18439:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
Lustre: 9084:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 510 < left 1756, rollback = 2
Lustre: 9084:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 47 previous similar messages
Lustre: 9084:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/2, destroy: 1/4/0
Lustre: 9084:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 47 previous similar messages
Lustre: 9084:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 19/1756/0
Lustre: 9084:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 47 previous similar messages
Lustre: 9084:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 7/105/0
Lustre: 9084:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 47 previous similar messages
Lustre: 9084:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 14/287/2, delete: 2/5/1
Lustre: 9084:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 47 previous similar messages
Lustre: 9084:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 2/2/0
Lustre: 9084:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 47 previous similar messages
LustreError: 9084:0:(mdt_xattr.c:415:mdt_dir_layout_update()) lustre-MDT0001: [0x240000403:0x90:0x0] migrate mdt count mismatch 2 != 1
LustreError: 19877:0:(llite_lib.c:1836:ll_update_lsm_md()) lustre: [0x240000403:0xb5:0x0] dir layout mismatch:
LustreError: 19877:0:(lustre_lmv.h:178:lmv_stripe_object_dump()) dump LMV: refs 215092432 magic=0x2 count=2 index=1 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool=
LustreError: 19877:0:(lustre_lmv.h:185:lmv_stripe_object_dump()) stripe[0] [0x240000400:0x8:0x0]
LustreError: 19877:0:(lustre_lmv.h:178:lmv_stripe_object_dump()) dump LMV: refs 215092432 magic=0x1 count=3 index=1 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=2 migrate_hash=fnv_1a_64:2 pool=
LustreError: 21243:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x200000403:0x10:0x0] mdc close failed: rc = -116
LustreError: 9082:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/6 failed: rc = -16
LustreError: 9082:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 15 previous similar messages
Lustre: 16833:0:(mdt_recovery.c:149:mdt_req_from_lrd()) @@@ restoring transno req@ffff8802cc3ece40 x1782000927694784/t4294970643(0) o101->caee2dfc-acdf-4df3-88cc-a51025f1266a@0@lo:281/0 lens 376/840 e 0 to 0 dl 1699448656 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
Lustre: 17654:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 514 < left 565, rollback = 2
Lustre: 17654:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 194 previous similar messages
Lustre: 17654:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 17654:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 194 previous similar messages
Lustre: 17654:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 7/565/0
Lustre: 17654:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 194 previous similar messages
Lustre: 17654:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/21/0, punch: 0/0/0, quota 6/80/0
Lustre: 17654:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 194 previous similar messages
Lustre: 17654:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 1/36/0, delete: 2/2/1
Lustre: 17654:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 194 previous similar messages
Lustre: 17654:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 1/1/0
Lustre: 17654:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 194 previous similar messages
Lustre: 16506:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0xfd:0x0] with magic=0xbd60bd0
Lustre: 19082:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0xee:0x0] with magic=0xbd60bd0
Lustre: 19082:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 9 previous similar messages
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d638b00[0]
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d638a80
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff880088a70a80[0x4, 1, [0x280000403:0x8d:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088a70b00(0) inode: ffff88028defe988 180144002325020813/41943044 100644 1 0 ffff880088a70b00 [0x280000403:0x8d:0x0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01a7f8entries: 4, valid, lsm{ffff880289eaa9b8 0x0BD60BD0 1 4}:
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d638b00[0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d638a80
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub())
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088a70a80
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff88028b43fcc0[0x0, 1, [0x240000404:0x1a1:0x0] hash]
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 26244:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x1a1:0x0]: rc = -5
LustreError: 26244:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0]
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d63b460
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff8802783e7890[0x4, 1, [0x280000404:0xfc:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802783e7910(0) inode: ffff88028dfee008 180144002341798140/41943044 100644 1 0 ffff8802783e7910 [0x280000404:0xfc:0x0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880326f325c0entries: 4, valid, lsm{ffff88028ed8c008 0x0BD60BD0 1 4}:
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d63b460
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub())
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802783e7890
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff88008fa5e188[0x0, 1, [0x240000404:0x213:0x0] hash]
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own.
Lustre: 9079:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x13d:0x0] with magic=0xbd60bd0
Lustre: 9079:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 23 previous similar messages
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d126208[0]
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d126188
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a42b890[0x4, 1, [0x200000404:0x5:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a42b910(0) inode: ffff88028dc33088 144115205322833925/33554436 101155 2 0 ffff88008a42b910 [0x200000404:0x5:0x0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880297f23310entries: 2, valid, lsm{ffff88028d71d5f8 0x0BD60BD0 1 5}:
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x8000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d126208[0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d126188
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 3, 0x1, 0, 0x10, 3, 2097152 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125928[0x4, 1, [0x300000402:0x9:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d1259a8[65536]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895cda8id: 0x300000402:9 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 4008 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125928
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125f70[0x4, 1, [0x340000402:0xb:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125ff0[65537]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828405c8id: 0x340000402:11 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125f70
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125b40[0x4, 1, [0x2c0000402:0xb:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125bc0[65538]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282840e68id: 0x2c0000402:11 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125b40
LustreError: 26244:0:(lov_object.c:185:lov_init_sub())
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a42b890
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff88008fa5fcc0[0x0, 1, [0x240000404:0x1f0:0x0] hash]
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own.
Lustre: 10981:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028908fd40[0]
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028908fcc0
LustreError: 26244:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43c008[0x4, 1, [0x200000404:0x26:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43c088(0) inode: ffff880090ba9408 144115205322833958/33554436 101755 2 0 ffff88028b43c088 [0x200000404:0x26:0x0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880289b9cd58entries: 1, valid, lsm{ffff88008d498f38 0x0BD10BD0 1 0}:
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028908fd40[0]
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028908fcc0
LustreError: 26244:0:(lov_object.c:185:lov_init_sub())
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43c008
LustreError: 26244:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) header@ffff880287627cc0[0x0, 1, [0x240000404:0x216:0x0] hash]
LustreError: 26244:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 26244:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x216:0x0]: rc = -5
LustreError: 26244:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 2 previous similar messages
LustreError: 26244:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 26244:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 2 previous similar messages
Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d126208[0]
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d126188
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a42b890[0x4, 1, [0x200000404:0x5:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a42b910(0) inode: ffff88028dc33088 144115205322833925/33554436 101155 2 0 ffff88008a42b910 [0x200000404:0x5:0x0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880297f23310entries: 2, valid, lsm{ffff88028d71d5f8 0x0BD60BD0 1 5}:
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x8000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d126208[0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d126188
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 3, 0x1, 0, 0x10, 3, 2097152 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125928[0x4, 1, [0x300000402:0x9:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d1259a8[65536]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895cda8id: 0x300000402:9 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 4008 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125928
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125f70[0x4, 1, [0x340000402:0xb:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125ff0[65537]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828405c8id: 0x340000402:11 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125f70
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125b40[0x4, 1, [0x2c0000402:0xb:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125bc0[65538]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282840e68id: 0x2c0000402:11 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125b40
LustreError: 27619:0:(lov_object.c:185:lov_init_sub())
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a42b890
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff880287625928[0x0, 1, [0x240000404:0x1f0:0x0] hash]
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 27619:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x1f0:0x0]: rc = -5
LustreError: 27619:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{
LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880293fb1148[0]
LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0
LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) } header@ffff880293fb10c8
LustreError: 28280:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) header@ffff880293ee0438[0x4, 1, [0x200000404:0x8d:0x0] hash]{
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880293ee04b8(0) inode: ffff880084c39d88 144115205322834061/33554436 100644 1 0 ffff880293ee04b8 [0x200000404:0x8d:0x0]
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01b9b8entries: 1, valid, lsm{ffff88028e50fcc8 0x0BD10BD0 1 0}:
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880293fb1148[0]
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293fb10c8
LustreError: 28280:0:(lov_object.c:185:lov_init_sub())
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293ee0438
LustreError: 28280:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 28280:0:(lov_object.c:186:lov_init_sub()) header@ffff880296178650[0x0, 1, [0x240000403:0x13c:0x0] hash]
LustreError: 28280:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d638b00[0]
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d638a80
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff880088a70a80[0x4, 1, [0x280000403:0x8d:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088a70b00(0) inode: ffff88028defe988 180144002325020813/41943044 100644 1 0 ffff880088a70b00 [0x280000403:0x8d:0x0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01a7f8entries: 4, valid, lsm{ffff880289eaa9b8 0x0BD60BD0 1 4}:
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d638a80[0x4, 2, [0x2c0000400:0x1b:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d638b00[0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88029606d368id: 0x2c0000400:27 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448629 atime: 1699448629 ctime: 1699448629 blocks: 0
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d638a80
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub())
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088a70a80
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff88029617a9e8[0x0, 1, [0x240000404:0x1a1:0x0] hash]
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0]
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028d63b460
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff8802783e7890[0x4, 1, [0x280000404:0xfc:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802783e7910(0) inode: ffff88028dfee008 180144002341798140/41943044 100644 1 0 ffff8802783e7910 [0x280000404:0xfc:0x0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880326f325c0entries: 4, valid, lsm{ffff88028ed8c008 0x0BD60BD0 1 4}:
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028d63b460[0x4, 2, [0x340000400:0x20:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028d63b4e0[0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282847028id: 0x340000400:32 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448643 atime: 1699448647 ctime: 1699448643 blocks: 8
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028d63b460
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 1, 2097152 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 1, 131072 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub())
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802783e7890
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff88029617a5b8[0x0, 1, [0x240000404:0x213:0x0] hash]
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d126208[0]
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d126188
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a42b890[0x4, 1, [0x200000404:0x5:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a42b910(0) inode: ffff88028dc33088 144115205322833925/33554436 101155 2 0 ffff88008a42b910 [0x200000404:0x5:0x0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880297f23310entries: 2, valid, lsm{ffff88028d71d5f8 0x0BD60BD0 1 5}:
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x8000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d126188[0x4, 2, [0x380000402:0x2:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d126208[0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895b3c8id: 0x380000402:2 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 8834 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d126188
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 3, 0x1, 0, 0x10, 3, 2097152 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125928[0x4, 1, [0x300000402:0x9:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d1259a8[65536]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895cda8id: 0x300000402:9 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 4008 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 1
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125928
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125f70[0x4, 1, [0x340000402:0xb:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125ff0[65537]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828405c8id: 0x340000402:11 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125f70
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d125b40[0x4, 1, [0x2c0000402:0xb:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d125bc0[65538]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880282840e68id: 0x2c0000402:11 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448644 atime: 1699448619 ctime: 1699448646 blocks: 0
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d125b40
LustreError: 27619:0:(lov_object.c:185:lov_init_sub())
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a42b890
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff88027a2c4220[0x0, 1, [0x240000404:0x1f0:0x0] hash]
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028908fd40[0]
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028908fcc0
LustreError: 27619:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43c008[0x4, 1, [0x200000404:0x26:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43c088(0) inode: ffff880090ba9408 144115205322833958/33554436 101755 2 0 ffff88028b43c088 [0x200000404:0x26:0x0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880289b9cd58entries: 1, valid, lsm{ffff88008d498f38 0x0BD10BD0 1 0}:
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) header@ffff88028908fcc0[0x4, 2, [0x340000402:0x5:0x0] hash]{
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028908fd40[0]
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88028895f308id: 0x340000402:5 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448650 blocks: 72
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028908fcc0
LustreError: 27619:0:(lov_object.c:185:lov_init_sub())
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43c008
LustreError: 27619:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) header@ffff880287624a80[0x0, 1, [0x240000404:0x216:0x0] hash]
LustreError: 27619:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 27619:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x216:0x0]: rc = -5
LustreError: 27619:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 4 previous similar messages
LustreError: 27619:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 27619:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 4 previous similar messages
LustreError: 29138:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x200000404:0x14e:0x0] mdc close failed: rc = -116
Lustre: 16855:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000404:0x1a1:0x0] with magic=0xbd60bd0
Lustre: 16855:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 13 previous similar messages
Lustre: 19400:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 514 < left 618, rollback = 7
Lustre: 19400:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message
LustreError: 29723:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x240000403:0x1c8:0x0] mdc close failed: rc = -116
LustreError: 29723:0:(file.c:246:ll_close_inode_openhandle()) Skipped 1 previous similar message
14[29966]: segfault at 8 ip 00007f50e349b7e8 sp 00007fffdddba8d0 error 4 in ld-2.17.so[7f50e3490000+22000]
LustreError: 27696:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000403:0x1:0x0]/19 failed: rc = -114
LustreError: 27696:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 15 previous similar messages
LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{
LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880293fb1148[0]
LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0
LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) } header@ffff880293fb10c8
LustreError: 30062:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) header@ffff880293ee0438[0x4, 1, [0x200000404:0x8d:0x0] hash]{
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880293ee04b8(0) inode: ffff880084c39d88 144115205322834061/33554436 100644 1 0 ffff880293ee04b8 [0x200000404:0x8d:0x0]
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008c01b9b8entries: 1, valid, lsm{ffff88028e50fcc8 0x0BD10BD0 1 0}:
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) header@ffff880293fb10c8[0x4, 2, [0x340000402:0xd:0x0] hash]{
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880293fb1148[0]
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802828433c8id: 0x340000402:13 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448640 atime: 1699448640 ctime: 1699448640 blocks: 0
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293fb10c8
LustreError: 30062:0:(lov_object.c:185:lov_init_sub())
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) } header@ffff880293ee0438
LustreError: 30062:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 30062:0:(lov_object.c:186:lov_init_sub()) header@ffff880287625f70[0x0, 1, [0x240000403:0x13c:0x0] hash]
LustreError: 30062:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 30062:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000403:0x13c:0x0]: rc = -5
LustreError: 30062:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
Lustre: 21222:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 534, rollback = 7
Lustre: 21222:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message
LustreError: 31331:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x240000403:0x26a:0x0] mdc close failed: rc = -116
LustreError: 31331:0:(file.c:246:ll_close_inode_openhandle()) Skipped 6 previous similar messages
Lustre: dir [0x200000404:0x1db:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: 29795:0:(mdt_recovery.c:149:mdt_req_from_lrd()) @@@ restoring transno req@ffff880283d147c0 x1782000930991424/t4294971292(0) o101->caee2dfc-acdf-4df3-88cc-a51025f1266a@0@lo:322/0 lens 376/816 e 0 to 0 dl 1699448697 ref 1 fl Interpret:H/202/0 rc 0/0 job:'dd.0' uid:0 gid:0
Lustre: 16682:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 499 < left 671, rollback = 2
Lustre: 16682:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 108 previous similar messages
Lustre: 16682:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/12, destroy: 0/0/0
Lustre: 16682:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 116 previous similar messages
Lustre: 16682:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0
Lustre: 16682:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 116 previous similar messages
Lustre: 16682:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 4/54/0
Lustre: 16682:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 116 previous similar messages
Lustre: 16682:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/4, delete: 0/0/0
Lustre: 16682:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 116 previous similar messages
Lustre: 16682:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0
Lustre: 16682:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 116 previous similar messages
Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20537:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message
Lustre: mdt02_000: service thread pid 9078 was inactive for 40.075 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Pid: 9078, comm: mdt02_000 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt]
[<0>] mdt_object_lock_try+0xa0/0x250 [mdt]
[<0>] mdt_getattr_name_lock+0x17f9/0x2b30 [mdt]
[<0>] mdt_intent_getattr+0x2c5/0x4b0 [mdt]
[<0>] mdt_intent_opc+0x1c2/0xc40 [mdt]
[<0>] mdt_intent_policy+0xfa/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x3b1/0xbb0 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x34d/0x17b0 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc]
[<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 9083, comm: mdt03_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt]
[<0>] mdt_object_lock+0x88/0x1c0 [mdt]
[<0>] mdt_object_find_lock+0x54/0x170 [mdt]
[<0>] mdt_reint_setxattr+0x169/0x1110 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x74c/0xba0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc]
[<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 9091, comm: mdt05_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] mdt_object_lock_internal+0x1a9/0x420 [mdt]
[<0>] mdt_object_lock_try+0xa0/0x250 [mdt]
[<0>] mdt_getattr_name_lock+0x17f9/0x2b30 [mdt]
[<0>] mdt_intent_getattr+0x2c5/0x4b0 [mdt]
[<0>] mdt_intent_opc+0x1c2/0xc40 [mdt]
[<0>] mdt_intent_policy+0xfa/0x460 [mdt]
[<0>] ldlm_lock_enqueue+0x3b1/0xbb0 [ptlrpc]
[<0>] ldlm_handle_enqueue+0x34d/0x17b0 [ptlrpc]
[<0>] tgt_enqueue+0x68/0x240 [ptlrpc]
[<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc]
[<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: mdt05_009: service thread pid 18596 was inactive for 40.545 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) header@ffff88027dff6188[0x4, 2, [0x300000400:0x30:0x0] hash]{
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027dff6208[0]
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027dff6188
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43f460[0x4, 1, [0x280000404:0x210:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43f4e0(0) inode: ffff8802775f9408 180144002341798416/41943044 100644 1 0 ffff88028b43f4e0 [0x280000404:0x210:0x0]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc4478entries: 1, valid, lsm{ffff88008df86018 0x0BD10BD0 1 0}:
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 3, 1048576 }
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff6188[0x4, 2, [0x300000400:0x30:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6208[0]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff6188
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff69e8[0x4, 1, [0x340000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6a68[1]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff69e8
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff8802883c8a80[0x4, 1, [0x380000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802883c8b00[2]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802883c8a80
LustreError: 302:0:(lov_object.c:185:lov_init_sub())
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43f460
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 302:0:(lov_object.c:186:lov_init_sub()) header@ffff88008711e7d0[0x0, 1, [0x200000403:0x1b2:0x0] hash]
LustreError: 302:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) header@ffff88027dff69e8[0x4, 2, [0x340000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027dff6a68[1]
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027dff69e8
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) stripe 1 is already owned.
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43f460[0x4, 1, [0x280000404:0x210:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43f4e0(0) inode: ffff8802775f9408 180144002341798416/41943044 100644 1 0 ffff88028b43f4e0 [0x280000404:0x210:0x0]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc4478entries: 1, valid, lsm{ffff88008df86018 0x0BD10BD0 1 0}:
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 3, 1048576 }
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff6188[0x4, 1, [0x300000400:0x30:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6208[0]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff6188
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff69e8[0x4, 2, [0x340000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6a68[1]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff69e8
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff8802883c8a80[0x4, 1, [0x380000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802883c8b00[2]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802883c8a80
LustreError: 302:0:(lov_object.c:185:lov_init_sub())
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43f460
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 302:0:(lov_object.c:186:lov_init_sub()) header@ffff88008711e7d0[0x0, 1, [0x200000403:0x1b2:0x0] hash]
LustreError: 302:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) header@ffff8802883c8a80[0x4, 2, [0x380000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802883c8b00[2]
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802883c8a80
LustreError: 302:0:(lov_object.c:184:lov_init_sub()) stripe 2 is already owned.
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88028b43f460[0x4, 1, [0x280000404:0x210:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028b43f4e0(0) inode: ffff8802775f9408 180144002341798416/41943044 100644 1 0 ffff88028b43f4e0 [0x280000404:0x210:0x0]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc4478entries: 1, valid, lsm{ffff88008df86018 0x0BD10BD0 1 0}:
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 3, 1048576 }
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff6188[0x4, 1, [0x300000400:0x30:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6208[0]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880280813c68id: 0x300000400:48 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 8
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff6188
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dff69e8[0x4, 1, [0x340000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027dff6a68[1]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a2e08id: 0x340000400:50 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dff69e8
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) header@ffff8802883c8a80[0x4, 2, [0x380000400:0x32:0x0] hash]{
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802883c8b00[2]
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027f5a1cc8id: 0x380000400:50 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448696 atime: 0 ctime: 1699448696 blocks: 0
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802883c8a80
LustreError: 302:0:(lov_object.c:185:lov_init_sub())
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028b43f460
LustreError: 302:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 302:0:(lov_object.c:186:lov_init_sub()) header@ffff88008711e7d0[0x0, 1, [0x200000403:0x1b2:0x0] hash]
LustreError: 302:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 302:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000403:0x1b2:0x0]: rc = -5
LustreError: 302:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 1040:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x240000404:0x373:0x0] mdc close failed: rc = -116
Lustre: dir [0x240000404:0x358:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
LustreError: 9119:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0x358:0x0]: rc = -2
Lustre: 20409:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20409:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 1 previous similar message
LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) header@ffff88008417a188[0x4, 2, [0x2c0000400:0x3b:0x0] hash]{
LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88008417a208[0]
LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880278ab9148id: 0x2c0000400:59 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448717 blocks: 72
LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) } header@ffff88008417a188
LustreError: 4080:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) header@ffff880282830650[0x4, 1, [0x280000403:0x216:0x0] hash]{
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802828306d0(0) inode: ffff880084da4388 180144002325021206/41943044 100700 1 0 ffff8802828306d0 [0x280000403:0x216:0x0]
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efc4240entries: 1, valid, lsm{ffff88027ae9a888 0x0BD10BD0 1 0}:
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) header@ffff88008417a188[0x4, 2, [0x2c0000400:0x3b:0x0] hash]{
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88008417a208[0]
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880278ab9148id: 0x2c0000400:59 idx: 0 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448717 blocks: 72
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008417a188
LustreError: 4080:0:(lov_object.c:185:lov_init_sub())
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) } header@ffff880282830650
LustreError: 4080:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 4080:0:(lov_object.c:186:lov_init_sub()) header@ffff880085868eb0[0x0, 1, [0x240000404:0x430:0x0] hash]
LustreError: 4080:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9121:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000403:0x1d2:0x0]: rc = -2
LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{
LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88029765b0b0[0]
LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16
LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) } header@ffff88029765b030
LustreError: 4487:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) header@ffff88008711df70[0x4, 1, [0x200000404:0x24e:0x0] hash]{
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008711dff0(0) inode: ffff880084da8a88 144115205322834510/33554436 100755 3 0 ffff88008711dff0 [0x200000404:0x24e:0x0]
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efc7780entries: 1, valid, lsm{ffff88028700e378 0x0BD10BD0 1 0}:
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88029765b0b0[0]
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) } header@ffff88029765b030
LustreError: 4487:0:(lov_object.c:185:lov_init_sub())
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008711df70
LustreError: 4487:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 4487:0:(lov_object.c:186:lov_init_sub()) header@ffff880279c729e8[0x4, 1, [0x280000403:0x250:0x0] hash]
LustreError: 4487:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 4487:0:(lov_object.c:1337:lov_layout_change()) lustre-clilov-ffff88008fa0e678: cannot apply new layout on [0x280000403:0x250:0x0] : rc = -5
LustreError: 4487:0:(vvp_io.c:1876:vvp_io_init()) lustre: refresh file layout [0x280000403:0x250:0x0] error -5.
Lustre: 16367:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x2b5:0x0] with magic=0xbd60bd0
Lustre: 16367:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 3 previous similar messages
LustreError: 29805:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200000404:0x27b:0x0]/11 failed: rc = -2
LustreError: 29805:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 65 previous similar messages
Lustre: 10970:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
Lustre: 10970:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 3 previous similar messages
Lustre: dir [0x200000404:0x2e3:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{
LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88029765b0b0[0]
LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16
LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) } header@ffff88029765b030
LustreError: 6589:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) header@ffff88008711df70[0x4, 1, [0x200000404:0x24e:0x0] hash]{
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008711dff0(0) inode: ffff880084da8a88 144115205322834510/33554436 100755 3 0 ffff88008711dff0 [0x200000404:0x24e:0x0]
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efc7780entries: 1, valid, lsm{ffff88028700e378 0x0BD10BD0 1 0}:
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) header@ffff88029765b030[0x4, 2, [0x380000402:0x29:0x0] hash]{
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88029765b0b0[0]
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880278ab9708id: 0x380000402:41 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6390 mtime: 1699448730 atime: 1699448730 ctime: 1699448730 blocks: 16
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) } header@ffff88029765b030
LustreError: 6589:0:(lov_object.c:185:lov_init_sub())
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008711df70
LustreError: 6589:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 6589:0:(lov_object.c:186:lov_init_sub()) header@ffff880279c729e8[0x4, 1, [0x280000403:0x250:0x0] hash]
LustreError: 6589:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 6589:0:(lov_object.c:1337:lov_layout_change()) lustre-clilov-ffff88008fa0e678: cannot apply new layout on [0x280000403:0x250:0x0] : rc = -5
LustreError: 6589:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000403:0x250:0x0]: rc = -5
LustreError: 6589:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 6589:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 6589:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 1 previous similar message
Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000403:0x271:0x0] with magic=0xbd60bd0
Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 7 previous similar messages
Lustre: 9100:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 503 < left 671, rollback = 2
Lustre: 9100:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 462 previous similar messages
Lustre: 9100:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 3/12/8, destroy: 0/0/0
Lustre: 9100:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 470 previous similar messages
Lustre: 9100:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0
Lustre: 9100:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 470 previous similar messages
Lustre: 9100:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 5/55/0, punch: 0/0/0, quota 1/3/0
Lustre: 9100:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 470 previous similar messages
Lustre: 9100:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 11/239/4, delete: 0/0/0
Lustre: 9100:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 470 previous similar messages
Lustre: 9100:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 5/5/0, ref_del: 0/0/0
Lustre: 9100:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 470 previous similar messages
LustreError: 7371:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x200000404:0x2fc:0x0] mdc close failed: rc = -116
LustreError: 7371:0:(file.c:246:ll_close_inode_openhandle()) Skipped 8 previous similar messages
LustreError: 16690:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000403:0x224:0x0]: rc = -2
Lustre: dir [0x280000404:0x305:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 6755:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0]
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}:
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 6755:0:(lov_object.c:185:lov_init_sub())
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00
LustreError: 6755:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 6755:0:(lov_object.c:186:lov_init_sub()) header@ffff880080451b40[0x0, 1, [0x200000403:0x32b:0x0] hash]
LustreError: 6755:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 7198:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0]
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}:
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 7198:0:(lov_object.c:185:lov_init_sub())
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00
LustreError: 7198:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 7198:0:(lov_object.c:186:lov_init_sub()) header@ffff88027cf4f890[0x0, 1, [0x200000403:0x32b:0x0] hash]
LustreError: 7198:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 8493:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0]
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}:
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 8493:0:(lov_object.c:185:lov_init_sub())
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00
LustreError: 8493:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8493:0:(lov_object.c:186:lov_init_sub()) header@ffff880275d95d58[0x0, 1, [0x200000403:0x32b:0x0] hash]
LustreError: 8493:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 8956:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0002_UUID lock: ffff88029214de00/0x6a008282726167e6 lrc: 3/0,0 mode: PR/PR res: [0x280000403:0x1a2:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x6a008282726167bc expref: 248 pid: 16506 timeout: 310 lvb_type: 0
LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{
LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802907cddd8[0]
LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0
LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802907cdd58
LustreError: 6677:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 11-0: lustre-MDT0002-mdc-ffff88009074b7e8: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-MDT0002-mdc-ffff88009074b7e8: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 167-0: lustre-MDT0002-mdc-ffff88009074b7e8: This client was evicted by lustre-MDT0002; in progress operations using this service will fail.
LustreError: 27089:0:(vvp_io.c:1876:vvp_io_init()) lustre: refresh file layout [0x280000403:0x1a2:0x0] error -108.
LustreError: 26878:0:(file.c:5373:ll_inode_revalidate_fini()) lustre: revalidate FID [0x280000403:0x1a2:0x0] error: rc = -108
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dca94f8[0x4, 1, [0x240000403:0x48e:0x0] hash]{
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dca9578(0) inode: ffff88026cd53088 162129603815539854/37748740 107514 2 0 ffff88027dca9578 [0x240000403:0x48e:0x0]
LustreError: 8194:0:(mdc_request.c:1465:mdc_read_page()) lustre-MDT0002-mdc-ffff88009074b7e8: [0x280000401:0x14:0x0] lock enqueue fails: rc = -5
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802793227f8entries: 1, valid, lsm{ffff8802820281b8 0x0BD10BD0 1 0}:
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802907cddd8[0]
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0
Lustre: lustre-MDT0002-mdc-ffff88009074b7e8: Connection restored to (at 0@lo)
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802907cdd58
LustreError: 6677:0:(lov_object.c:185:lov_init_sub())
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dca94f8
LustreError: 6677:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 6677:0:(lov_object.c:186:lov_init_sub()) header@ffff88027acb0a80[0x0, 1, [0x280000403:0x2ab:0x0] hash]
LustreError: 6677:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9083:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '0' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 0' to finish migration.
16[8823]: segfault at 8 ip 00007f8b288237e8 sp 00007ffcaa09f790 error 4 in ld-2.17.so[7f8b28818000+22000]
LustreError: 27680:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '1' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 1' to finish migration.
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802907cddd8[0]
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802907cdd58
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dca94f8[0x4, 1, [0x240000403:0x48e:0x0] hash]{
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dca9578(0) inode: ffff88026cd53088 162129603815539854/37748740 107514 2 0 ffff88027dca9578 [0x240000403:0x48e:0x0]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802793227f8entries: 1, valid, lsm{ffff8802820281b8 0x0BD10BD0 1 0}:
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802907cddd8[0]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802907cdd58
LustreError: 9800:0:(lov_object.c:185:lov_init_sub())
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dca94f8
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) header@ffff880087c13460[0x0, 1, [0x280000403:0x2ab:0x0] hash]
LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
4[9398]: segfault at 8 ip 00007f42328a67e8 sp 00007fff260698c0 error 4 in ld-2.17.so[7f423289b000+22000]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff88008a6fac00[0x4, 1, [0x240000403:0x298:0x0] hash]{
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88008a6fac80(0) inode: ffff8802776af308 162129603815539352/37748740 100755 2 0 ffff88008a6fac80 [0x240000403:0x298:0x0]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880278bc5870entries: 1, valid, lsm{ffff88027d1c8a28 0x0BD10BD0 1 0}:
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff880279ca0a80[0x4, 2, [0x2c0000401:0x4e:0x0] hash]{
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880279ca0b00[0]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111f028id: 0x2c0000401:78 idx: 0 gen: 0 kms_valid: 1 kms 262144 rc: 0 force_sync: 0 min_xid: 0 size: 262144 mtime: 1699448751 atime: 1699391508 ctime: 1699448751 blocks: 512
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff880279ca0a80
LustreError: 9800:0:(lov_object.c:185:lov_init_sub())
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008a6fac00
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) header@ffff880087c125b8[0x0, 1, [0x200000403:0x32b:0x0] hash]
LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e884b8[0]
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e88438
LustreError: 9800:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff88027cf4e9e8[0x4, 1, [0x280000403:0x234:0x0] hash]{
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027cf4ea68(0) inode: ffff88026cdf8108 180144002325021236/41943044 100755 1 0 ffff88027cf4ea68 [0x280000403:0x234:0x0]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802821cb548entries: 1, valid, lsm{ffff88027b30e1c8 0x0BD10BD0 1 0}:
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e884b8[0]
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e88438
LustreError: 9800:0:(lov_object.c:185:lov_init_sub())
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027cf4e9e8
LustreError: 9800:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) header@ffff880087c110c8[0x0, 1, [0x200000403:0x32e:0x0] hash]
LustreError: 9800:0:(lov_object.c:186:lov_init_sub()) try to own.
Lustre: 10969:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 540, rollback = 7
Lustre: 10969:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 19 previous similar messages
LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{
LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802907cddd8[0]
LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0
LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802907cdd58
LustreError: 8376:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dca94f8[0x4, 1, [0x240000403:0x48e:0x0] hash]{
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dca9578(0) inode: ffff88026cd53088 162129603815539854/37748740 107514 2 0 ffff88027dca9578 [0x240000403:0x48e:0x0]
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802793227f8entries: 1, valid, lsm{ffff8802820281b8 0x0BD10BD0 1 0}:
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) header@ffff8802907cdd58[0x4, 2, [0x380000401:0x87:0x0] hash]{
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802907cddd8[0]
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008111a288id: 0x380000401:135 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 9470 mtime: 1699448749 atime: 0 ctime: 1699448749 blocks: 0
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802907cdd58
LustreError: 8376:0:(lov_object.c:185:lov_init_sub())
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dca94f8
LustreError: 8376:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8376:0:(lov_object.c:186:lov_init_sub()) header@ffff8800804510c8[0x0, 1, [0x280000403:0x2ab:0x0] hash]
LustreError: 8376:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 20324:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '0' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 0' to finish migration.
LustreError: 20324:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) Skipped 1 previous similar message
LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f794f8[0x4, 2, [0x300000400:0x5a:0x0] hash]{
LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f79578[0]
LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880082308e68id: 0x300000400:90 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448773 atime: 1699448773 ctime: 1699448773 blocks: 0
LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f794f8
LustreError: 11129:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) header@ffff88027dcab030[0x4, 1, [0x280000405:0x2e:0x0] hash]{
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027dcab0b0(0) inode: ffff88026cdc0108 180144002358575150/41943044 100644 1 0 ffff88027dcab0b0 [0x280000405:0x2e:0x0]
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280a19aa8entries: 2, valid, lsm{ffff88026efb2a38 0x0BD60BD0 1 2}:
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x40000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 65536 }
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f794f8[0x4, 2, [0x300000400:0x5a:0x0] hash]{
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f79578[0]
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880082308e68id: 0x300000400:90 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448773 atime: 1699448773 ctime: 1699448773 blocks: 0
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f794f8
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) [0x40000, 0xffffffffffffffff): { 0x0BD10BD0, 2, 0x1, 65535, 0x0, 1, 2097152 }
LustreError: 11129:0:(lov_object.c:185:lov_init_sub())
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027dcab030
LustreError: 11129:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 11129:0:(lov_object.c:186:lov_init_sub()) header@ffff88027cf4f248[0x0, 1, [0x240000404:0x5b9:0x0] hash]
LustreError: 11129:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{
LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e884b8[0]
LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1
LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e88438
LustreError: 11688:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) header@ffff88027cf4e9e8[0x4, 1, [0x280000403:0x234:0x0] hash]{
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027cf4ea68(0) inode: ffff88026cdf8108 180144002325021236/41943044 100755 1 0 ffff88027cf4ea68 [0x280000403:0x234:0x0]
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802821cb548entries: 1, valid, lsm{ffff88027b30e1c8 0x0BD10BD0 1 0}:
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e884b8[0]
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e88438
LustreError: 11688:0:(lov_object.c:185:lov_init_sub())
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027cf4e9e8
LustreError: 11688:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 11688:0:(lov_object.c:186:lov_init_sub()) header@ffff8802889a94f8[0x0, 1, [0x200000403:0x32e:0x0] hash]
LustreError: 11688:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{
LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e884b8[0]
LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1
LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e88438
LustreError: 11609:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) header@ffff88027cf4e9e8[0x4, 1, [0x280000403:0x234:0x0] hash]{
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027cf4ea68(0) inode: ffff88026cdf8108 180144002325021236/41943044 100755 1 0 ffff88027cf4ea68 [0x280000403:0x234:0x0]
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802821cb548entries: 1, valid, lsm{ffff88027b30e1c8 0x0BD10BD0 1 0}:
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e88438[0x4, 2, [0x340000400:0x40:0x0] hash]{
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e884b8[0]
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088b9b988id: 0x340000400:64 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448757 blocks: 1
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e88438
LustreError: 11609:0:(lov_object.c:185:lov_init_sub())
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027cf4e9e8
LustreError: 11609:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 11609:0:(lov_object.c:186:lov_init_sub()) header@ffff8802889a9b40[0x0, 1, [0x200000403:0x32e:0x0] hash]
LustreError: 11609:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{
LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f7ae98[0]
LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72
LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f7ae18
LustreError: 11168:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) header@ffff880083424eb0[0x4, 1, [0x200000404:0x336:0x0] hash]{
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880083424f30(0) inode: ffff880084e11408 144115205322834742/33554436 100700 1 0 ffff880083424f30 [0x200000404:0x336:0x0]
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008409e150entries: 1, valid, lsm{ffff880295c1e018 0x0BD10BD0 1 0}:
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f7ae98[0]
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f7ae18
LustreError: 11168:0:(lov_object.c:185:lov_init_sub())
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) } header@ffff880083424eb0
LustreError: 11168:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 11168:0:(lov_object.c:186:lov_init_sub()) header@ffff880083425d58[0x0, 1, [0x240000403:0x5da:0x0] hash]
LustreError: 11168:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9087:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0000: '1' migration was interrupted, run 'lfs migrate -m 2 -c 3 -H crush 1' to finish migration.
LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{
LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f7ae98[0]
LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72
LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f7ae18
LustreError: 12370:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) header@ffff880083424eb0[0x4, 1, [0x200000404:0x336:0x0] hash]{
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880083424f30(0) inode: ffff880084e11408 144115205322834742/33554436 100700 1 0 ffff880083424f30 [0x200000404:0x336:0x0]
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008409e150entries: 1, valid, lsm{ffff880295c1e018 0x0BD10BD0 1 0}:
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f7ae98[0]
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f7ae18
LustreError: 12370:0:(lov_object.c:185:lov_init_sub())
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) } header@ffff880083424eb0
LustreError: 12370:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 12370:0:(lov_object.c:186:lov_init_sub()) header@ffff880280b44868[0x0, 1, [0x240000403:0x5da:0x0] hash]
LustreError: 12370:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{
LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880280f7ae98[0]
LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72
LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) } header@ffff880280f7ae18
LustreError: 12597:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) header@ffff880083424eb0[0x4, 1, [0x200000404:0x336:0x0] hash]{
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880083424f30(0) inode: ffff880084e11408 144115205322834742/33554436 100700 1 0 ffff880083424f30 [0x200000404:0x336:0x0]
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008409e150entries: 1, valid, lsm{ffff880295c1e018 0x0BD10BD0 1 0}:
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) header@ffff880280f7ae18[0x4, 2, [0x300000402:0x46:0x0] hash]{
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880280f7ae98[0]
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880271b7bc68id: 0x300000402:70 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448759 blocks: 72
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280f7ae18
LustreError: 12597:0:(lov_object.c:185:lov_init_sub())
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) } header@ffff880083424eb0
LustreError: 12597:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 12597:0:(lov_object.c:186:lov_init_sub()) header@ffff88027cf4d2e0[0x0, 1, [0x240000403:0x5da:0x0] hash]
LustreError: 12597:0:(lov_object.c:186:lov_init_sub()) try to own.
Lustre: 9093:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000404:0x3a2:0x0] with magic=0xbd60bd0
Lustre: 9093:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 15 previous similar messages
LustreError: 12913:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88009074b7e8: inode [0x240000404:0x6ba:0x0] mdc close failed: rc = -116
LustreError: 12913:0:(file.c:246:ll_close_inode_openhandle()) Skipped 27 previous similar messages
LustreError: 3642:0:(mdt_reint.c:2443:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000403:0x1:0x0]/14 failed: rc = -114
LustreError: 3642:0:(mdt_reint.c:2443:mdt_reint_migrate()) Skipped 41 previous similar messages
Lustre: 10973:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 618, rollback = 7
Lustre: 10973:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 5 previous similar messages
19[27046]: segfault at 8 ip 00007fda6aecb7e8 sp 00007fffca60a630 error 4 in ld-2.17.so[7fda6aec0000+22000]
Lustre: 19400:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 19400:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 650 previous similar messages
Lustre: 19400:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 19400:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 650 previous similar messages
Lustre: 10973:0:(osd_handler.c:1986:osd_trans_dump_creds()) write: 2/564/0, punch: 0/0/0, quota 1/3/0
Lustre: 10973:0:(osd_handler.c:1986:osd_trans_dump_creds()) Skipped 651 previous similar messages
Lustre: 10973:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10973:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 651 previous similar messages
Lustre: 10973:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10973:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 651 previous similar messages
LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188
LustreError: 28367:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0]
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}:
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
Lustre: 9098:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 508 < left 671, rollback = 2
Lustre: 9098:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 617 previous similar messages
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188
LustreError: 28367:0:(lov_object.c:185:lov_init_sub())
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460
LustreError: 28367:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 28367:0:(lov_object.c:186:lov_init_sub()) header@ffff88028fae9710[0x0, 1, [0x240000404:0x77d:0x0] hash]
LustreError: 28367:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 28367:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000404:0x77d:0x0]: rc = -5
LustreError: 28367:0:(lcommon_cl.c:196:cl_file_inode_init()) Skipped 14 previous similar messages
LustreError: 28367:0:(llite_lib.c:3611:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 28367:0:(llite_lib.c:3611:ll_prep_inode()) Skipped 14 previous similar messages
LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188
LustreError: 29749:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0]
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}:
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188
LustreError: 29749:0:(lov_object.c:185:lov_init_sub())
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460
LustreError: 29749:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 29749:0:(lov_object.c:186:lov_init_sub()) header@ffff880280eec868[0x0, 1, [0x240000404:0x77d:0x0] hash]
LustreError: 29749:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9088:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0002: '12' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 12' to finish migration.
LustreError: 9111:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0x7a1:0x0]: rc = -2
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff88008cc712e0[0x4, 2, [0x300000401:0xd0:0x0] hash]{
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88008cc71360[0]
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088ad5928id: 0x300000401:208 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448875 atime: 1699448875 ctime: 1699448875 blocks: 0
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff88008cc712e0
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845db890[0x4, 1, [0x240000403:0x777:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802845db910(0) inode: ffff880084fb7308 162129603815540599/37748740 100644 1 0 ffff8802845db910 [0x240000403:0x777:0x0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88008bd2eea0entries: 1, valid, lsm{ffff880289e9b608 0x0BD10BD0 1 0}:
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88008cc712e0[0x4, 2, [0x300000401:0xd0:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88008cc71360[0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088ad5928id: 0x300000401:208 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448875 atime: 1699448875 ctime: 1699448875 blocks: 0
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88008cc712e0
LustreError: 31257:0:(lov_object.c:185:lov_init_sub())
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845db890
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff8802a00bdf70[0x0, 1, [0x200000404:0x569:0x0] hash]
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188
LustreError: 31333:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0]
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}:
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188
LustreError: 31333:0:(lov_object.c:185:lov_init_sub())
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460
LustreError: 31333:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 31333:0:(lov_object.c:186:lov_init_sub()) header@ffff88028e6ae9e8[0x0, 1, [0x240000404:0x77d:0x0] hash]
LustreError: 31333:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff88027b22b460[0x4, 2, [0x340000400:0x79:0x0] hash]{
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027b22b4e0[0]
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fea68id: 0x340000400:121 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 26842 mtime: 1699448856 atime: 1699448853 ctime: 1699448856 blocks: 16
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027b22b460
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a4c6e18[0x4, 1, [0x280000404:0x4a7:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028a4c6e98(0) inode: ffff880084f39d88 180144002341799079/41943044 103675 1 0 ffff88028a4c6e98 [0x280000404:0x4a7:0x0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8802882570d8entries: 1, valid, lsm{ffff8800870aa1c8 0x0BD10BD0 1 0}:
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88027b22b460[0x4, 2, [0x340000400:0x79:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027b22b4e0[0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fea68id: 0x340000400:121 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 26842 mtime: 1699448856 atime: 1699448853 ctime: 1699448856 blocks: 16
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027b22b460
LustreError: 31257:0:(lov_object.c:185:lov_init_sub())
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a4c6e18
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff8802845d9928[0x0, 1, [0x240000404:0x78e:0x0] hash]
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028e402188
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8802990d3460[0x4, 1, [0x280000405:0x1c9:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802990d34e0(0) inode: ffff880084f85688 180144002358575561/41943044 100700 1 0 ffff8802990d34e0 [0x280000405:0x1c9:0x0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800872808e8entries: 1, valid, lsm{ffff880286b4f608 0x0BD10BD0 1 0}:
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028e402188[0x4, 2, [0x300000400:0x78:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028e402208[0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88026d2fb0e8id: 0x300000400:120 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448858 blocks: 0
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028e402188
LustreError: 31257:0:(lov_object.c:185:lov_init_sub())
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802990d3460
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff880293afe9e8[0x0, 1, [0x240000404:0x77d:0x0] hash]
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800bba184b8[0]
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800bba18438
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028faeb030[0x4, 1, [0x280000404:0x506:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028faeb0b0(0) inode: ffff8802671a8108 180144002341799174/41943044 106101 4 0 ffff88028faeb0b0 [0x280000404:0x506:0x0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880275ff2ea0entries: 1, valid, lsm{ffff880285604f38 0x0BD10BD0 1 0}:
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800bba184b8[0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800bba18438
LustreError: 31257:0:(lov_object.c:185:lov_init_sub())
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028faeb030
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff880280eedf70[0x0, 1, [0x200000404:0x586:0x0] hash]
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800bba184b8[0]
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800bba18438
LustreError: 31257:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff88028faeb030[0x4, 1, [0x280000404:0x506:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028faeb0b0(0) inode: ffff8802671a8108 180144002341799174/41943044 106101 4 0 ffff88028faeb0b0 [0x280000404:0x506:0x0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880275ff2ea0entries: 1, valid, lsm{ffff880285604f38 0x0BD10BD0 1 0}:
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800bba184b8[0]
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800bba18438
LustreError: 31257:0:(lov_object.c:185:lov_init_sub())
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028faeb030
LustreError: 31257:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) header@ffff8802845da9e8[0x0, 1, [0x200000404:0x586:0x0] hash]
LustreError: 31257:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 16876:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0001: '14' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 14' to finish migration.
Lustre: dir [0x280000405:0x246:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 8 previous similar messages
LustreError: 715:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 715:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 715:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 715:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0
LustreError: 715:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0]
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}:
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0
LustreError: 715:0:(lov_object.c:185:lov_init_sub())
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0
LustreError: 715:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 715:0:(lov_object.c:186:lov_init_sub()) header@ffff88028ef18650[0x0, 1, [0x280000404:0x5f9:0x0] hash]
LustreError: 715:0:(lov_object.c:186:lov_init_sub()) try to own.
Lustre: 16876:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x84f:0x0] with magic=0xbd60bd0
Lustre: 16876:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 15 previous similar messages
LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{
LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800bba184b8[0]
LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320
LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800bba18438
LustreError: 1213:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) header@ffff88028faeb030[0x4, 1, [0x280000404:0x506:0x0] hash]{
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028faeb0b0(0) inode: ffff8802671a8108 180144002341799174/41943044 106101 4 0 ffff88028faeb0b0 [0x280000404:0x506:0x0]
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880275ff2ea0entries: 1, valid, lsm{ffff880285604f38 0x0BD10BD0 1 0}:
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) header@ffff8800bba18438[0x4, 2, [0x380000400:0x7f:0x0] hash]{
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800bba184b8[0]
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce0e4a8id: 0x380000400:127 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 6305753 mtime: 1699448876 atime: 1699391508 ctime: 1699448876 blocks: 12320
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800bba18438
LustreError: 1213:0:(lov_object.c:185:lov_init_sub())
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028faeb030
LustreError: 1213:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 1213:0:(lov_object.c:186:lov_init_sub()) header@ffff8800807a2c00[0x4, 1, [0x200000404:0x586:0x0] hash]
LustreError: 1213:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 1213:0:(lov_object.c:1337:lov_layout_change()) lustre-clilov-ffff88009074b7e8: cannot apply new layout on [0x200000404:0x586:0x0] : rc = -5
LustreError: 1213:0:(vvp_io.c:1876:vvp_io_init()) lustre: refresh file layout [0x200000404:0x586:0x0] error -5.
LustreError: 6701:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x609:0x0]: rc = -2
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080962a68[0]
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800809629e8
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18eb0[0x4, 1, [0x200000403:0x55d:0x0] hash]{
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef18f30(0) inode: ffff880084ff6988 144115205306058077/33554436 100644 1 0 ffff88028ef18f30 [0x200000403:0x55d:0x0]
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880262c01638entries: 4, valid, lsm{ffff88028fb92448 0x0BD60BD0 1 4}:
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 65537, 0x1, 0, 0x10, 1, 65536 }
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080962a68[0]
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800809629e8
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 65538, 0x1, 65535, 0x0, 2, 2097152 }
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 131075, 0x1, 0, 0x10, 1, 65536 }
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e1b460[0x4, 1, [0x2c0000402:0x6f:0x0] hash]{
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e1b4e0[131072]
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc0b88id: 0x2c0000402:111 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448924 atime: 1699448924 ctime: 1699448924 blocks: 0
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e1b460
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 131076, 0x1, 65535, 0x0, 2, 2097152 }
LustreError: 3262:0:(lov_object.c:185:lov_init_sub())
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18eb0
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8c98[0x0, 1, [0x280000405:0x34b:0x0] hash]
LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 3262:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0]
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}:
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 3262:0:(lov_object.c:185:lov_init_sub())
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8
LustreError: 3262:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8c98[0x0, 1, [0x240000403:0x7d9:0x0] hash]
LustreError: 3262:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0
LustreError: 15379:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0]
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}:
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0
LustreError: 15379:0:(lov_object.c:185:lov_init_sub())
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0
LustreError: 15379:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15379:0:(lov_object.c:186:lov_init_sub()) header@ffff88028ef1a3a0[0x0, 1, [0x280000404:0x5f9:0x0] hash]
LustreError: 15379:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0
LustreError: 3389:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0]
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}:
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0
LustreError: 3389:0:(lov_object.c:185:lov_init_sub())
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0
LustreError: 3389:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 3389:0:(lov_object.c:186:lov_init_sub()) header@ffff8802845dae18[0x0, 1, [0x280000404:0x5f9:0x0] hash]
LustreError: 3389:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 562:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 562:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 562:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 562:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 562:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0]
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}:
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 562:0:(lov_object.c:185:lov_init_sub())
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8
LustreError: 562:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 562:0:(lov_object.c:186:lov_init_sub()) header@ffff880271ace188[0x0, 1, [0x240000403:0x7d9:0x0] hash]
LustreError: 562:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 562:0:(statahead.c:791:ll_statahead_interpret_work()) lustre: getattr callback for sleep [0x240000403:0x7d9:0x0]: rc = -5
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080962a68[0]
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800809629e8
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18eb0[0x4, 1, [0x200000403:0x55d:0x0] hash]{
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef18f30(0) inode: ffff880084ff6988 144115205306058077/33554436 100644 1 0 ffff88028ef18f30 [0x200000403:0x55d:0x0]
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880262c01638entries: 4, valid, lsm{ffff88028fb92448 0x0BD60BD0 1 4}:
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 65537, 0x1, 0, 0x10, 1, 65536 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080962a68[0]
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800809629e8
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 65538, 0x1, 65535, 0x0, 2, 2097152 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 131075, 0x1, 0, 0x10, 1, 65536 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e1b460[0x4, 1, [0x2c0000402:0x6f:0x0] hash]{
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e1b4e0[131072]
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc0b88id: 0x2c0000402:111 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448924 atime: 1699448924 ctime: 1699448924 blocks: 0
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e1b460
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 131076, 0x1, 65535, 0x0, 2, 2097152 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub())
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18eb0
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) header@ffff88028ef18438[0x0, 1, [0x280000405:0x34b:0x0] hash]
LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080962a68[0]
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) } header@ffff8800809629e8
LustreError: 4488:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18eb0[0x4, 1, [0x200000403:0x55d:0x0] hash]{
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef18f30(0) inode: ffff880084ff6988 144115205306058077/33554436 100644 1 0 ffff88028ef18f30 [0x200000403:0x55d:0x0]
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880262c01638entries: 4, valid, lsm{ffff88028fb92448 0x0BD60BD0 1 4}:
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 65537, 0x1, 0, 0x10, 1, 65536 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff8800809629e8[0x4, 2, [0x380000402:0x6c:0x0] hash]{
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080962a68[0]
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc78c8id: 0x380000402:108 idx: 3 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800809629e8
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 65538, 0x1, 65535, 0x0, 2, 2097152 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x50000): { 0x0BD10BD0, 131075, 0x1, 0, 0x10, 1, 65536 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e1b460[0x4, 1, [0x2c0000402:0x6f:0x0] hash]{
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e1b4e0[131072]
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc0b88id: 0x2c0000402:111 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448924 atime: 1699448924 ctime: 1699448924 blocks: 0
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e1b460
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) [0x50000, 0xffffffffffffffff): { 0x0BD10BD0, 131076, 0x1, 65535, 0x0, 2, 2097152 }
LustreError: 4488:0:(lov_object.c:185:lov_init_sub())
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18eb0
LustreError: 4488:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) header@ffff880271ace7d0[0x0, 1, [0x280000405:0x34b:0x0] hash]
LustreError: 4488:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 16656:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) lustre-MDD0000: '19' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 19' to finish migration.
LustreError: 16656:0:(mdd_dir.c:4378:mdd_migrate_cmd_check()) Skipped 1 previous similar message
Lustre: dir [0x200000404:0x64a:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
LustreError: 11035:0:(out_handler.c:886:out_tx_end()) lustre-MDT0000-osd: undo for /home/green/git/lustre-release/lustre/ptlrpc/../../lustre/target/out_handler.c:418: rc = -524
LustreError: 10364:0:(llog_cat.c:737:llog_cat_cancel_arr_rec()) lustre-MDT0000-osp-MDT0002: fail to cancel 1 llog-records: rc = -2
LustreError: 10364:0:(llog_cat.c:773:llog_cat_cancel_records()) lustre-MDT0000-osp-MDT0002: fail to cancel 1 of 1 llog-records: rc = -2
LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 8418:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0]
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}:
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 8418:0:(lov_object.c:185:lov_init_sub())
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8
LustreError: 8418:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8418:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7cc98[0x0, 1, [0x240000403:0x7d9:0x0] hash]
LustreError: 8418:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 8645:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) header@ffff880088b0d4f8[0x4, 1, [0x200000404:0x4f0:0x0] hash]{
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880088b0d578(0) inode: ffff8802671d5688 144115205322835184/33554436 100700 1 0 ffff880088b0d578 [0x200000404:0x4f0:0x0]
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280d75ce0entries: 1, valid, lsm{ffff880280fc2888 0x0BD10BD0 1 0}:
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) header@ffff88028f8e90c8[0x4, 2, [0x300000402:0x66:0x0] hash]{
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028f8e9148[0]
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027ce09fa8id: 0x300000402:102 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448864 blocks: 72
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028f8e90c8
LustreError: 8645:0:(lov_object.c:185:lov_init_sub())
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) } header@ffff880088b0d4f8
LustreError: 8645:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8645:0:(lov_object.c:186:lov_init_sub()) header@ffff8802822d3030[0x0, 1, [0x240000403:0x7d9:0x0] hash]
LustreError: 8645:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960eb0
LustreError: 9577:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bfcc0[0x4, 1, [0x240000403:0x7d3:0x0] hash]{
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bfd40(0) inode: ffff880267309408 162129603815540691/37748740 100644 1 0 ffff8802a00bfd40 [0x240000403:0x7d3:0x0]
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34f90entries: 1, valid, lsm{ffff880280fc1298 0x0BD10BD0 1 0}:
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960eb0[0x4, 2, [0x340000401:0xd9:0x0] hash]{
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880080960f30[0]
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880281b0c508id: 0x340000401:217 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 5 mtime: 1699448901 atime: 1699448907 ctime: 1699448901 blocks: 8
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960eb0
LustreError: 9577:0:(lov_object.c:185:lov_init_sub())
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bfcc0
LustreError: 9577:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 9577:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7dd58[0x0, 1, [0x280000404:0x5f9:0x0] hash]
LustreError: 9577:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d16a208[0]
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d16a188
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff880280adb890[0x4, 1, [0x280000405:0x2f3:0x0] hash]{
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280adb910(0) inode: ffff880262899d88 180144002358575859/41943044 100644 1 0 ffff880280adb910 [0x280000405:0x2f3:0x0]
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f36ea0entries: 4, valid, lsm{ffff88028c6c13f8 0x0BD60BD0 1 4}:
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d16a208[0]
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d16a188
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub())
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280adb890
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) header@ffff88028a5fd928[0x0, 1, [0x240000403:0x8ec:0x0] hash]
LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e19360[0]
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e192e0
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff880280eecc98[0x4, 1, [0x280000404:0x5c5:0x0] hash]{
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280eecd18(0) inode: ffff880267324388 180144002341799365/41943044 100644 1 0 ffff880280eecd18 [0x280000404:0x5c5:0x0]
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880279ccf9b8entries: 1, valid, lsm{ffff88028ac695f8 0x0BD10BD0 1 0}:
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e19360[0]
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e192e0
LustreError: 8812:0:(lov_object.c:185:lov_init_sub())
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280eecc98
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec23a0[0x0, 1, [0x200000403:0x65e:0x0] hash]
LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b6208[0]
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b6188
LustreError: 8812:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cee18[0x4, 1, [0x280000405:0x301:0x0] hash]{
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cee98(0) inode: ffff8802673b2708 180144002358575873/41943044 100644 1 0 ffff88027f6cee98 [0x280000405:0x301:0x0]
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e6139b8entries: 4, valid, lsm{ffff88008770a0a8 0x0BD60BD0 1 4}:
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b6208[0]
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b6188
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 }
LustreError: 8812:0:(lov_object.c:185:lov_init_sub())
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cee18
LustreError: 8812:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec1710[0x0, 1, [0x200000403:0x632:0x0] hash]
LustreError: 8812:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 6701:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000405:0x455:0x0]: rc = -2
LustreError: 9524:0:(file.c:246:ll_close_inode_openhandle()) lustre-clilmv-ffff88008fa0e678: inode [0x280000405:0x455:0x0] mdc close failed: rc = -2
LustreError: 9524:0:(file.c:246:ll_close_inode_openhandle()) Skipped 22 previous similar messages
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d16a208[0]
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d16a188
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff880280adb890[0x4, 1, [0x280000405:0x2f3:0x0] hash]{
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280adb910(0) inode: ffff880262899d88 180144002358575859/41943044 100644 1 0 ffff880280adb910 [0x280000405:0x2f3:0x0]
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f36ea0entries: 4, valid, lsm{ffff88028c6c13f8 0x0BD60BD0 1 4}:
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d16a208[0]
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d16a188
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub())
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280adb890
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7ec00[0x0, 1, [0x240000403:0x8ec:0x0] hash]
LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e19360[0]
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e192e0
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff880280eecc98[0x4, 1, [0x280000404:0x5c5:0x0] hash]{
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280eecd18(0) inode: ffff880267324388 180144002341799365/41943044 100644 1 0 ffff880280eecd18 [0x280000404:0x5c5:0x0]
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880279ccf9b8entries: 1, valid, lsm{ffff88028ac695f8 0x0BD10BD0 1 0}:
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e19360[0]
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e192e0
LustreError: 12659:0:(lov_object.c:185:lov_init_sub())
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280eecc98
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) header@ffff88026fa6b678[0x0, 1, [0x200000403:0x65e:0x0] hash]
LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b6208[0]
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b6188
LustreError: 12659:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cee18[0x4, 1, [0x280000405:0x301:0x0] hash]{
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cee98(0) inode: ffff8802673b2708 180144002358575873/41943044 100644 1 0 ffff88027f6cee98 [0x280000405:0x301:0x0]
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e6139b8entries: 4, valid, lsm{ffff88008770a0a8 0x0BD60BD0 1 4}:
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b6208[0]
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b6188
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 }
LustreError: 12659:0:(lov_object.c:185:lov_init_sub())
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cee18
LustreError: 12659:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec14f8[0x0, 1, [0x200000403:0x632:0x0] hash]
LustreError: 12659:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{
LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028a77a208[0]
LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72
LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028a77a188
LustreError: 13292:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a5fd0c8[0x4, 1, [0x280000405:0x46f:0x0] hash]{
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028a5fd148(0) inode: ffff8802629b7308 180144002358576239/41943044 100755 1 0 ffff88028a5fd148 [0x280000405:0x46f:0x0]
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880292ca3bf0entries: 1, valid, lsm{ffff88028761abe8 0x0BD10BD0 1 0}:
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028a77a208[0]
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a77a188
LustreError: 13292:0:(lov_object.c:185:lov_init_sub())
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a5fd0c8
LustreError: 13292:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 13292:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7e3a0[0x0, 1, [0x240000404:0xb40:0x0] hash]
LustreError: 13292:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890
LustreError: 14549:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0]
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}:
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890
LustreError: 14549:0:(lov_object.c:185:lov_init_sub())
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008
LustreError: 14549:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 14549:0:(lov_object.c:186:lov_init_sub()) header@ffff88028fb33248[0x0, 1, [0x280000405:0x427:0x0] hash]
LustreError: 14549:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) header@ffff880284e514f8[0x4, 2, [0x2c0000400:0xc1:0x0] hash]{
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880284e51578[0]
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880279fb7308id: 0x2c0000400:193 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) } header@ffff880284e514f8
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff8800807a3cc0[0x4, 1, [0x280000404:0x74e:0x0] hash]{
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8800807a3d40(0) inode: ffff880262a4b088 180144002341799758/41943044 100644 1 0 ffff8800807a3d40 [0x280000404:0x74e:0x0]
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e613548entries: 1, valid, lsm{ffff8800823f70f8 0x0BD10BD0 1 0}:
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff880284e514f8[0x4, 2, [0x2c0000400:0xc1:0x0] hash]{
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880284e51578[0]
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880279fb7308id: 0x2c0000400:193 idx: 0 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff880284e514f8
LustreError: 14588:0:(lov_object.c:185:lov_init_sub())
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff8800807a3cc0
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7e3a0[0x0, 1, [0x200000404:0x73a:0x0] hash]
LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88026d1c1578[0]
LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72
LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) } header@ffff88026d1c14f8
LustreError: 14531:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) header@ffff880280ad8650[0x4, 1, [0x280000405:0x57e:0x0] hash]{
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280ad86d0(0) inode: ffff880262b00108 180144002358576510/41943044 100755 1 0 ffff880280ad86d0 [0x280000405:0x57e:0x0]
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800877f1f18entries: 1, valid, lsm{ffff880296950a28 0x0BD10BD0 1 0}:
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88026d1c1578[0]
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) } header@ffff88026d1c14f8
LustreError: 14531:0:(lov_object.c:185:lov_init_sub())
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280ad8650
LustreError: 14531:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 14531:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7cc98[0x0, 1, [0x240000404:0xb97:0x0] hash]
LustreError: 14531:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88028a77a208[0]
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) } header@ffff88028a77a188
LustreError: 14588:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a5fd0c8[0x4, 1, [0x280000405:0x46f:0x0] hash]{
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028a5fd148(0) inode: ffff8802629b7308 180144002358576239/41943044 100755 1 0 ffff88028a5fd148 [0x280000405:0x46f:0x0]
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880292ca3bf0entries: 1, valid, lsm{ffff88028761abe8 0x0BD10BD0 1 0}:
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) header@ffff88028a77a188[0x4, 2, [0x300000400:0xbb:0x0] hash]{
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88028a77a208[0]
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027d159cc8id: 0x300000400:187 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448956 blocks: 72
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a77a188
LustreError: 14588:0:(lov_object.c:185:lov_init_sub())
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028a5fd0c8
LustreError: 14588:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8a80[0x0, 1, [0x240000404:0xb40:0x0] hash]
LustreError: 14588:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890
LustreError: 15691:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0]
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}:
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890
LustreError: 15691:0:(lov_object.c:185:lov_init_sub())
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008
LustreError: 15691:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15691:0:(lov_object.c:186:lov_init_sub()) header@ffff880088a40008[0x0, 1, [0x280000405:0x427:0x0] hash]
LustreError: 15691:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890
LustreError: 15587:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0]
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}:
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890
LustreError: 15587:0:(lov_object.c:185:lov_init_sub())
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008
LustreError: 15587:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15587:0:(lov_object.c:186:lov_init_sub()) header@ffff880088a43890[0x0, 1, [0x280000405:0x427:0x0] hash]
LustreError: 15587:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027d16a208[0]
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027d16a188
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff880280adb890[0x4, 1, [0x280000405:0x2f3:0x0] hash]{
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280adb910(0) inode: ffff880262899d88 180144002358575859/41943044 100644 1 0 ffff880280adb910 [0x280000405:0x2f3:0x0]
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f36ea0entries: 4, valid, lsm{ffff88028c6c13f8 0x0BD60BD0 1 4}:
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff88027d16a188[0x4, 2, [0x300000400:0x98:0x0] hash]{
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027d16a208[0]
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802dd478e68id: 0x300000400:152 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448940 atime: 1699448940 ctime: 1699448940 blocks: 0
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027d16a188
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 }
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b7890
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cc008[0x4, 1, [0x240000404:0x83a:0x0] hash]{
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cc088(0) inode: ffff8802673a9d88 162129603832318010/37748740 100700 1 0 ffff88027f6cc088 [0x240000404:0x83a:0x0]
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080735f18entries: 1, valid, lsm{ffff88008ac6c6c8 0x0BD10BD0 1 0}:
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b7890[0x4, 2, [0x300000401:0xe5:0x0] hash]{
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b7910[0]
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285fc6788id: 0x300000401:229 idx: 1 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448919 blocks: 72
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b7890
LustreError: 15625:0:(lov_object.c:185:lov_init_sub())
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cc008
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) header@ffff880080a7c220[0x0, 1, [0x280000405:0x427:0x0] hash]
LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) header@ffff880080960868[0x4, 2, [0x300000401:0xda:0x0] hash]{
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8800809608e8[0]
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff8802779b1fa8id: 0x300000401:218 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448892 atime: 1699448892 ctime: 1699448892 blocks: 0
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) } header@ffff880080960868
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff8802a00bc008[0x4, 1, [0x240000404:0x7dd:0x0] hash]{
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff8802a00bc088(0) inode: ffff880267308a88 162129603832317917/37748740 100644 1 0 ffff8802a00bc088 [0x240000404:0x7dd:0x0]
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880280f34d58entries: 1, valid, lsm{ffff880275d5d298 0x0BD10BD0 1 0}:
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff880080960868[0x4, 2, [0x300000401:0xda:0x0] hash]{
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8800809608e8[0]
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff8802779b1fa8id: 0x300000401:218 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448892 atime: 1699448892 ctime: 1699448892 blocks: 0
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff880080960868
LustreError: 15625:0:(lov_object.c:185:lov_init_sub())
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802a00bc008
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) header@ffff88026ee99f70[0x0, 1, [0x280000405:0x36a:0x0] hash]
LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15936:0:(lov_object.c:185:lov_init_sub())
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280adb890
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) header@ffff880263ec0868[0x0, 1, [0x240000403:0x8ec:0x0] hash]
LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff880275e19360[0]
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) } header@ffff880275e192e0
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff880280eecc98[0x4, 1, [0x280000404:0x5c5:0x0] hash]{
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280eecd18(0) inode: ffff880267324388 180144002341799365/41943044 100644 1 0 ffff880280eecd18 [0x280000404:0x5c5:0x0]
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880279ccf9b8entries: 1, valid, lsm{ffff88028ac695f8 0x0BD10BD0 1 0}:
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff880275e192e0[0x4, 2, [0x340000400:0x95:0x0] hash]{
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff880275e19360[0]
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e7164a8id: 0x340000400:149 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448898 atime: 1699448898 ctime: 1699448898 blocks: 0
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff880275e192e0
LustreError: 15936:0:(lov_object.c:185:lov_init_sub())
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280eecc98
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) header@ffff88026ee9b248[0x0, 1, [0x200000403:0x65e:0x0] hash]
LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff8802845b6208[0]
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) } header@ffff8802845b6188
LustreError: 15936:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f6cee18[0x4, 1, [0x280000405:0x301:0x0] hash]{
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88027f6cee98(0) inode: ffff8802673b2708 180144002358575873/41943044 100644 1 0 ffff88027f6cee98 [0x280000405:0x301:0x0]
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026e6139b8entries: 4, valid, lsm{ffff88008770a0a8 0x0BD60BD0 1 4}:
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x0, 0x4000000): { 0x0BD10BD0, 1, 0x1, 0, 0x10, 1, 262144 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) header@ffff8802845b6188[0x4, 2, [0x300000400:0x9b:0x0] hash]{
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff8802845b6208[0]
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880285c71148id: 0x300000400:155 idx: 1 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448920 atime: 1699448920 ctime: 1699448920 blocks: 0
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff8802845b6188
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x4000000, 0x8000000): { 0x0BD10BD0, 2, 0x1, 65535, 0x40, 1, 65536 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0x8000000): { 0x0BD10BD0, 3, 0x1, 65535, 0x0, 4, 2097152 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) [0x8000000, 0xffffffffffffffff): { 0x0BD10BD0, 4, 0x1, 65535, 0x40, 4, 131072 }
LustreError: 15936:0:(lov_object.c:185:lov_init_sub())
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f6cee18
LustreError: 15936:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) header@ffff880271acf678[0x0, 1, [0x200000403:0x632:0x0] hash]
LustreError: 15936:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{
LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88026d1c1578[0]
LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72
LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) } header@ffff88026d1c14f8
LustreError: 16216:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) header@ffff880280ad8650[0x4, 1, [0x280000405:0x57e:0x0] hash]{
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880280ad86d0(0) inode: ffff880262b00108 180144002358576510/41943044 100755 1 0 ffff880280ad86d0 [0x280000405:0x57e:0x0]
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff8800877f1f18entries: 1, valid, lsm{ffff880296950a28 0x0BD10BD0 1 0}:
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) header@ffff88026d1c14f8[0x4, 2, [0x340000400:0xda:0x0] hash]{
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88026d1c1578[0]
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88008e715368id: 0x340000400:218 idx: 2 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448978 blocks: 72
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) } header@ffff88026d1c14f8
LustreError: 16216:0:(lov_object.c:185:lov_init_sub())
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) } header@ffff880280ad8650
LustreError: 16216:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 16216:0:(lov_object.c:186:lov_init_sub()) header@ffff880280ad8c98[0x0, 1, [0x240000404:0xb97:0x0] hash]
LustreError: 16216:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) header@ffff88026eebf678[0x4, 2, [0x340000401:0xd8:0x0] hash]{
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88026eebf6f8[0]
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff880088ad6788id: 0x340000401:216 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448925 atime: 1699448925 ctime: 1699448925 blocks: 0
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) } header@ffff88026eebf678
LustreError: 15625:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff88028ef18650[0x4, 1, [0x240000403:0x7c6:0x0] hash]{
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff88028ef186d0(0) inode: ffff8802673bba08 162129603815540678/37748740 100644 1 0 ffff88028ef186d0 [0x240000403:0x7c6:0x0]
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff880080734008entries: 1, valid, lsm{ffff880286b4fcc8 0x0BD10BD0 1 0}:
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) header@ffff88026eebf678[0x4, 2, [0x340000401:0xd8:0x0] hash]{
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88026eebf6f8[0]
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff880088ad6788id: 0x340000401:216 idx: 2 gen: 0 kms_valid: 0 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 0 mtime: 1699448925 atime: 1699448925 ctime: 1699448925 blocks: 0
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff88026eebf678
LustreError: 15625:0:(lov_object.c:185:lov_init_sub())
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) } header@ffff88028ef18650
LustreError: 15625:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) header@ffff88028da1f030[0x0, 1, [0x280000405:0x37c:0x0] hash]
LustreError: 15625:0:(lov_object.c:186:lov_init_sub()) try to own.
LustreError: 9113:0:(mdd_object.c:3621:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x6da:0x0]: rc = -2
LustreError: 9113:0:(mdd_object.c:3621:mdd_close()) Skipped 1 previous similar message
Lustre: dir [0x240000404:0xc1f:0x0] stripe 3 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 7 previous similar messages
LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) header@ffff88027f589928[0x4, 2, [0x380000400:0xfa:0x0] hash]{
LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) ....lovsub@ffff88027f5899a8[0]
LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) ....osc@ffff88027e13fba8id: 0x380000400:250 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448999 blocks: 72
LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) } header@ffff88027f589928
LustreError: 18442:0:(lov_object.c:184:lov_init_sub()) stripe 0 is already owned.
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) header@ffff880271c729e8[0x4, 1, [0x280000405:0x647:0x0] hash]{
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....vvp@ffff880271c72a68(0) inode: ffff88025ec76988 180144002358576711/41943044 100700 1 0 ffff880271c72a68 [0x280000405:0x647:0x0]
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....lov@ffff88026efba7f8entries: 1, valid, lsm{ffff88008a5a3b18 0x0BD10BD0 1 0}:
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 4194304 }
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) header@ffff88027f589928[0x4, 2, [0x380000400:0xfa:0x0] hash]{
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....lovsub@ffff88027f5899a8[0]
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) ....osc@ffff88027e13fba8id: 0x380000400:250 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1699391508 ctime: 1699448999 blocks: 72
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) } header@ffff88027f589928
LustreError: 18442:0:(lov_object.c:185:lov_init_sub())
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) } header@ffff880271c729e8
LustreError: 18442:0:(lov_object.c:185:lov_init_sub()) owned.
LustreError: 18442:0:(lov_object.c:186:lov_init_sub()) header@ffff8802822d3cc0[0x0, 1, [0x200000404:0x7bc:0x0] hash]
LustreError: 18442:0:(lov_object.c:186:lov_init_sub()) try to own.
ptlrpc_watchdog_fire: 1 callbacks suppressed
Lustre: ll_ost01_002: service thread pid 10907 was inactive for 40.094 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Lustre: Skipped 2 previous similar messages
Pid: 10907, comm: ll_ost01_002 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x963/0xe80 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] tgt_extent_lock+0xea/0x2a0 [ptlrpc]
[<0>] ofd_getattr_hdl+0x385/0x750 [ofd]
[<0>] tgt_request_handle+0x73e/0x19b0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x262/0xcd0 [ptlrpc]
[<0>] ptlrpc_main+0xc37/0x16d0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: 10976:0:(osd_internal.h:1357:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
Lustre: 10976:0:(osd_internal.h:1357:osd_trans_exec_op()) Skipped 63 previous similar messages
Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000404:0x97f:0x0] with magic=0xbd60bd0
Lustre: 16388:0:(lod_lov.c:1422:lod_parse_striping()) Skipped 21 previous similar messages
LustreError: 9112:0:(llog_osd.c:624:llog_osd_write_rec()) lustre-MDT0000-osp-MDT0002: index 1259 already set in llog bitmap [0x200000402:0x2:0x0]

LustreError: 19981:0:(llog_osd.c:630:llog_osd_write_rec()) LBUG
Pid: 19981, comm: mdt_rdpg06_001 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] libcfs_call_trace+0x90/0xf0 [libcfs]
[<0>] lbug_with_loc+0x4c/0xa0 [libcfs]
[<0>] llog_osd_write_rec+0x1b54/0x1b60 [obdclass]
[<0>] llog_write_rec+0x290/0x590 [obdclass]
[<0>] llog_cat_add_rec+0x1e1/0x990 [obdclass]
[<0>] llog_add+0x17f/0x1f0 [obdclass]
[<0>] sub_updates_write+0x303/0xe3e [ptlrpc]
[<0>] top_trans_stop+0x49a/0xfb0 [ptlrpc]
[<0>] lod_trans_stop+0x25c/0x340 [lod]
[<0>] mdd_trans_stop+0x28/0x16e [mdd]
[<0>] mdd_attr_set+0x897/0x1200 [mdd]
[<0>] mdt_mfd_close+0x76e/0xbb0 [mdt]
[<0>] mdt_close_internal+0xb4/0x240 [mdt]
[<0>] mdt_close+0x291/0x9a0 [mdt]
[<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc]
[<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe

Lustre: 19949:0:(mdt_recovery.c:150:mdt_req_from_lrd()) @@@ restoring transno req@ffff88009bf05b40 x1756624771794368/t4294968090(0) o101->61984978-aa7d-4c6a-adf8-385000fbd621@0@lo:398/0 lens 400/816 e 0 to 0 dl 1675248003 ref 1 fl Interpret:H/2/0 rc 0/0 job:'cat.0'
LustreError: 27984:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000404:0x9e:0x0] mdc close failed: rc = -116
LustreError: 27984:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000404:0x9e:0x0] mdc close failed: rc = -116
Lustre: 21934:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 511 < left 1133, rollback = 2
Lustre: 21934:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/1, destroy: 1/4/0
Lustre: 21934:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 12/1133/0
Lustre: 21934:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 7/105/0
Lustre: 21934:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 13/251/2, delete: 2/5/1
Lustre: 21934:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 1/1/0
LustreError: 19977:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x1:0x0]: rc = -2
LustreError: 28983:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x240000404:0x25:0x0] mdc close failed: rc = -116
LustreError: 28983:0:(file.c:242:ll_close_inode_openhandle()) Skipped 2 previous similar messages
LustreError: 28785:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880322108958: inode [0x280000404:0x9e:0x0] mdc close failed: rc = -116
LustreError: 28785:0:(file.c:242:ll_close_inode_openhandle()) Skipped 2 previous similar messages
Lustre: 22802:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 512 < left 805, rollback = 2
Lustre: 22802:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 14 previous similar messages
Lustre: 22802:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 2/8/0, destroy: 1/4/0
Lustre: 22802:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 22802:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 10/805/0
Lustre: 22802:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 22802:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 1/10/0, punch: 0/0/0, quota 1/3/0
Lustre: 22802:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 22802:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 12/194/2, delete: 3/6/1
Lustre: 22802:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 22802:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 3/3/0
Lustre: 22802:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 23877:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 506 < left 671, rollback = 2
Lustre: 23877:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 14 previous similar messages
Lustre: 23877:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/5, destroy: 0/0/0
Lustre: 23877:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 23877:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0
Lustre: 23877:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 23877:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0
Lustre: 23877:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 23877:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/4, delete: 0/0/0
Lustre: 23877:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 23877:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
Lustre: 23877:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 14 previous similar messages
Lustre: 19954:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 502 < left 955, rollback = 2
Lustre: 19954:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 7 previous similar messages
Lustre: 19954:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/8, destroy: 1/4/0
Lustre: 19954:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 19954:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 10/955/0
Lustre: 19954:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 19954:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 1/3/0
Lustre: 19954:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 19954:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 14/287/4, delete: 2/5/1
Lustre: 19954:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 7 previous similar messages
Lustre: 19954:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 7/7/0, ref_del: 1/1/0
Lustre: 19954:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 7 previous similar messages
LustreError: 29192:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0xf7:0x0]: rc = -2
LustreError: 32063:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880322108958: inode [0x240000403:0xf7:0x0] mdc close failed: rc = -2
LustreError: 32063:0:(file.c:242:ll_close_inode_openhandle()) Skipped 1 previous similar message
LustreError: 29526:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '17' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 17' to finish migration.
LustreError: 19979:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000404:0xae:0x0]: rc = -2
Lustre: 19956:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 509 < left 582, rollback = 2
Lustre: 19956:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 78 previous similar messages
Lustre: 19956:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/3, destroy: 0/0/0
Lustre: 19956:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 78 previous similar messages
Lustre: 19956:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 6/582/0
Lustre: 19956:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 78 previous similar messages
Lustre: 19956:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0
Lustre: 19956:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 78 previous similar messages
Lustre: 19956:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/3, delete: 0/0/0
Lustre: 19956:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 78 previous similar messages
Lustre: 19956:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
Lustre: 19956:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 78 previous similar messages
Lustre: 20791:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
LustreError: 1933:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000403:0x1b5:0x0] mdc close failed: rc = -116
LustreError: 1933:0:(file.c:242:ll_close_inode_openhandle()) Skipped 2 previous similar messages
Lustre: dir [0x200000404:0x3e:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: 29526:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 504 < left 671, rollback = 2
Lustre: 29526:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 62 previous similar messages
Lustre: 29526:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/8, destroy: 0/0/0
Lustre: 29526:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 64 previous similar messages
Lustre: 29526:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 7/671/0
Lustre: 29526:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 64 previous similar messages
Lustre: 29526:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0
Lustre: 29526:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 64 previous similar messages
Lustre: 29526:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/3, delete: 0/0/0
Lustre: 29526:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 64 previous similar messages
Lustre: 29526:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
Lustre: 29526:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 64 previous similar messages
Lustre: dir [0x280000403:0x1cf:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
Lustre: 19947:0:(mdt_recovery.c:150:mdt_req_from_lrd()) @@@ restoring transno req@ffff8802c155a740 x1756624777229056/t4294972655(0) o101->12ddc59d-106f-448a-9d1b-b5d57dd1e845@0@lo:499/0 lens 376/816 e 0 to 0 dl 1675248104 ref 1 fl Interpret:H/2/0 rc 0/0 job:'dd.0'
Lustre: dir [0x200000403:0x63:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: 23449:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
Lustre: 23449:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 1 previous similar message
Lustre: 23874:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 618, rollback = 7
Lustre: 23874:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 3 previous similar messages
LustreError: 6447:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x48:0x0]: rc = -2
LustreError: 7180:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x200000404:0x48:0x0] mdc close failed: rc = -2
LustreError: 7180:0:(file.c:242:ll_close_inode_openhandle()) Skipped 1 previous similar message
Lustre: 20773:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20773:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 3 previous similar messages
Lustre: 19960:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 499 < left 877, rollback = 2
Lustre: 19960:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 128 previous similar messages
Lustre: 19960:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/12, destroy: 0/0/0
Lustre: 19960:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 138 previous similar messages
Lustre: 19960:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 2/2/0, xattr_set: 11/877/0
Lustre: 19960:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 138 previous similar messages
Lustre: 19960:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0
Lustre: 19960:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 138 previous similar messages
Lustre: 19960:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 12/215/4, delete: 1/1/0
Lustre: 19960:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 138 previous similar messages
Lustre: 19960:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 6/6/0, ref_del: 1/1/0
Lustre: 19960:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 138 previous similar messages
Lustre: 20769:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20769:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 1 previous similar message
Lustre: 19948:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x232:0x0] with magic=0xbd60bd0
Lustre: 25133:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x223:0x0] with magic=0xbd60bd0
Lustre: 25133:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 5 previous similar messages
Lustre: 19965:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000403:0x247:0x0] with magic=0xbd60bd0
Lustre: 19965:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 7 previous similar messages
LustreError: 19973:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000404:0x286:0x0]: rc = -2
LustreError: 19973:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message
4[12413]: segfault at 8 ip 00007fddfd3907e8 sp 00007ffff7d6c7e0 error 4 in ld-2.17.so[7fddfd385000+22000]
Lustre: 718:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000404:0x29a:0x0] with magic=0xbd60bd0
Lustre: 718:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 1 previous similar message
Lustre: 13413:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
Lustre: 13413:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 1 previous similar message
9[14785]: segfault at 8 ip 00007f1d2e93c7e8 sp 00007ffdb5053d20 error 4 in ld-2.17.so[7f1d2e931000+22000]
LustreError: 24027:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0x36b:0x0]: rc = -2
LustreError: 24027:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message
LustreError: 19948:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '14' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 14' to finish migration.
Lustre: 26533:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 501 < left 1116, rollback = 2
Lustre: 26533:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 363 previous similar messages
Lustre: 26533:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/10, destroy: 0/0/0
Lustre: 26533:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 369 previous similar messages
Lustre: 26533:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 12/1116/0
Lustre: 26533:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 369 previous similar messages
Lustre: 26533:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0
Lustre: 26533:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 369 previous similar messages
Lustre: 26533:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/4, delete: 0/0/0
Lustre: 26533:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 369 previous similar messages
Lustre: 26533:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
Lustre: 26533:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 369 previous similar messages
LustreError: 13527:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0001: '6' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 6' to finish migration.
LustreError: 22826:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '14' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 14' to finish migration.
Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x1a9:0x0] with magic=0xbd60bd0
Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 1 previous similar message
LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{
LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) ....lovsub@ffff8802b044e208[0]
LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72
LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) } header@ffff8802b044e188
LustreError: 19016:0:(lov_object.c:190:lov_init_sub()) stripe 0 is already owned.
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) header@ffff880098dc4220[0x4, 1, [0x240000403:0x1c6:0x0] hash]{
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....vvp@ffff880098dc42a0(0) inode: ffff88023bf641d8 162129603815539142/37748740 100755 1 0 ffff880098dc42a0 [0x240000403:0x1c6:0x0]
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....lov@ffff8802c2753780entries: 1, valid, lsm{ffff88008b434a28 0x0BD10BD0 1 0}:
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 1048576 }
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....lovsub@ffff8802b044e208[0]
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) } header@ffff8802b044e188
LustreError: 19016:0:(lov_object.c:191:lov_init_sub())
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) } header@ffff880098dc4220
LustreError: 19016:0:(lov_object.c:191:lov_init_sub()) owned.
LustreError: 19016:0:(lov_object.c:192:lov_init_sub()) header@ffff8802ad033aa8[0x0, 1, [0x200000404:0x179:0x0] hash]
LustreError: 19016:0:(lov_object.c:192:lov_init_sub()) try to own.
LustreError: 19016:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x179:0x0]: rc = -5
LustreError: 19016:0:(llite_lib.c:3299:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 19480:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880322108958: inode [0x280000403:0x403:0x0] mdc close failed: rc = -116
LustreError: 19480:0:(file.c:242:ll_close_inode_openhandle()) Skipped 12 previous similar messages
Lustre: 20790:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20790:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 5 previous similar messages
Lustre: dir [0x200000403:0x1a1:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 4 previous similar messages
LustreError: 25390:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0000: '17' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 17' to finish migration.
Lustre: ll_ost05_003: service thread pid 23694 was inactive for 62.227 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Pid: 23694, comm: ll_ost05_003 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] tgt_extent_lock+0xea/0x2a0 [ptlrpc]
[<0>] ofd_getattr_hdl+0x385/0x750 [ofd]
[<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc]
[<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
LustreError: 19970:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0001: failed to get lu_attr of [0x240000403:0x379:0x0]: rc = -2
LustreError: 19970:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message
LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{
LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) ....lovsub@ffff8802b044e208[0]
LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72
LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) } header@ffff8802b044e188
LustreError: 23192:0:(lov_object.c:190:lov_init_sub()) stripe 0 is already owned.
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) header@ffff880098dc4220[0x4, 1, [0x240000403:0x1c6:0x0] hash]{
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....vvp@ffff880098dc42a0(0) inode: ffff88023bf641d8 162129603815539142/37748740 100755 1 0 ffff880098dc42a0 [0x240000403:0x1c6:0x0]
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....lov@ffff8802c2753780entries: 1, valid, lsm{ffff88008b434a28 0x0BD10BD0 1 0}:
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) [0x0, 0xffffffffffffffff): { 0x0BD10BD0, 0, 0x1, 0, 0x10, 1, 1048576 }
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) header@ffff8802b044e188[0x4, 2, [0x380000401:0x39:0x0] hash]{
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....lovsub@ffff8802b044e208[0]
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) ....osc@ffff8802ca01d368id: 0x380000401:57 idx: 3 gen: 0 kms_valid: 1 kms 0 rc: 0 force_sync: 0 min_xid: 0 size: 33128 mtime: 1605565498 atime: 1675240483 ctime: 1675248079 blocks: 72
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) } header@ffff8802b044e188
LustreError: 23192:0:(lov_object.c:191:lov_init_sub())
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) } header@ffff880098dc4220
LustreError: 23192:0:(lov_object.c:191:lov_init_sub()) owned.
LustreError: 23192:0:(lov_object.c:192:lov_init_sub()) header@ffff8800994bc650[0x0, 1, [0x200000404:0x179:0x0] hash]
LustreError: 23192:0:(lov_object.c:192:lov_init_sub()) try to own.
LustreError: 23192:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000404:0x179:0x0]: rc = -5
LustreError: 23192:0:(llite_lib.c:3299:ll_prep_inode()) new_inode -fatal: rc -5
LustreError: 23008:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0001: '17' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush 17' to finish migration.
LustreError: 23008:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 1 previous similar message
Lustre: dir [0x200000404:0x20f:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
2[26717]: segfault at 8 ip 00007f93b1f157e8 sp 00007ffcaa4ff550 error 4 in ld-2.17.so[7f93b1f0a000+22000]
3[26973]: segfault at 8 ip 00007f73cb2427e8 sp 00007ffe34d64370 error 4 in ld-2.17.so[7f73cb237000+22000]
Lustre: dir [0x200000404:0x2a3:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
4[28052]: segfault at 8 ip 00007fa990fac7e8 sp 00007fff53409370 error 4 in ld-2.17.so[7fa990fa1000+22000]
LustreError: 19951:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '12' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 12' to finish migration.
Lustre: 20787:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20787:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 11 previous similar messages
LustreError: 4218:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000404:0x35c:0x0]: rc = -2
LustreError: 4218:0:(mdd_object.c:3458:mdd_close()) Skipped 2 previous similar messages
Lustre: dir [0x240000403:0x628:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 3 previous similar messages
LustreError: 19937:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: filter-lustre-OST0000_UUID lock: ffff8802c72ed2c0/0x8c82fea2730d0c21 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 0@lo remote: 0x8c82fea2730d0c0c expref: 30 pid: 2182 timeout: 77798 lvb_type: 0
LustreError: 11-0: lustre-OST0000-osc-ffff880322108958: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-OST0000-osc-ffff880322108958: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 167-0: lustre-OST0000-osc-ffff880322108958: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x280000404:0x9bc:0x0]// may get corrupted (rc -108)
Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x280000404:0xa19:0x0]// may get corrupted (rc -108)
LustreError: 1050:0:(ldlm_resource.c:1124:ldlm_resource_complain()) lustre-OST0000-osc-ffff880322108958: namespace resource [0x2c0000400:0xd3:0x0].0x0 (ffff8802de1a0cc0) refcount nonzero (1) after lock cleanup; forcing cleanup.
Lustre: mdt07_003: service thread pid 23101 was inactive for 62.068 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Pid: 23101, comm: mdt07_003 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] mdt_object_local_lock+0x52f/0xba0 [mdt]
[<0>] mdt_object_lock_internal+0x70/0x390 [mdt]
[<0>] mdt_reint_object_lock+0x2c/0x60 [mdt]
[<0>] mdt_reint_striped_lock+0x89/0x6a0 [mdt]
[<0>] mdt_attr_set+0xb0/0x910 [mdt]
[<0>] mdt_reint_setattr+0x71a/0x1240 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x76c/0xba0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc]
[<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Pid: 28540, comm: mdt01_010 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] mdt_object_local_lock+0x52f/0xba0 [mdt]
[<0>] mdt_object_lock_internal+0x70/0x390 [mdt]
[<0>] mdt_reint_object_lock+0x2c/0x60 [mdt]
[<0>] mdt_reint_striped_lock+0x89/0x6a0 [mdt]
[<0>] mdt_attr_set+0xb0/0x910 [mdt]
[<0>] mdt_reint_setattr+0x71a/0x1240 [mdt]
[<0>] mdt_reint_rec+0x87/0x240 [mdt]
[<0>] mdt_reint_internal+0x76c/0xba0 [mdt]
[<0>] mdt_reint+0x67/0x150 [mdt]
[<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc]
[<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: mdt06_003: service thread pid 22833 was inactive for 62.211 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: lustre-OST0000: haven't heard from client 61984978-aa7d-4c6a-adf8-385000fbd621 (at 0@lo) in 32 seconds. I think it's dead, and I am evicting it. exp ffff8802a2e16678, cur 1675248188 expire 1675248158 last 1675248156
LustreError: 19937:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 0@lo ns: mdt-lustre-MDT0001_UUID lock: ffff88008ce00040/0x8c82fea2731808b5 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x286:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0x8c82fea2731808a7 expref: 249 pid: 30655 timeout: 77837 lvb_type: 0
LustreError: 11-0: lustre-MDT0001-mdc-ffff880098fe92a8: operation mds_close to node 0@lo failed: rc = -107
Lustre: lustre-MDT0001-mdc-ffff880098fe92a8: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 167-0: lustre-MDT0001-mdc-ffff880098fe92a8: This client was evicted by lustre-MDT0001; in progress operations using this service will fail.
LustreError: 31838:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x240000403:0x660:0x0] mdc close failed: rc = -108
LustreError: 31838:0:(file.c:242:ll_close_inode_openhandle()) Skipped 14 previous similar messages
LustreError: 32652:0:(llite_lib.c:3299:ll_prep_inode()) new_inode -fatal: rc -2
LustreError: 30669:0:(mdc_request.c:1474:mdc_read_page()) lustre-MDT0001-mdc-ffff880098fe92a8: [0x240000403:0x5f5:0x0] lock enqueue fails: rc = -108
LustreError: 32425:0:(llite_lib.c:1959:ll_md_setattr()) md_setattr fails: rc = -108
Lustre: lustre-MDT0001-mdc-ffff880098fe92a8: Connection restored to (at 0@lo)
Lustre: mdt05_011: service thread pid 8426 was inactive for 40.062 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
Lustre: 25090:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 501 < left 1027, rollback = 2
Lustre: 25090:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 782 previous similar messages
Lustre: 25090:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 3/12/10, destroy: 0/0/0
Lustre: 25090:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 800 previous similar messages
Lustre: 25090:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 11/1027/0
Lustre: 25090:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 800 previous similar messages
Lustre: 25090:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 5/136/0, punch: 0/0/0, quota 4/54/0
Lustre: 25090:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 800 previous similar messages
Lustre: 25090:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 10/203/4, delete: 0/0/0
Lustre: 25090:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 800 previous similar messages
Lustre: 25090:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0
Lustre: 25090:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 800 previous similar messages
Lustre: lustre-OST0000-osc-ffff880322108958: Connection restored to (at 0@lo)
LustreError: 11-0: lustre-OST0000-osc-ffff880322108958: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-OST0000-osc-ffff880322108958: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 167-0: lustre-OST0000-osc-ffff880322108958: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
2[9003]: segfault at 0 ip (null) sp 00007ffdf03b5d38 error 14 in 2[400000+6000]
Lustre: dir [0x280000404:0x822:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: 19948:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0002-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x280000403:0x909:0x0] with magic=0xbd60bd0
Lustre: 19948:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 29 previous similar messages
Lustre: 20782:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7
Lustre: 20782:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 15 previous similar messages
LustreError: 22791:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '9' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 9' to finish migration.
LustreError: 19979:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0xa15:0x0]: rc = -2
LustreError: 19979:0:(mdd_object.c:3458:mdd_close()) Skipped 1 previous similar message
Lustre: dir [0x200000404:0x53c:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 14 previous similar messages
LustreError: 29526:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0001: '11' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush 11' to finish migration.
LustreError: 29526:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 9 previous similar messages
Lustre: 22872:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x414:0x0] with magic=0xbd60bd0
Lustre: 22872:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 23 previous similar messages
13[30765]: segfault at 8 ip 00007fe3727e97e8 sp 00007ffceba597d0 error 4 in ld-2.17.so[7fe3727de000+22000]
19[30568]: segfault at 8 ip 00007fdad6ff77e8 sp 00007ffc2d500850 error 4 in ld-2.17.so[7fdad6fec000+22000]
Lustre: 16533:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 16533:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 850 previous similar messages
Lustre: 16533:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 16533:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 850 previous similar messages
Lustre: 16533:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 4/150/0
Lustre: 16533:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 850 previous similar messages
Lustre: 16533:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 16533:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 850 previous similar messages
Lustre: 16533:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 16533:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 850 previous similar messages
Lustre: 19958:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000404:0x6da:0x0] with magic=0xbd60bd0
Lustre: 19958:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 13 previous similar messages
Lustre: 20809:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 508 < left 728, rollback = 2
Lustre: 20809:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 762 previous similar messages
LustreError: 394:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x280000403:0xd60:0x0] mdc close failed: rc = -2
LustreError: 394:0:(file.c:242:ll_close_inode_openhandle()) Skipped 28 previous similar messages
18[2874]: segfault at 8 ip 00007f2cc9b6b7e8 sp 00007ffc4eed7730 error 4 in ld-2.17.so[7f2cc9b60000+22000]
Lustre: 23112:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 515 < left 558, rollback = 7
Lustre: 23112:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 105 previous similar messages
9[7688]: segfault at 8 ip 00007f72b4e9d7e8 sp 00007ffd8cc0b8d0 error 4 in ld-2.17.so[7f72b4e92000+22000]
9[14105]: segfault at 8 ip 00007fd1bd9767e8 sp 00007ffdadd01120 error 4 in ld-2.17.so[7fd1bd96b000+22000]
Lustre: 24476:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0000-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x200000403:0x4c5:0x0] with magic=0xbd60bd0
Lustre: 24476:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 9 previous similar messages
LustreError: 19968:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200000403:0x544:0x0]: rc = -2
LustreError: 19968:0:(mdd_object.c:3458:mdd_close()) Skipped 7 previous similar messages
Lustre: dir [0x280000403:0x12ec:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 13 previous similar messages
LustreError: 19944:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0000: '15' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 15' to finish migration.
LustreError: 19944:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 12 previous similar messages
7[25140]: segfault at 8 ip 00007fc4177427e8 sp 00007ffd09483e30 error 4 in ld-2.17.so[7fc417737000+22000]
17[27974]: segfault at 0 ip 0000000000403e5f sp 00007ffeb7244300 error 6 in 17[400000+6000]
LustreError: 20269:0:(llog_cat.c:739:llog_cat_cancel_arr_rec()) lustre-MDT0002-osp-MDT0001: fail to cancel 1 llog-records: rc = -116
LustreError: 20269:0:(llog_cat.c:775:llog_cat_cancel_records()) lustre-MDT0002-osp-MDT0001: fail to cancel 1 of 1 llog-records: rc = -116
LustreError: 20269:0:(llog_cat.c:739:llog_cat_cancel_arr_rec()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 llog-records: rc = -116
LustreError: 20269:0:(llog_cat.c:739:llog_cat_cancel_arr_rec()) Skipped 4 previous similar messages
LustreError: 20269:0:(llog_cat.c:775:llog_cat_cancel_records()) lustre-MDT0000-osp-MDT0001: fail to cancel 1 of 1 llog-records: rc = -116
LustreError: 20269:0:(llog_cat.c:775:llog_cat_cancel_records()) Skipped 4 previous similar messages
11[12593]: segfault at 8 ip 00007f1cf34127e8 sp 00007ffc57803f00 error 4 in ld-2.17.so[7f1cf3407000+22000]
10[25812]: segfault at 0 ip (null) sp 00007fff0649a048 error 14 in 10[400000+6000]
LustreError: 2798:0:(mdd_object.c:403:mdd_xattr_get()) lustre-MDD0000: object [0x200000404:0xdb4:0x0] not found: rc = -2
ptlrpc_watchdog_fire: 2 callbacks suppressed
Lustre: ll_ost04_009: service thread pid 9443 was inactive for 62.132 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
Lustre: Skipped 1 previous similar message
Pid: 9443, comm: ll_ost04_009 3.10.0-7.9-debug #2 SMP Tue Feb 1 18:17:58 EST 2022
Call Trace:
[<0>] ldlm_completion_ast+0x7f7/0xa50 [ptlrpc]
[<0>] ldlm_cli_enqueue_local+0x259/0x880 [ptlrpc]
[<0>] tgt_extent_lock+0xea/0x2a0 [ptlrpc]
[<0>] ofd_getattr_hdl+0x385/0x750 [ofd]
[<0>] tgt_request_handle+0x93a/0x19c0 [ptlrpc]
[<0>] ptlrpc_server_handle_request+0x251/0xc00 [ptlrpc]
[<0>] ptlrpc_main+0xbd1/0x15e0 [ptlrpc]
[<0>] kthread+0xe4/0xf0
[<0>] ret_from_fork_nospec_begin+0x7/0x21
[<0>] 0xfffffffffffffffe
Lustre: 24327:0:(osd_handler.c:1876:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 24327:0:(osd_handler.c:1876:osd_trans_dump_creds()) Skipped 2646 previous similar messages
Lustre: 24327:0:(osd_handler.c:1883:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 24327:0:(osd_handler.c:1883:osd_trans_dump_creds()) Skipped 2646 previous similar messages
Lustre: 24327:0:(osd_handler.c:1893:osd_trans_dump_creds()) write: 2/618/0, punch: 0/0/0, quota 4/150/0
Lustre: 24327:0:(osd_handler.c:1893:osd_trans_dump_creds()) Skipped 2646 previous similar messages
Lustre: 24327:0:(osd_handler.c:1900:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 24327:0:(osd_handler.c:1900:osd_trans_dump_creds()) Skipped 2646 previous similar messages
Lustre: 24327:0:(osd_handler.c:1907:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 24327:0:(osd_handler.c:1907:osd_trans_dump_creds()) Skipped 2646 previous similar messages
Lustre: 19958:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 502 < left 1086, rollback = 2
Lustre: 19958:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 2502 previous similar messages
2[32653]: segfault at 8 ip 00007fbff35f97e8 sp 00007ffd09143420 error 4 in ld-2.17.so[7fbff35ee000+22000]
LustreError: 412:0:(file.c:242:ll_close_inode_openhandle()) lustre-clilmv-ffff880098fe92a8: inode [0x200000404:0xe57:0x0] mdc close failed: rc = -2
LustreError: 412:0:(file.c:242:ll_close_inode_openhandle()) Skipped 55 previous similar messages
LustreError: 28524:0:(mdd_object.c:403:mdd_xattr_get()) lustre-MDD0000: object [0x200000404:0xe57:0x0] not found: rc = -2
Lustre: 10327:0:(osd_internal.h:1334:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 618, rollback = 7
Lustre: 10327:0:(osd_internal.h:1334:osd_trans_exec_op()) Skipped 121 previous similar messages
2[4969]: segfault at 0 ip (null) sp 00007ffdd00f78f8 error 14 in 10[400000+6000]
13[4733]: segfault at 8 ip 00007fb21aa6e7e8 sp 00007fffdd04f080 error 4 in ld-2.17.so[7fb21aa63000+22000]
LustreError: 19937:0:(ldlm_lockd.c:261:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 0@lo ns: filter-lustre-OST0001_UUID lock: ffff8800adb4e940/0x8c82fea2735f3e60 lrc: 3/0,0 mode: PW/PW res: [0x300000400:0x24d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 0@lo remote: 0x8c82fea2735f3e59 expref: 18 pid: 9113 timeout: 78253 lvb_type: 0
LustreError: 11-0: lustre-OST0001-osc-ffff880322108958: operation ldlm_enqueue to node 0@lo failed: rc = -107
Lustre: lustre-OST0001-osc-ffff880322108958: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 167-0: lustre-OST0001-osc-ffff880322108958: This client was evicted by lustre-OST0001; in progress operations using this service will fail.
Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x280000403:0x1c89:0x0]// may get corrupted (rc -108)
Lustre: 16721:0:(llite_lib.c:3694:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.2@tcp:/lustre/fid: [0x240000405:0x1388:0x0]/ may get corrupted (rc -108)
LustreError: 5481:0:(ldlm_resource.c:1124:ldlm_resource_complain()) lustre-OST0001-osc-ffff880322108958: namespace resource [0x300000400:0x24d:0x0].0x0 (ffff8802b5338040) refcount nonzero (1) after lock cleanup; forcing cleanup.
LustreError: 5481:0:(ldlm_resource.c:1124:ldlm_resource_complain()) Skipped 2 previous similar messages
Lustre: lustre-OST0001-osc-ffff880322108958: Connection restored to (at 0@lo)
Lustre: Skipped 1 previous similar message
Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) lustre-MDT0001-mdtlov: EXTENSION flags=40 set on component[2]=1 of non-SEL file [0x240000405:0x111a:0x0] with magic=0xbd60bd0
Lustre: 7429:0:(lod_lov.c:1326:lod_parse_striping()) Skipped 19 previous similar messages
13[6072]: segfault at 8 ip 00007fac24b7e7e8 sp 00007fffd295cef0 error 4 in ld-2.17.so[7fac24b73000+22000]
8[7687]: segfault at 8 ip 00007f9ab74657e8 sp 00007fffa6f9a6a0 error 4 in ld-2.17.so[7f9ab745a000+22000]
8[7915]: segfault at 8 ip 00007f02a50407e8 sp 00007ffc44ab61a0 error 4 in ld-2.17.so[7f02a5035000+22000]
LustreError: 22748:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) lustre-MDD0002: '7' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush 7' to finish migration.
LustreError: 22748:0:(mdd_dir.c:4340:mdd_migrate_cmd_check()) Skipped 31 previous similar messages
LustreError: 19978:0:(mdd_object.c:3458:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000403:0x2394:0x0]: rc = -2
LustreError: 19978:0:(mdd_object.c:3458:mdd_close()) Skipped 19 previous similar messages
Lustre: dir [0x240000405:0x1618:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 67 previous similar messages
3[10077]: segfault at 8 ip 00007fc645a9c7e8 sp 00007ffd5cb08330 error 4 in ld-2.17.so[7fc645a91000+22000]
LustreError: 17523:0:(llite_nfs.c:338:ll_dir_get_parent_fid()) lustre: failure inode [0x200000404:0x10bf:0x0] get parent: rc = -116
19[19380]: segfault at 8 ip 00007f79677047e8 sp 00007ffe8a169b90 error 4 in ld-2.17.so[7f79676f9000+22000]
LustreError: 23560:0:(llite_lib.c:1959:ll_md_setattr()) md_setattr fails: rc = -5
LustreError: 19981:0:(llog_osd.c:628:llog_osd_write_rec()) lustre-MDT0001-osp-MDT0002: index 2522 already set in llog bitmap [0x240000401:0x2:0x0]