Editing crashreport #75427

ReasonCrashing FunctionWhere to cut BacktraceReports Count
ASSERTION( sai == entry->se_sai ) failedrevalidate_statahead_dentryrevalidate_statahead_dentry
ll_revalidate_statahead
ll_revalidate_dentry
lookup_open
do_last
path_openat
do_filp_open
do_sys_openat2
do_sys_open
__x64_sys_openat
do_syscall_64
entry_SYSCALL_64_after_hwframe
2

Added fields:

Match messages in logs
(every line would be required to be present in log output
Copy from "Messages before crash" column below):
Match messages in full crash
(every line would be required to be present in crash log output
Copy from "Full Crash" column below):
Limit to a test:
(Copy from below "Failing text"):
Delete these reports as invalid (real bug in review or some such)
Bug or comment:
Extra info:

Failures list (last 100):

Failing TestFull CrashMessages before crashLink and date
racer test 1: racer on clients: centos-20.localnet DURATION=2700
LustreError: 667872:0:(statahead.c:2054:revalidate_statahead_dentry()) ASSERTION( sai == entry->se_sai ) failed:
LustreError: 667872:0:(statahead.c:2054:revalidate_statahead_dentry()) LBUG
CPU: 8 PID: 667872 Comm: ls Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
Call Trace:
dump_stack+0x99/0xca
lbug_with_loc.cold.4+0xd/0x86 [libcfs]
revalidate_statahead_dentry+0x145d/0x1aa0 [lustre]
? set_next_entity+0xb4/0xd0
ll_revalidate_statahead+0xa7/0x400 [lustre]
ll_revalidate_dentry+0x335/0x3c0 [lustre]
lookup_open+0x132/0xb30
do_last+0xa17/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f74087bb935
Lustre: 11504:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8dc3bc005780 x1879881262139008/t4294967635(0) o101->8f37b97c-00ad-47b2-b5c2-1a6c8feabf78@0@lo:230/0 lens 376/840 e 0 to 0 dl 1792794540 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
Lustre: 11991:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8dc3b9ab9500 x1879881263546624/t4294968607(0) o101->8f37b97c-00ad-47b2-b5c2-1a6c8feabf78@0@lo:238/0 lens 384/840 e 0 to 0 dl 1792794548 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
4[17437]: segfault at 8 ip 00007f141cc57875 sp 00007ffc7ebd2b90 error 4 in ld-2.28.so[7f141cc36000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
9[24956]: segfault at 8 ip 00007fa588681875 sp 00007fff75a0be70 error 4 in ld-2.28.so[7fa588660000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
4[25349]: segfault at 8 ip 00007f987eec4875 sp 00007ffdfa349df0 error 4 in ld-2.28.so[7f987eea3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[27356]: segfault at 8 ip 00007f43821cb875 sp 00007ffd9d9b6ab0 error 4 in ld-2.28.so[7f43821aa000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
18[29532]: segfault at 8 ip 00007f8e6fa5f875 sp 00007fffe6e43090 error 4 in ld-2.28.so[7f8e6fa3e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 21914:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8dc3e0906580 x1879881284311808/t4294979145(0) o101->8f37b97c-00ad-47b2-b5c2-1a6c8feabf78@0@lo:345/0 lens 376/816 e 0 to 0 dl 1792794655 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
11[44196]: segfault at 8 ip 00007f35d5f2a875 sp 00007ffe185529a0 error 4 in ld-2.28.so[7f35d5f09000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[46989]: segfault at 8 ip 00007f5d0f508875 sp 00007fff0fa45080 error 4 in ld-2.28.so[7f5d0f4e7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
4[49053]: segfault at 8 ip 00007f255cbf1875 sp 00007ffef6deb130 error 4 in ld-2.28.so[7f255cbd0000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[48512]: segfault at 8 ip 00007fdca1626875 sp 00007ffe201e6260 error 4 in ld-2.28.so[7fdca1605000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
10[62255]: segfault at 8 ip 00007f40fb62b875 sp 00007ffd80ec19f0 error 4 in ld-2.28.so[7f40fb60a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 64083:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc34ae8a800: inode [0x240000404:0x58a:0x0] mdc close failed: rc = -13
3[64342]: segfault at 8 ip 00007f94a40d9875 sp 00007ffd34d0b7b0 error 4 in ld-2.28.so[7f94a40b8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 72115:0:(file.c:6392:ll_inode_revalidate_fini()) lustre: revalidate FID [0x240000404:0x656:0x0] error: rc = -4
LustreError: 76608:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc3ac8b0800: inode [0x280000404:0x6c9:0x0] mdc close failed: rc = -13
2[78395]: segfault at 0 ip 000055a395f8cb47 sp 00007ffde0c8f3d0 error 6 in 2[55a395f88000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
hrtimer: interrupt took 3652523 ns
ODEBUG: object 00000000fa998e09 is on stack 000000009bdcc582, but NOT annotated.
WARNING: CPU: 11 PID: 15830 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_zfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) zfs(O) spl(O) libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 11 PID: 15830 Comm: mdt00_038 Kdump: loaded Tainted: G O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: fe ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffa4ced33e7320 EFLAGS: 00010002
RAX: 0000000000000050 RBX: ffffa4ced33e7468 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8dc5724de5a8 RDI: ffff8dc5724de5a8
RBP: ffffa4ced33e7370 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffa4ced33e70f0 R12: ffffffffadad8888
R13: ffffffffac705aa0 R14: 00000000000534c0 R15: ffff8dc3b08be410
FS: 0000000000000000(0000) GS:ffff8dc5724c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000557c3fd1df38 CR3: 00000001888c1000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x110c/0x2620 [lod]
? slab_free_freelist_hook.isra.49+0x111/0x230
lod_comp_prep_create+0x1780/0x1c30 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
? lod_sub_declare_create+0xcc/0x150 [lod]
lod_declare_create+0x237/0xc30 [lod]
? osd_xattr_get+0x2d7/0x8d0 [osd_zfs]
dt_declare_create+0x79/0x230 [obdclass]
mdd_declare_create_object_internal+0xb6/0x2f0 [mdd]
mdd_declare_create_object.isra.20+0x5f/0xfb0 [mdd]
mdd_declare_create+0x5a/0x680 [mdd]
mdd_create+0x6d7/0x2550 [mdd]
mdt_reint_open+0x3338/0x3c40 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.50+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 94da3ed630080185 ]---
10[101091]: segfault at 1 ip 0000560a6e97b950 sp 00007ffe89d0a1d8 error 6 in 10[560a6e977000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0[106037]: segfault at 8 ip 00007f9f7f157875 sp 00007fff8ba41e30 error 4 in ld-2.28.so[7f9f7f136000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
15[111604]: segfault at 1b ip 0000565284ba1b54 sp 00007ffe0ef7eb70 error 6 in 15[565284b9d000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 75 1b 48 83 c4 20 5b 5d 41 5c c3 e8 5b ca ff ff c7 00 16 00 00
LustreError: 113627:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc3ac8b0800: inode [0x280000404:0xacd:0x0] mdc close failed: rc = -13
LustreError: 20452:0:(osd_io.c:1017:osd_grow_blocksize()) object [0x2c0000402:0x133:0x0]: change block size4096 -> 1048576 error rc = -95
LustreError: 20373:0:(osd_io.c:1017:osd_grow_blocksize()) object [0x2c0000401:0x162:0x0]: change block size4096 -> 1048576 error rc = -95
13[128863]: segfault at 0 ip 000055eb0b1ddb47 sp 00007ffe23b13eb0 error 6 in 13[55eb0b1d9000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9[131490]: segfault at 8 ip 00007f4a30d24875 sp 00007fffd39b2ac0 error 4 in ld-2.28.so[7f4a30d03000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[135447]: segfault at 560ae56ae376 ip 0000560ae54a7b4a sp 00007ffe8cffda90 error 4 in 13[560ae54a3000+7000]
Code: 00 0f 1f 44 00 00 f3 0f 1e fa f2 ff 25 35 68 20 00 0f 1f 44 00 00 f3 0f 1e fa f2 ff 25 2d 68 20 00 0f 1f 44 00 00 f3 0f 1e fa <f2> ff 25 25 68 20 00 0f 1f 44 00 00 f3 0f 1e fa f2 ff 25 1d 68 20
18[141224]: segfault at 8 ip 00007fc5c9169875 sp 00007fffa3c34ff0 error 4 in ld-2.28.so[7fc5c9148000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
4[142661]: segfault at 0 ip 000055b5a872eb47 sp 00007ffff4f18890 error 6 in 6[55b5a872a000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9[151513]: segfault at 8 ip 00007fea032b9875 sp 00007ffe8fc662e0 error 4 in ld-2.28.so[7fea03298000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 156220:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc34ae8a800: inode [0x240000404:0xf9d:0x0] mdc close failed: rc = -13
LustreError: 11559:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0001: unable to read [0x280000404:0x104b:0x0] ACL: rc = -2
13[165818]: segfault at 0 ip 0000564bf6fcfa88 sp 00007ffd3ee1cba0 error 6 in 13[564bf6fce000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5[173542]: segfault at 0 ip 000055ed182b3b47 sp 00007ffd7e989270 error 6 in 5[55ed182af000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
10[174168]: segfault at 8 ip 00007ff3b045c875 sp 00007ffeafdae5a0 error 4 in ld-2.28.so[7ff3b043b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 97019:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0000: unable to read [0x200000403:0x1892:0x0] ACL: rc = -2
16[197714]: segfault at 8 ip 00007f343706c875 sp 00007fff3e65e910 error 4 in ld-2.28.so[7f343704b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
6[205627]: segfault at 8 ip 00007f2186260875 sp 00007ffd42f31e60 error 4 in ld-2.28.so[7f218623f000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[207750]: segfault at 0 ip 00005602361bfb47 sp 00007ffdafc6f4d0 error 6 in 2[5602361bb000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0[207507]: segfault at 8 ip 00007f547732f875 sp 00007ffce9c45030 error 4 in ld-2.28.so[7f547730e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[208692]: segfault at 8 ip 00007f6e70546875 sp 00007ffc5084d060 error 4 in ld-2.28.so[7f6e70525000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 66841:0:(out_handler.c:887:out_tx_end()) lustre-MDT0002-osd: error during execution of #0 from /home/green/git/lustre-release/lustre/ptlrpc/../target/out_handler.c:141: rc = -2
3[220698]: segfault at 8 ip 00007f4044afe875 sp 00007fffb6112020 error 4 in ld-2.28.so[7f4044add000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[225289]: segfault at 8 ip 00007ff97120c875 sp 00007ffe14226130 error 4 in ld-2.28.so[7ff9711eb000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[228720]: segfault at 8 ip 00007ff405be7875 sp 00007fff3e87c650 error 4 in ld-2.28.so[7ff405bc6000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
12[231333]: segfault at 8 ip 00007f8a065f2875 sp 00007ffd6b26b650 error 4 in ld-2.28.so[7f8a065d1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[232356]: segfault at 8 ip 00007f6d2d291875 sp 00007fff917a9c90 error 4 in ld-2.28.so[7f6d2d270000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
6[234745]: segfault at 8 ip 00007fbddfa88875 sp 00007ffd933724a0 error 4 in ld-2.28.so[7fbddfa67000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 1[238440] general protection fault ip:56521bc42d32 sp:7ffd739ccb08 error:54 in 1[56521bc40000+7000]
17[243199]: segfault at 8 ip 00007f7e53e3d875 sp 00007ffc414d2a10 error 4 in ld-2.28.so[7f7e53e1c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 243721:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc3ac8b0800: inode [0x280000403:0x1837:0x0] mdc close failed: rc = -13
19[246261]: segfault at 8 ip 00007f737ebe9875 sp 00007ffeeecd7d20 error 4 in ld-2.28.so[7f737ebc8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 11[249188] trap invalid opcode ip:556922960cca sp:7fff3191ce28 error:0 in 11[55692295a000+7000]
LustreError: 13301:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0002: unable to read [0x240000404:0x1918:0x0] ACL: rc = -2
0[258500]: segfault at 55777de51000 ip 000055777de51000 sp 00007ffc2dbcee98 error 14 in 0[55777e051000+1000]
Code: Unable to access opcode bytes at RIP 0x55777de50fd6.
LustreError: 262767:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc34ae8a800: inode [0x200000404:0x2230:0x0] mdc close failed: rc = -13
4[273687]: segfault at e ip 0000558ce4e10230 sp 00007ffe6cd3e4e0 error 6 in 4[558ce4e0a000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 45 0e 10 86 02 41 0e 18 83 03 44 0e 20 02 40 0a 0e 18 41 0e 10
4[275678]: segfault at 8 ip 00007f9ac7081875 sp 00007ffed2e7e900 error 4 in ld-2.28.so[7f9ac7060000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 280139:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc3ac8b0800: inode [0x200000403:0x263f:0x0] mdc close failed: rc = -13
ODEBUG: object 0000000057e81c95 is on stack 000000008bb7eedb, but NOT annotated.
WARNING: CPU: 1 PID: 13214 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_zfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) zfs(O) spl(O) libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 1 PID: 13214 Comm: mdt00_027 Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: fe ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffa4cecaf0b320 EFLAGS: 00010002
RAX: 0000000000000050 RBX: ffffa4cecaf0b468 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8dc57225e5a8 RDI: ffff8dc57225e5a8
RBP: ffffa4cecaf0b370 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffa4cecaf0b0f0 R12: ffffffffadaa1c08
R13: ffffffffac705aa0 R14: 000000000001c840 R15: ffff8dc3af624578
FS: 0000000000000000(0000) GS:ffff8dc572240000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa180133d40 CR3: 00000001e84e0000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x110c/0x2620 [lod]
? __might_sleep+0x60/0xc0
lod_comp_prep_create+0x1780/0x1c30 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
? lod_sub_declare_create+0xcc/0x150 [lod]
? __osd_sa_xattr_get.isra.8+0xb3/0x290 [osd_zfs]
lod_declare_create+0x237/0xc30 [lod]
dt_declare_create+0x79/0x230 [obdclass]
? lod_alloc_comp_entries+0x28e/0x640 [lod]
mdd_declare_create_object_internal+0xb6/0x2f0 [mdd]
mdd_declare_create_object.isra.20+0x5f/0xfb0 [mdd]
mdd_declare_create+0x5a/0x680 [mdd]
mdd_create+0x6d7/0x2550 [mdd]
mdt_reint_open+0x3338/0x3c40 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.50+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 94da3ed630080186 ]---
LustreError: 22173:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0000: unable to read [0x200000404:0x26a3:0x0] ACL: rc = -2
18[312003]: segfault at 8 ip 00007f5fda49b875 sp 00007ffc42907e10 error 4 in ld-2.28.so[7f5fda47a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
12[320800]: segfault at 0 ip 00005563142bbb47 sp 00007fffc0752320 error 6 in 12[5563142b7000+7000]
Code: Unable to access opcode bytes at RIP 0x5563142bbb1d.
17[329824]: segfault at 8 ip 00007f554a49a875 sp 00007ffdf28fcd80 error 4 in ld-2.28.so[7f554a479000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 320940:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0002: unable to read [0x240000404:0x2137:0x0] ACL: rc = -2
18[332006]: segfault at 8 ip 00007f815aa04875 sp 00007ffdf589c500 error 4 in ld-2.28.so[7f815a9e3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
9[332899]: segfault at 8 ip 00007f66e228d875 sp 00007ffdbc9c2060 error 4 in ld-2.28.so[7f66e226c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[339358]: segfault at 8 ip 00007f5cdc22f875 sp 00007fff514acff0 error 4 in ld-2.28.so[7f5cdc20e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 9[347790] trap invalid opcode ip:556201870df0 sp:7ffd9545e410 error:0 in 9[55620186c000+7000]
12[356104]: segfault at 55ee1e4c6870 ip 000055ee1e4c6870 sp 00007ffe9f76b118 error 7 in 10[55ee1e4c5000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
18[357569]: segfault at 8 ip 00007fb265cc1875 sp 00007ffdd468d520 error 4 in ld-2.28.so[7fb265ca0000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
ODEBUG: object 00000000b4933dc3 is on stack 00000000e731796b, but NOT annotated.
WARNING: CPU: 5 PID: 120327 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_zfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) zfs(O) spl(O) libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 5 PID: 120327 Comm: mdt00_094 Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: fe ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffa4cee2d93320 EFLAGS: 00010006
RAX: 0000000000000050 RBX: ffffa4cee2d93468 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8dc57235e5a8 RDI: ffff8dc57235e5a8
RBP: ffffa4cee2d93370 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffa4cee2d930f0 R12: ffffffffadab7328
R13: ffffffffac705aa0 R14: 0000000000031f60 R15: ffff8dc3c9b473e8
FS: 0000000000000000(0000) GS:ffff8dc572340000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb638037298 CR3: 0000000275819000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x110c/0x2620 [lod]
? slab_free_freelist_hook.isra.49+0x111/0x230
lod_comp_prep_create+0x1780/0x1c30 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
? lod_sub_declare_create+0xcc/0x150 [lod]
lod_declare_create+0x237/0xc30 [lod]
? osd_xattr_get+0x2d7/0x8d0 [osd_zfs]
dt_declare_create+0x79/0x230 [obdclass]
mdd_declare_create_object_internal+0xb6/0x2f0 [mdd]
mdd_declare_create_object.isra.20+0x5f/0xfb0 [mdd]
mdd_declare_create+0x5a/0x680 [mdd]
mdd_create+0x6d7/0x2550 [mdd]
mdt_reint_open+0x3338/0x3c40 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.50+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 94da3ed630080187 ]---
11[363009]: segfault at 562fe08fa000 ip 0000562fe08fa000 sp 00007ffd6bef7b10 error 14 in 11[562fe0afa000+1000]
Code: Unable to access opcode bytes at RIP 0x562fe08f9fd6.
16[364278]: segfault at 8 ip 00007fba13326875 sp 00007ffcf334e210 error 4 in ld-2.28.so[7fba13305000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 270147:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0002: unable to read [0x240000404:0x27dc:0x0] ACL: rc = -2
4[369430]: segfault at 0 ip 0000564a9b429b47 sp 00007fffe0a873b0 error 6 in 4[564a9b425000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
12[370578]: segfault at 0 ip 000055a22cb62b47 sp 00007ffec4b88f20 error 6 in 12[55a22cb5e000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
LustreError: 370578:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc34ae8a800: inode [0x200000404:0x3223:0x0] mdc close failed: rc = -13
16[371499]: segfault at 8 ip 00007f93e7063875 sp 00007fff78cb40b0 error 4 in ld-2.28.so[7f93e7042000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[373884]: segfault at 8 ip 00007fc97e8a9875 sp 00007ffd7799d8b0 error 4 in ld-2.28.so[7fc97e888000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 14[375367] trap invalid opcode ip:55aaeef479c3 sp:7ffc44635db8 error:0 in 14[55aaeef41000+7000]
Lustre: mdt00_025: service thread pid 13181 was inactive for 41.316 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
task:mdt_io00_003 state:I
Lustre: Skipped 1 previous similar message
task:mdt00_025 state:I stack:0 pid:13181 ppid:2 flags:0x80004080
Call Trace:
__schedule+0x351/0xcb0
stack:0 pid:13364 ppid:2 flags:0x80004080
task:mdt00_117 state:I stack:0 pid:284757 ppid:2 flags:0x80004080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_timeout+0xb1/0x190
? __next_timer_interrupt+0x170/0x170
? do_raw_spin_unlock+0x79/0x190
ldlm_completion_ast+0xc27/0x1100 [ptlrpc]
? woken_wake_function+0x40/0x40
ldlm_cli_enqueue_local+0x5b4/0xb70 [ptlrpc]
? ldlm_cli_enqueue_local+0xb70/0xb70 [ptlrpc]
? mdt_object_put+0x130/0x130 [mdt]
mdt_object_lock_internal+0x219/0x5b0 [mdt]
? mdt_object_put+0x130/0x130 [mdt]
? ldlm_cli_enqueue_local+0xb70/0xb70 [ptlrpc]
mdt_object_check_lock+0x24b/0x4c0 [mdt]
mdt_object_stripes_lock+0xb0/0x690 [mdt]
mdt_reint_unlink+0x11c8/0x1a30 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_reint+0x16c/0x190 [mdt]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
schedule+0xc4/0x180
schedule_timeout+0xb1/0x190
? __next_timer_interrupt+0x170/0x170
? do_raw_spin_unlock+0x79/0x190
ldlm_completion_ast+0xc27/0x1100 [ptlrpc]
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_timeout+0xb1/0x190
? __next_timer_interrupt+0x170/0x170
? do_raw_spin_unlock+0x79/0x190
ldlm_completion_ast+0xc27/0x1100 [ptlrpc]
? woken_wake_function+0x40/0x40
ldlm_cli_enqueue_fini+0xa80/0x1470 [ptlrpc]
ldlm_cli_enqueue+0x474/0xe90 [ptlrpc]
? ldlm_cli_enqueue_local+0xb70/0xb70 [ptlrpc]
? woken_wake_function+0x40/0x40
? mdt_object_sync.isra.46+0x610/0x610 [mdt]
ldlm_cli_enqueue_local+0x5b4/0xb70 [ptlrpc]
osp_md_object_lock+0x2e9/0x410 [osp]
lod_object_lock+0x18f/0xa00 [lod]
? fld_local_lookup+0x7a/0x360 [fld]
mdd_object_lock+0x41/0x120 [mdd]
mdt_remote_object_lock_try+0x2ba/0x550 [mdt]
mdt_object_pdo_lock+0x345/0x930 [mdt]
mdt_parent_lock+0x93/0x370 [mdt]
? mdd_xattr_get+0xb3/0x7c0 [mdd]
mdt_lock_two_dirs+0xa5/0x220 [mdt]
? mdt_attr_get_pfid+0xce/0x5b0 [mdt]
mdt_reint_rename+0x11ff/0x34d0 [mdt]
? mdt_root_squash+0x29/0x5b0 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_reint+0x16c/0x190 [mdt]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
? ldlm_cli_enqueue_local+0xb70/0xb70 [ptlrpc]
? mdt_object_put+0x130/0x130 [mdt]
mdt_object_lock_internal+0x219/0x5b0 [mdt]
? mdt_object_put+0x130/0x130 [mdt]
? ldlm_cli_enqueue_local+0xb70/0xb70 [ptlrpc]
mdt_object_lock+0xa1/0x250 [mdt]
mdt_getattr_name_lock+0x2693/0x32f0 [mdt]
? __req_capsule_get+0x494/0xb30 [ptlrpc]
mdt_intent_getattr+0x301/0x650 [mdt]
mdt_intent_opc.constprop.50+0x153/0xfa0 [mdt]
? mdt_getattr_name_lock+0x32f0/0x32f0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
18[379674]: segfault at 8 ip 00007f611de4e875 sp 00007ffd98809930 error 4 in ld-2.28.so[7f611de2d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
12[379476]: segfault at 8 ip 00007f7f8df4b875 sp 00007fffa1277490 error 4 in ld-2.28.so[7f7f8df2a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
17[380194]: segfault at 8 ip 00007feaa68d9875 sp 00007ffc3fbb5890 error 4 in ld-2.28.so[7feaa68b8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: mdt00_059: service thread pid 21907 was inactive for 43.861 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
14[393760]: segfault at 8 ip 00007f6d7e8c9875 sp 00007ffcbd9b3b60 error 4 in ld-2.28.so[7f6d7e8a8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 7419:0:(ldlm_lockd.c:255:expired_lock_main()) ### lock callback timer expired after 102s: evicting client at 0@lo ns: mdt-lustre-MDT0000_UUID lock: 00000000acf1c741/0xa4fa08c5d3335074 lrc: 3/0,0 mode: PR/PR res: [0x200000404:0x2fde:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 0@lo remote: 0xa4fa08c5d333505f expref: 1224 pid: 22185 timeout: 1317 lvb_type: 0 lru_score: 0 lru_type: 0 lru_pins: 0
LustreError: lustre-MDT0000-mdc-ffff8dc3ac8b0800: operation mds_getattr to node 0@lo failed: rc = -107
Lustre: lustre-MDT0000-mdc-ffff8dc3ac8b0800: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 11675:0:(client.c:1411:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff8dc3f2572a00 x1879881552632192/t0(0) o104->lustre-MDT0000@0@lo:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
LustreError: lustre-MDT0000-mdc-ffff8dc3ac8b0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
LustreError: 21800:0:(ldlm_lockd.c:1445:ldlm_handle_enqueue()) ### lock on destroyed export 00000000db1b88dc ns: mdt-lustre-MDT0000_UUID lock: 000000003964a961/0xa4fa08c5d347b13b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 34 type: IBT gid 0 flags: 0x50200000000000 nid: 0@lo remote: 0xa4fa08c5d347b118 expref: 871 pid: 21800 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 lru_pins: 0
LustreError: 392502:0:(file.c:6392:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000404:0x33f9:0x0] error: rc = -5
LustreError: 392502:0:(file.c:6392:ll_inode_revalidate_fini()) Skipped 10 previous similar messages
LustreError: 396210:0:(llite_lib.c:2150:ll_md_setattr()) md_setattr fails: rc = -5
LustreError: 396173:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc3ac8b0800: inode [0x200000404:0x36d9:0x0] mdc close failed: rc = -108
LustreError: 396371:0:(ldlm_resource.c:1227:ldlm_resource_complain()) lustre-MDT0000-mdc-ffff8dc3ac8b0800: namespace resource [0x200000404:0x2fde:0x0].0x0 (ffff8dc3ee9d9d00) refcount nonzero (1) after lock cleanup; forcing cleanup.
Lustre: lustre-MDT0000-mdc-ffff8dc3ac8b0800: Connection restored to 0@lo (at 0@lo)
9[400856]: segfault at 8 ip 00007fcfb716a875 sp 00007ffd967b3c30 error 4 in ld-2.28.so[7fcfb7149000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[400852]: segfault at 8 ip 00007faedeb22875 sp 00007ffd70b17d40 error 4 in ld-2.28.so[7faedeb01000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
INFO: task dir_create.sh:11202 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:dir_create.sh state:D stack:0 pid:11202 ppid:10776 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
? libcfs_debug_msg+0xec9/0x13f0 [libcfs]
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
down_read+0xa9/0x2c0
lookup_slow+0x34/0x80
walk_component+0x264/0x4e0
link_path_walk+0x417/0x890
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0x135/0x500
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f834b8c7332
Code: Unable to access opcode bytes at RIP 0x7f834b8c7308.
RSP: 002b:00007ffeebf9fd60 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 000055a805cb88a0 RCX: 00007f834b8c7332
RDX: 0000000000000241 RSI: 000055a805cb18f0 RDI: 00000000ffffff9c
RBP: 00007ffeebf9fe60 R08: 0000000000000020 R09: 000055a805c97012
R10: 00000000000001b6 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000003 R14: 0000000000000001 R15: 000055a805cb18f0
INFO: task mrename:362915 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:mrename state:D stack:0 pid:362915 ppid:11087 flags:0x80004080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
__mutex_lock.isra.10+0x93e/0xec0
__mutex_lock_slowpath+0x23/0x30
mutex_lock+0x5f/0x80
lock_rename+0x38/0x160
do_renameat2+0x326/0x750
__x64_sys_rename+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f21516ad6cb
Code: Unable to access opcode bytes at RIP 0x7f21516ad6a1.
RSP: 002b:00007ffc4a9f1658 EFLAGS: 00000202 ORIG_RAX: 0000000000000052
RAX: ffffffffffffffda RBX: 00007ffc4a9f1748 RCX: 00007f21516ad6cb
RDX: 00007ffc4a9f1768 RSI: 00007ffc4a9f3b73 RDI: 00007ffc4a9f3b5c
RBP: 0000000000400800 R08: 00007f21519fcd20 R09: 00007f21519fcd20
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000400710
R13: 00007ffc4a9f1740 R14: 0000000000000000 R15: 0000000000000000
INFO: task ls:363219 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363219 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7fecc14cb935
Code: Unable to access opcode bytes at RIP 0x7fecc14cb90b.
RSP: 002b:00007ffc1175e910 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 00005608a5ef4290 RCX: 00007fecc14cb935
RDX: 0000000000090800 RSI: 00005608a5ef42c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 00005608a5ef2c70 R09: 00005608a5eeb012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007fecc1dc4690
R13: 0000000000000000 R14: 00005608a5ef42c0 R15: 00005608a51c6b24
INFO: task ls:363220 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363220 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7fda20a63935
Code: Unable to access opcode bytes at RIP 0x7fda20a6390b.
RSP: 002b:00007fff0d274ab0 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 0000560819ba4290 RCX: 00007fda20a63935
RDX: 0000000000090800 RSI: 0000560819ba42c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 0000560819ba2c70 R09: 0000560819b9b012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007fda2135c690
R13: 0000000000000000 R14: 0000560819ba42c0 R15: 0000560817d5eb24
INFO: task ls:363221 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363221 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f4a8e1c0935
Code: Unable to access opcode bytes at RIP 0x7f4a8e1c090b.
RSP: 002b:00007ffc179af780 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 0000562fd458c290 RCX: 00007f4a8e1c0935
RDX: 0000000000090800 RSI: 0000562fd458c2c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 0000562fd458ac70 R09: 0000562fd4583012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007f4a8eab9690
R13: 0000000000000000 R14: 0000562fd458c2c0 R15: 0000562fd3d27b24
INFO: task ls:363222 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363222 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f2eec7b6935
Code: Unable to access opcode bytes at RIP 0x7f2eec7b690b.
RSP: 002b:00007ffc74273980 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 000055c48a12a290 RCX: 00007f2eec7b6935
RDX: 0000000000090800 RSI: 000055c48a12a2c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 000055c48a128c70 R09: 000055c48a121012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007f2eed0af690
R13: 0000000000000000 R14: 000055c48a12a2c0 R15: 000055c4891bbb24
INFO: task ls:363223 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363223 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7fb2ebaaa935
Code: Unable to access opcode bytes at RIP 0x7fb2ebaaa90b.
RSP: 002b:00007ffd43d75980 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 000055850fc8a1a0 RCX: 00007fb2ebaaa935
RDX: 0000000000090800 RSI: 000055850fc8a1d0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 000055850fc8a260 R09: 000055850fc81012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007fb2ec3a3690
R13: 0000000000000000 R14: 000055850fc8a1d0 R15: 000055850de7fb24
INFO: task ls:363224 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363224 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7fbb43836935
Code: Unable to access opcode bytes at RIP 0x7fbb4383690b.
RSP: 002b:00007ffe80961210 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 000055ac5bf10290 RCX: 00007fbb43836935
RDX: 0000000000090800 RSI: 000055ac5bf102c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 000055ac5bf0ec70 R09: 000055ac5bf07012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007fbb4412f690
R13: 0000000000000000 R14: 000055ac5bf102c0 R15: 000055ac5a406b24
INFO: task ls:363227 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363227 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? getname_flags+0x73/0x320
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f745cdff935
Code: Unable to access opcode bytes at RIP 0x7f745cdff90b.
RSP: 002b:00007ffcdcc288a0 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 0000555bdc961290 RCX: 00007f745cdff935
RDX: 0000000000090800 RSI: 0000555bdc9612c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 0000555bdc95fc70 R09: 0000555bdc958012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007f745d6f8690
R13: 0000000000000000 R14: 0000555bdc9612c0 R15: 0000555bda9f1b24
INFO: task ls:363228 blocked for more than 120 seconds.
Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:ls state:D stack:0 pid:363228 ppid:11101 flags:0x80000080
Call Trace:
__schedule+0x351/0xcb0
schedule+0xc4/0x180
schedule_preempt_disabled+0x25/0x40
rwsem_down_read_slowpath+0x44e/0x600
? lookup_fast+0x100/0x4d0
down_read+0xa9/0x2c0
do_last+0x9f5/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? do_raw_spin_unlock+0x79/0x190
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f746ab03935
Code: Unable to access opcode bytes at RIP 0x7f746ab0390b.
RSP: 002b:00007ffc6028d6e0 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 000055d85abf5290 RCX: 00007f746ab03935
RDX: 0000000000090800 RSI: 000055d85abf52c0 RDI: 00000000ffffff9c
RBP: 0000000000000000 R08: 000055d85abf3c70 R09: 000055d85abec012
R10: 0000000000000000 R11: 0000000000000287 R12: 00007f746b3fc690
R13: 0000000000000000 R14: 000055d85abf52c0 R15: 000055d85a3bcb24
11[413855]: segfault at 8 ip 00007f21178d2875 sp 00007fff97b44590 error 4 in ld-2.28.so[7f21178b1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[416258]: segfault at 8 ip 00007f624ebef875 sp 00007ffdec89ea40 error 4 in ld-2.28.so[7f624ebce000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[417107]: segfault at 8 ip 00007efe8504a875 sp 00007ffcb99c28b0 error 4 in ld-2.28.so[7efe85029000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[418181]: segfault at 8 ip 00007f4b73cf2875 sp 00007fffa806eb00 error 4 in ld-2.28.so[7f4b73cd1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 21800:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0000: unable to read [0x200000405:0x3af:0x0] ACL: rc = -2
2[420781]: segfault at 8 ip 00007fd7e0e96875 sp 00007fffce7a47e0 error 4 in ld-2.28.so[7fd7e0e75000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
10[428801]: segfault at 8 ip 00007f7f3077a875 sp 00007fff09634070 error 4 in ld-2.28.so[7f7f30759000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[448014]: segfault at 8 ip 00007fee9194d875 sp 00007fff80fe7230 error 4 in ld-2.28.so[7fee9192c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[450284]: segfault at 8 ip 00007f0341390875 sp 00007fffde8355e0 error 4 in ld-2.28.so[7f034136f000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
12[453746]: segfault at 8 ip 00007fd35f660875 sp 00007ffda371ca30 error 4 in ld-2.28.so[7fd35f63f000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: mdt00_065: service thread pid 21933 was inactive for 224.994 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one.
LustreError: 13181:0:(ldlm_request.c:103:ldlm_expired_completion_wait()) ### lock timed out (enqueued at 1792795652, 328s ago); not entering recovery in server code, just going back to sleep ns: mdt-lustre-MDT0001_UUID lock: ffff8dc3bbf9bc00/0xa4fa08c5d3335b41 lrc: 3/1,0 mode: --/PR res: [0x280000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 9 type: IBT flags: 0x40210400000020 pid: 13181 initiator: MDT1 lru_score: 0 lru_type: 0 lru_pins: 0
Lustre: lustre-MDT0001-osp-MDT0002: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 13364:0:(ldlm_request.c:123:ldlm_expired_completion_wait()) ### lock timed out (enqueued at 1792795653, 327s ago), entering recovery for lustre-MDT0001_UUID@0@lo ns: lustre-MDT0001-osp-MDT0002 lock: 0000000095537206/0xa4fa08c5d3337e64 lrc: 4/0,1 mode: --/CW res: [0x280000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa4fa08c5d3337e95 expref: -99 pid: 13364 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 lru_pins: 0
Lustre: lustre-MDT0001: Received new MDS connection from 0@lo, keep former export from same NID
Lustre: lustre-MDT0001-osp-MDT0002: Connection restored to 0@lo (at 0@lo)
LustreError: 13181:0:(ldlm_request.c:103:ldlm_expired_completion_wait()) Skipped 1 previous similar message
LustreError: dumping log to /tmp/lustre-log.1792795981.21907
ODEBUG: object 00000000ea68ec42 is on stack 00000000e6b7fa22, but NOT annotated.
WARNING: CPU: 5 PID: 270113 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_zfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) zfs(O) spl(O) libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 5 PID: 270113 Comm: mdt00_109 Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: fe ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffa4cee85e3320 EFLAGS: 00010002
RAX: 0000000000000050 RBX: ffffa4cee85e3468 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8dc57235e5a8 RDI: ffff8dc57235e5a8
RBP: ffffa4cee85e3370 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffa4cee85e30f0 R12: ffffffffadaf59c8
R13: ffffffffac705aa0 R14: 0000000000070600 R15: ffff8dc3c30db0c8
FS: 0000000000000000(0000) GS:ffff8dc572340000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc4e2674000 CR3: 0000000182e57000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? vsnprintf+0x1fe/0x7f0
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x110c/0x2620 [lod]
? kvmalloc_node+0x60/0xe0
lod_comp_prep_create+0x1780/0x1c30 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
? lod_sub_declare_create+0xcc/0x150 [lod]
lod_declare_create+0x237/0xc30 [lod]
dt_declare_create+0x79/0x230 [obdclass]
? lod_alloc_comp_entries+0x28e/0x640 [lod]
mdd_declare_create_object_internal+0xb6/0x2f0 [mdd]
mdd_declare_create_object.isra.20+0x5f/0xfb0 [mdd]
mdd_declare_create+0x5a/0x680 [mdd]
mdd_create+0x6d7/0x2550 [mdd]
mdt_reint_open+0x3338/0x3c40 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.50+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 94da3ed630080188 ]---
12[464073]: segfault at 8 ip 00007f3aa8c38875 sp 00007fffe5d92f50 error 4 in ld-2.28.so[7f3aa8c17000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 16962:0:(mdt_handler.c:749:mdt_pack_acl2body()) lustre-MDT0000: unable to read [0x200000404:0x44c7:0x0] ACL: rc = -2
LustreError: 468941:0:(file.c:252:ll_close_inode_openhandle()) lustre-clilmv-ffff8dc3ac8b0800: inode [0x200000404:0x4523:0x0] mdc close failed: rc = -13
LustreError: 468941:0:(file.c:252:ll_close_inode_openhandle()) Skipped 74 previous similar messages
2[474450]: segfault at 8 ip 00007f06deb50875 sp 00007fffb6a71450 error 4 in ld-2.28.so[7f06deb2f000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[481751]: segfault at 8 ip 00007fa49d269875 sp 00007ffdcecb19d0 error 4 in ld-2.28.so[7fa49d248000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[486335]: segfault at 0 ip 0000565332b7bb47 sp 00007ffc50cdadd0 error 6 in 0[565332b77000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6[487386]: segfault at 8 ip 00007fe23665c875 sp 00007ffc8e758700 error 4 in ld-2.28.so[7fe23663b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[493301]: segfault at 8 ip 00007ff198cc2875 sp 00007ffd53055f90 error 4 in ld-2.28.so[7ff198ca1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 21933:0:(ldlm_request.c:103:ldlm_expired_completion_wait()) ### lock timed out (enqueued at 1792795755, 324s ago); not entering recovery in server code, just going back to sleep ns: mdt-lustre-MDT0001_UUID lock: ffff8dc473dfbc00/0xa4fa08c5d347cdea lrc: 3/0,1 mode: --/CW res: [0x280000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 21933 initiator: MDT1 lru_score: 0 lru_type: 0 lru_pins: 0
Lustre: lustre-MDT0002-osp-MDT0000: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 284757:0:(ldlm_request.c:123:ldlm_expired_completion_wait()) ### lock timed out (enqueued at 1792795755, 324s ago), entering recovery for lustre-MDT0002_UUID@0@lo ns: lustre-MDT0002-osp-MDT0000 lock: 000000006df18e30/0xa4fa08c5d347c944 lrc: 4/0,1 mode: --/EX res: [0x240000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa4fa08c5d347c94b expref: -99 pid: 284757 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 lru_pins: 0
Lustre: lustre-MDT0002: Received new MDS connection from 0@lo, keep former export from same NID
Lustre: lustre-MDT0002-osp-MDT0000: Connection restored to 0@lo (at 0@lo)
13[499115]: segfault at 0 ip 000056276fb8aa88 sp 00007ffc7e60b2b0 error 6 in 13[56276fb89000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8[499536]: segfault at 0 ip 000055fa46a69b47 sp 00007ffe9c7525d0 error 6 in 8[55fa46a65000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5[501020]: segfault at 8 ip 00007f50327ca875 sp 00007ffe777c69a0 error 4 in ld-2.28.so[7f50327a9000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 17[504163] general protection fault ip:5592dd9b4ba8 sp:7ffc4ce0ed98 error:0 in 16[5592dd9af000+7000]
traps: 5[504642] trap invalid opcode ip:55ef6ff69a3a sp:7ffdf7375278 error:0 in 5[55ef6ff64000+7000]
5[505098]: segfault at 8 ip 00007f792f2f2875 sp 00007fffff2e0ac0 error 4 in ld-2.28.so[7f792f2d1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 9[513623] trap invalid opcode ip:55d7f54995b6 sp:7ffd73332cd8 error:0 in 9[55d7f5494000+7000]
10[514859]: segfault at 8 ip 00007f074aa78875 sp 00007fff771092d0 error 4 in ld-2.28.so[7f074aa57000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[520348]: segfault at 8 ip 00007fc223822875 sp 00007ffe69604a40 error 4 in ld-2.28.so[7fc223801000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[521071]: segfault at 8 ip 00007f0c46a16875 sp 00007ffc7a0948b0 error 4 in ld-2.28.so[7f0c469f5000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[523147]: segfault at 8 ip 00007f7a835be875 sp 00007ffdfe8f90e0 error 4 in ld-2.28.so[7f7a8359d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[523129]: segfault at 8 ip 00007f114557f875 sp 00007ffc4b9ec330 error 4 in ld-2.28.so[7f114555e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[527189]: segfault at 0 ip 000056520acdab47 sp 00007ffefc5b3250 error 6 in 7[56520acd6000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4[527244]: segfault at 0 ip 000055c3836c1a88 sp 00007ffd725f2480 error 6 in 4[55c3836c0000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
12[527561]: segfault at 8 ip 00007f3882736875 sp 00007ffcb789ae70 error 4 in ld-2.28.so[7f3882715000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[529957]: segfault at 5607b5a7bde3 ip 000056079087836a sp 00007ffc4eae3cf8 error 6 in 7[560790873000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 25 73 3a 20 25 73 00 2b 00 68 65 6c 70 00 76 65 72 73 69 6f 6e
8[531545]: segfault at 8 ip 00007f4bb69ae875 sp 00007fffbaf048f0 error 4 in ld-2.28.so[7f4bb698d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
16[537167]: segfault at 0 ip 000055d68a5dd200 sp 00007ffc67f68068 error 6 in 16[55d68a5db000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2[545988]: segfault at 0 ip 000055698b2aeb47 sp 00007fffd4ff2650 error 6 in 17[55698b2aa000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7[546644]: segfault at 0 ip 0000557097462b47 sp 00007ffc7fb225c0 error 6 in 7[55709745e000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4[552156]: segfault at 8 ip 00007fbd683ad875 sp 00007ffef8d7e8d0 error 4 in ld-2.28.so[7fbd6838c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[554431]: segfault at 8 ip 00007f1448dde875 sp 00007ffe43f49320 error 4 in ld-2.28.so[7f1448dbd000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[557529]: segfault at 8 ip 00007f1a74964875 sp 00007ffd9a3bc280 error 4 in ld-2.28.so[7f1a74943000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 10[563636] trap invalid opcode ip:55e69487658a sp:7ffffa52b030 error:0 in 10[55e694871000+7000]
Lustre: 13249:0:(service.c:1628:ptlrpc_at_send_early_reply()) @@@ Could not add any time (5/5), not sending early reply req@ffff8dc3bbc70380 x1879881529503104/t0(0) o101->8f37b97c-00ad-47b2-b5c2-1a6c8feabf78@0@lo:432/0 lens 576/3856 e 24 to 0 dl 1792796252 ref 2 fl Interpret:/600/0 rc 0/0 job:'ls.0' uid:0 gid:0 projid:0
Lustre: 7448:0:(service.c:1628:ptlrpc_at_send_early_reply()) @@@ Could not add any time (5/5), not sending early reply req@ffff8dc3bdcef000 x1879881529449472/t0(0) o36->8f37b97c-00ad-47b2-b5c2-1a6c8feabf78@0@lo:432/0 lens 504/2888 e 24 to 0 dl 1792796252 ref 2 fl Interpret:/200/0 rc 0/0 job:'mrename.0' uid:0 gid:0 projid:4294967295
Lustre: 13214:0:(service.c:1628:ptlrpc_at_send_early_reply()) @@@ Could not add any time (5/5), not sending early reply req@ffff8dc3b3ad6900 x1879881529726720/t0(0) o101->d9b3a234-7075-4017-a1f0-c2dfb3cf313e@0@lo:434/0 lens 576/3856 e 24 to 0 dl 1792796254 ref 2 fl Interpret:/600/0 rc 0/0 job:'ln.0' uid:0 gid:0 projid:0
5[565640]: segfault at 0 ip 000055c32a9bdb47 sp 00007fff94538660 error 6 in 5 (deleted)[55c32a9b9000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6[566615]: segfault at 8 ip 00007fcb0f971875 sp 00007ffc129200d0 error 4 in ld-2.28.so[7fcb0f950000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
4[568251]: segfault at 0 ip 00005567b91c1b47 sp 00007fff8a6efaf0 error 6 in 4[5567b91bd000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Lustre: 362928:0:(client.c:2521:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1792795652/real 1792795652] req@ffff8dc3bdced080 x1879881529449472/t0(0) o36->lustre-MDT0002-mdc-ffff8dc34ae8a800@0@lo:13/10 lens 504/1016 e 24 to 1 dl 1792796257 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mrename.0' uid:0 gid:0 projid:4294967295
Lustre: lustre-MDT0002-mdc-ffff8dc34ae8a800: Connection to lustre-MDT0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: lustre-MDT0002: Client 8f37b97c-00ad-47b2-b5c2-1a6c8feabf78 (at 0@lo) reconnecting
Lustre: lustre-MDT0002-mdc-ffff8dc34ae8a800: Connection restored to 0@lo (at 0@lo)
Lustre: 366168:0:(client.c:2521:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1792795654/real 1792795654] req@ffff8dc3b3ad3100 x1879881529726720/t0(0) o101->lustre-MDT0002-mdc-ffff8dc3ac8b0800@0@lo:12/10 lens 576/1936 e 24 to 1 dl 1792796259 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'ln.0' uid:0 gid:0 projid:0
Lustre: 366168:0:(client.c:2521:ptlrpc_expire_one_request()) Skipped 1 previous similar message
Lustre: lustre-MDT0002: Client d9b3a234-7075-4017-a1f0-c2dfb3cf313e (at 0@lo) reconnecting
Lustre: Skipped 1 previous similar message
8[572464]: segfault at 0 ip 000055569a2edb47 sp 00007ffd1172c370 error 6 in 8[55569a2e9000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
14[580279]: segfault at 0 ip 0000556b6c3b8b47 sp 00007ffef1bfe820 error 6 in 15[556b6c3b4000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
traps: 5[596621] trap invalid opcode ip:564622e04882 sp:7fffb88e99f8 error:0 in 5[564622dfe000+7000]
17[608884]: segfault at 8 ip 00007f7018a49875 sp 00007ffd06b8cd30 error 4 in ld-2.28.so[7f7018a28000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 270129:0:(service.c:1628:ptlrpc_at_send_early_reply()) @@@ Could not add any time (3/3), not sending early reply req@ffff8dc3d915b800 x1879881552698112/t0(0) o36->d9b3a234-7075-4017-a1f0-c2dfb3cf313e@0@lo:536/0 lens 488/2888 e 4 to 0 dl 1792796356 ref 2 fl Interpret:/200/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295
11[609337]: segfault at 8 ip 00007fc0b25c7875 sp 00007ffd5e37cf90 error 4 in ld-2.28.so[7fc0b25a6000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 365748:0:(client.c:2521:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1792795755/real 1792795756] req@ffff8dc39a439f80 x1879881552698112/t0(0) o36->lustre-MDT0001-mdc-ffff8dc3ac8b0800@0@lo:12/10 lens 488/1016 e 4 to 1 dl 1792796360 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:4294967295
Lustre: lustre-MDT0001-mdc-ffff8dc3ac8b0800: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
Lustre: Skipped 2 previous similar messages
Lustre: lustre-MDT0001: Client d9b3a234-7075-4017-a1f0-c2dfb3cf313e (at 0@lo) reconnecting
Lustre: lustre-MDT0001-mdc-ffff8dc3ac8b0800: Connection restored to 0@lo (at 0@lo)
Lustre: Skipped 2 previous similar messages
9[614086]: segfault at 8 ip 00007fb3a06e4875 sp 00007ffd06ca21a0 error 4 in ld-2.28.so[7fb3a06c3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[621258]: segfault at 8 ip 00007f4371f5c875 sp 00007ffd49cf0410 error 4 in ld-2.28.so[7f4371f3b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 16[622604] trap invalid opcode ip:55ecd586e84e sp:7ffe07ad5be0 error:0 in 16[55ecd5869000+7000]
17[622742]: segfault at 555af1ec6000 ip 0000555af1ec6000 sp 00007ffd5f0d5b68 error 14 in 17[555af20c6000+1000]
Code: Unable to access opcode bytes at RIP 0x555af1ec5fd6.
7[624936]: segfault at 8 ip 00007f75ead2c875 sp 00007ffe4dd1f0a0 error 4 in ld-2.28.so[7f75ead0b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[629066]: segfault at 8 ip 00007f0dcc13c875 sp 00007ffcf9545dd0 error 4 in ld-2.28.so[7f0dcc11b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
11[634288]: segfault at 8 ip 00007fae5c361875 sp 00007ffded87ae80 error 4 in ld-2.28.so[7fae5c340000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
16[635181]: segfault at 8 ip 00007ff6fb7cf875 sp 00007fff65d9dae0 error 4 in ld-2.28.so[7ff6fb7ae000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[637871]: segfault at 0 ip 000055b452226b47 sp 00007fff1c8b5fd0 error 6 in 0[55b452222000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0[637855]: segfault at 0 ip 000055f84c1c2b47 sp 00007ffe265b4450 error 6 in 0[55f84c1be000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
14[650329]: segfault at 8 ip 00007f410c94e875 sp 00007ffde43fd930 error 4 in ld-2.28.so[7f410c92d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
ODEBUG: object 000000001fd631ed is on stack 00000000b5d171e3, but NOT annotated.
WARNING: CPU: 2 PID: 35334 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_zfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) zfs(O) spl(O) libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 2 PID: 35334 Comm: mdt00_090 Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: fe ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffa4cee6c33320 EFLAGS: 00010006
RAX: 0000000000000050 RBX: ffffa4cee6c33468 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8dc57229e5a8 RDI: ffff8dc57229e5a8
RBP: ffffa4cee6c33370 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffa4cee6c330f0 R12: ffffffffada8e208
R13: ffffffffac705aa0 R14: 0000000000008e40 R15: ffff8dc36b992898
FS: 0000000000000000(0000) GS:ffff8dc572280000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b84a29008 CR3: 000000015dcc5000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x110c/0x2620 [lod]
? slab_free_freelist_hook.isra.49+0x111/0x230
lod_comp_prep_create+0x1780/0x1c30 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
? lod_sub_declare_create+0xcc/0x150 [lod]
? __osd_sa_xattr_get.isra.8+0x74/0x290 [osd_zfs]
lod_declare_create+0x237/0xc30 [lod]
? osd_xattr_get+0x2d7/0x8d0 [osd_zfs]
dt_declare_create+0x79/0x230 [obdclass]
mdd_declare_create_object_internal+0xb6/0x2f0 [mdd]
mdd_declare_create_object.isra.20+0x5f/0xfb0 [mdd]
mdd_declare_create+0x5a/0x680 [mdd]
mdd_create+0x6d7/0x2550 [mdd]
mdt_reint_open+0x3338/0x3c40 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6ad/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.50+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 94da3ed630080189 ]---
13[656305]: segfault at 558c0bd68000 ip 0000558c0bd68000 sp 00007ffe1cd23ac8 error 14 in 13[558c0bf68000+1000]
Code: Unable to access opcode bytes at RIP 0x558c0bd67fd6.
traps: 16[658179] trap invalid opcode ip:55ba47fc98e0 sp:7ffd7d5197d8 error:0 in 16[55ba47fc4000+7000]
4[661894]: segfault at 8 ip 00007fd4e2336875 sp 00007ffc7d3eb870 error 4 in ld-2.28.so[7fd4e2315000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[666898]: segfault at 0 ip 00005644048e7b47 sp 00007fffe38d4110 error 6 in 13[5644048e3000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5[667408]: segfault at 8 ip 00007f56cbbdb875 sp 00007ffe89b45ac0 error 4 in ld-2.28.so[7f56cbbba000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Link to test (2026-10-23 23:02)
racer test 1: racer on clients: centos-105.localnet DURATION=2700
LustreError: 719236:0:(statahead.c:2054:revalidate_statahead_dentry()) ASSERTION( sai == entry->se_sai ) failed:
LustreError: 719236:0:(statahead.c:2054:revalidate_statahead_dentry()) LBUG
CPU: 11 PID: 719236 Comm: ls Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
Call Trace:
dump_stack+0x99/0xca
lbug_with_loc.cold.4+0xd/0x86 [libcfs]
revalidate_statahead_dentry+0x145d/0x1aa0 [lustre]
? lockref_put_or_lock+0x33/0x50
ll_revalidate_statahead+0xa7/0x400 [lustre]
ll_revalidate_dentry+0x335/0x3c0 [lustre]
lookup_open+0x132/0xb30
do_last+0xa17/0xf90
? nd_jump_root+0xe5/0x160
? path_init+0x437/0x520
path_openat+0xf7/0x500
? getname_flags+0x73/0x320
do_filp_open+0x99/0x140
? do_raw_spin_unlock+0x79/0x190
? _raw_spin_unlock+0x16/0x30
? __alloc_fd+0x12a/0x240
do_sys_openat2+0x2aa/0x410
do_sys_open+0x75/0xa0
__x64_sys_openat+0x28/0x40
do_syscall_64+0xc6/0x3f0
entry_SYSCALL_64_after_hwframe+0x49/0xae
RIP: 0033:0x7f7607530935
Lustre: 6769:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8d3a48e09180 x1875928552849536/t4294967401(0) o101->ce1c3a67-e1ce-40e5-9caf-6a7b17e276a3@0@lo:365/0 lens 376/816 e 0 to 0 dl 1789024960 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
Lustre: 10030:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8d3a093d7380 x1875928553171456/t4294967469(0) o101->ce1c3a67-e1ce-40e5-9caf-6a7b17e276a3@0@lo:367/0 lens 376/864 e 0 to 0 dl 1789024962 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
Lustre: 11563:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8d3a48670e00 x1875928553937792/t4294967762(0) o101->ce1c3a67-e1ce-40e5-9caf-6a7b17e276a3@0@lo:376/0 lens 376/816 e 0 to 0 dl 1789024971 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
Lustre: 10474:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 615, rollback = 7
Lustre: 10474:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 10474:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 10474:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 10474:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10474:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10474:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 510 < left 610, rollback = 7
Lustre: 10474:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 2 previous similar messages
Lustre: 10474:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 10474:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 2 previous similar messages
Lustre: 10474:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 10474:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 2 previous similar messages
Lustre: 10474:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/5, punch: 0/0/0, quota 4/150/0
Lustre: 10474:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 2 previous similar messages
Lustre: 10474:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10474:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 2 previous similar messages
Lustre: 10474:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10474:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 2 previous similar messages
Lustre: 10176:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 615, rollback = 7
Lustre: 10176:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 10176:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 10176:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 10176:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10176:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10474:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 615, rollback = 7
Lustre: 10474:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
Lustre: 10474:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 10474:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 1 previous similar message
Lustre: 10474:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 10474:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 1 previous similar message
Lustre: 10474:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 10474:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 1 previous similar message
Lustre: 10474:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10474:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 1 previous similar message
Lustre: 10474:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10474:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 1 previous similar message
Lustre: 7566:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 516 < left 615, rollback = 7
Lustre: 7566:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 3 previous similar messages
Lustre: 7566:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 7566:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7566:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 7566:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7566:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 7566:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7566:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 7566:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7566:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 7566:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 3 previous similar messages
LustreError: 18138:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a3a3f5800: inode [0x200000404:0x4f:0x0] mdc close failed: rc = -13
Lustre: 7567:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 515 < left 615, rollback = 7
Lustre: 7567:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 3 previous similar messages
Lustre: 7567:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 7567:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7567:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 7567:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7567:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 1/3/0
Lustre: 7567:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7567:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 7567:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 3 previous similar messages
Lustre: 7567:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 7567:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 3 previous similar messages
11[21612]: segfault at 8 ip 00007f681c1d3875 sp 00007ffe10d484b0 error 4 in ld-2.28.so[7f681c1b2000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 19585:0:(out_handler.c:886:out_tx_end()) lustre-MDT0000-osd: error during execution of #2 from /home/green/git/lustre-release/lustre/ptlrpc/../target/out_handler.c:565: rc = -2
LustreError: 19585:0:(out_lib.c:1177:out_tx_index_delete_undo()) lustre-MDT0000-osd: Oops, can not rollback index_delete yet: rc = -524
LustreError: 12561:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 7409:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 8602:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 7409:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 7409:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 3 previous similar messages
8[23996]: segfault at 8 ip 00007f9342914875 sp 00007ffcffa47200 error 4 in ld-2.28.so[7f93428f3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 7566:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 516 < left 615, rollback = 7
Lustre: 7566:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 15 previous similar messages
Lustre: 7566:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 7566:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 15 previous similar messages
Lustre: 7566:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 7566:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 15 previous similar messages
LustreError: 11874:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 11874:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 15 previous similar messages
Lustre: 10176:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 10176:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 17 previous similar messages
Lustre: 10176:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10176:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 17 previous similar messages
Lustre: 10176:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10176:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 17 previous similar messages
LustreError: 6769:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 6769:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 61 previous similar messages
Lustre: 14012:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 514 < left 520, rollback = 2
LustreError: 28781:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x280000403:0x1b9:0x0] mdc close failed: rc = -13
11[29535]: segfault at 8 ip 00007fc65d26a875 sp 00007ffe92998990 error 4 in ld-2.28.so[7fc65d249000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[30064]: segfault at 8 ip 00007feace835875 sp 00007fff08566170 error 4 in ld-2.28.so[7feace814000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[31710]: segfault at 8 ip 00007fd355018875 sp 00007ffd1f30b450 error 4 in ld-2.28.so[7fd354ff7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[32500]: segfault at 8 ip 00007ffbae6a8875 sp 00007ffd1b0d62d0 error 4 in ld-2.28.so[7ffbae687000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[32457]: segfault at 8 ip 00007fe7aa2d4875 sp 00007ffe391b3d70 error 4 in ld-2.28.so[7fe7aa2b3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 13602:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 13602:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 59 previous similar messages
Lustre: 7567:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 516 < left 615, rollback = 7
Lustre: 7567:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 43 previous similar messages
Lustre: 7567:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 7567:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 45 previous similar messages
Lustre: 7567:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 7567:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 45 previous similar messages
7[36459]: segfault at 0 ip 0000559150518200 sp 00007ffe38b1adb8 error 6 in 7[559150516000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8[40129]: segfault at 8 ip 00007fe9cce3e875 sp 00007ffd2266b220 error 4 in ld-2.28.so[7fe9cce1d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 14401:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 1/3/0
Lustre: 14401:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 49 previous similar messages
Lustre: 14401:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 14401:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 49 previous similar messages
Lustre: 14401:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 14401:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 49 previous similar messages
LustreError: 22495:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 22495:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 117 previous similar messages
traps: 15[42532] trap invalid opcode ip:557306223213 sp:7ffdf6855608 error:0 in 15[55730621e000+7000]
4[49092]: segfault at 55e527a12870 ip 000055e527a12870 sp 00007ffc2fe00278 error 7 in 4[55e527a11000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Lustre: 14317:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 515 < left 615, rollback = 7
Lustre: 14317:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 35 previous similar messages
Lustre: 14317:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 14317:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 35 previous similar messages
Lustre: 14317:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 14317:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 35 previous similar messages
4[51309]: segfault at 8 ip 00007f967bae7875 sp 00007ffe7f909300 error 4 in ld-2.28.so[7f967bac6000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
15[53047]: segfault at 8 ip 00007fec4fcf8875 sp 00007ffd01085690 error 4 in ld-2.28.so[7fec4fcd7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[54290]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[54290]: Failed to get EXE, ignoring: No such process
Lustre: 14317:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 1/3/0
Lustre: 14317:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 37 previous similar messages
Lustre: 14317:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 14317:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 37 previous similar messages
Lustre: 14317:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 14317:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 37 previous similar messages
LustreError: 15052:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 15052:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 225 previous similar messages
14[55901]: segfault at 8 ip 00007fe7a6c26875 sp 00007ffecb8fe370 error 4 in ld-2.28.so[7fe7a6c05000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 6781:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 514 < left 520, rollback = 2
Lustre: 6781:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
Lustre: 6782:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 513 < left 520, rollback = 2
Lustre: 6782:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
3[62417]: segfault at 8 ip 00007f1179c19875 sp 00007fff27959720 error 4 in ld-2.28.so[7f1179bf8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 63364:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x200000404:0x61f:0x0] mdc close failed: rc = -13
18[65838]: segfault at 8 ip 00007f3c9d57d875 sp 00007ffce1ee0a80 error 4 in ld-2.28.so[7f3c9d55c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
18[65876]: segfault at 8 ip 00007f8ac252f875 sp 00007ffe6e88afe0 error 4 in ld-2.28.so[7f8ac250e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 65812:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x240000403:0x4ed:0x0] mdc close failed: rc = -13
LustreError: 68728:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x280000403:0x5e1:0x0] mdc close failed: rc = -13
17[71405]: segfault at 8 ip 00007f7288f02875 sp 00007ffefc8fa8b0 error 4 in ld-2.28.so[7f7288ee1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 71515:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a3a3f5800: inode [0x200000404:0x7b5:0x0] mdc close failed: rc = -13
15[73139]: segfault at 8 ip 00007f9ffbff9875 sp 00007ffc7184e9d0 error 4 in ld-2.28.so[7f9ffbfd8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
9[73416]: segfault at 8 ip 00007f698f8d8875 sp 00007fff4bc0ac10 error 4 in ld-2.28.so[7f698f8b7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[78581]: segfault at 8 ip 00007f6319b2b875 sp 00007fff14a10f70 error 4 in ld-2.28.so[7f6319b0a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[79056]: segfault at 8 ip 00007eff26b96875 sp 00007fffd91fe9e0 error 4 in ld-2.28.so[7eff26b75000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
0[79253]: segfault at 8 ip 00007f15a2751875 sp 00007ffcc53cc550 error 4 in ld-2.28.so[7f15a2730000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
6[80867]: segfault at 8 ip 00007f02ce7a1875 sp 00007ffe39c5e150 error 4 in ld-2.28.so[7f02ce780000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 10176:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 515 < left 573, rollback = 7
Lustre: 10176:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 123 previous similar messages
Lustre: 10176:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 10176:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 126 previous similar messages
Lustre: 10176:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 10176:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 126 previous similar messages
LustreError: 85749:0:(statahead.c:2464:start_statahead_thread()) lustre: unsupported statahead pattern 0X0.
9[87841]: segfault at 8 ip 00007fdafa556875 sp 00007ffe6b960590 error 4 in ld-2.28.so[7fdafa535000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 48196:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 48196:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 428 previous similar messages
Lustre: 10418:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 10418:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 132 previous similar messages
Lustre: 10418:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10418:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 132 previous similar messages
Lustre: 10418:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10418:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 132 previous similar messages
traps: 10[92741] trap stack segment ip:55ca9961a556 sp:7ffc31bdf2a8 error:0 in 10[55ca99614000+7000]
LustreError: 6757:0:(ldlm_lockd.c:255:expired_lock_main()) ### lock callback timer expired after 102s: evicting client at 0@lo ns: filter-lustre-OST0002_UUID lock: 00000000cad77962/0xcef6c55fe0573d62 lrc: 3/0,0 mode: PW/PW res: [0x340000401:0x79:0x0].0x0 rrc: 5 type: EXT [0->32767] (req 16384->32767) gid 0 flags: 0x60000400020020 nid: 0@lo remote: 0xcef6c55fe0573d0e expref: 51 pid: 17152 timeout: 461 lvb_type: 0 lru_score: 0 lru_type: 0
LustreError: lustre-OST0002-osc-ffff8d3a3a3f5800: operation ost_setattr to node 0@lo failed: rc = -107
Lustre: lustre-OST0002-osc-ffff8d3a3a3f5800: Connection to lustre-OST0002 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: lustre-OST0002-osc-ffff8d3a3a3f5800: This client was evicted by lustre-OST0002; in progress operations using this service will fail.
Lustre: 3343:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x200000404:0xb0c:0x0]/ may get corrupted (rc -108)
Lustre: 3343:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x200000404:0xb1a:0x0]/ may get corrupted (rc -108)
Lustre: 3353:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x240000404:0x8df:0x0]// may get corrupted (rc -108)
Lustre: 3342:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x280000404:0x91b:0x0]// may get corrupted (rc -108)
Lustre: 3342:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x240000404:0x884:0x0]// may get corrupted (rc -108)
Lustre: 3346:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x240000403:0x70c:0x0]/ may get corrupted (rc -108)
Lustre: 3346:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x200000404:0xb30:0x0]/ may get corrupted (rc -108)
Lustre: 3356:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x280000404:0x959:0x0]/ may get corrupted (rc -108)
Lustre: 3356:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x200000404:0x948:0x0]/ may get corrupted (rc -108)
Lustre: 3355:0:(llite_lib.c:4342:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.123.107@tcp:/lustre/fid: [0x280000404:0x961:0x0]// may get corrupted (rc -108)
LustreError: 99432:0:(ldlm_resource.c:1207:ldlm_resource_complain()) lustre-OST0002-osc-ffff8d3a3a3f5800: namespace resource [0x340000400:0xdf:0x0].0x0 (ffff8d3ad17db400) refcount nonzero (1) after lock cleanup; forcing cleanup.
Lustre: lustre-OST0002-osc-ffff8d3a3a3f5800: Connection restored to 0@lo (at 0@lo)
5[103514]: segfault at 8 ip 00007f2d3158d875 sp 00007ffcb38df3d0 error 4 in ld-2.28.so[7f2d3156c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[103684]: segfault at 8 ip 00007f95f6abb875 sp 00007ffff0779ca0 error 4 in ld-2.28.so[7f95f6a9a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 4[105687] trap invalid opcode ip:55ec9837197e sp:7ffd61c69350 error:0 in 4[55ec9836c000+7000]
3[106519]: segfault at 8 ip 00007fe3336e4875 sp 00007ffc9086ce90 error 4 in ld-2.28.so[7fe3336c3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[111698]: segfault at 8 ip 00007ff5802e9875 sp 00007ffc470582a0 error 4 in ld-2.28.so[7ff5802c8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
11[113590]: segfault at 8 ip 00007f2366c19875 sp 00007fff7a67b480 error 4 in ld-2.28.so[7f2366bf8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
11[113424]: segfault at 8 ip 00007f3ec3b78875 sp 00007fff96ba4b00 error 4 in ld-2.28.so[7f3ec3b57000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 115659:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x200000403:0xdf7:0x0] mdc close failed: rc = -13
0[122680]: segfault at 0 ip 0000000000000000 sp 00007fff3ebcbc98 error 14 in 0[56372dd5d000+7000]
Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
0[122855]: segfault at 0 ip 0000000000000000 sp 00007ffccd931038 error 14 in 0[557603c38000+7000]
Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
3[125233]: segfault at ffffffffc0b60fc0 ip 00005622f2945bb6 sp 00007ffeb3ca58e0 error 7 in 3[5622f2941000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 95 c0 0f b6 c0 f7 d8 5b 5d 41 5c c3 0f 1f 44 00 00 85 c0 75 12
systemd-coredump[126031]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[126031]: Failed to get EXE, ignoring: No such process
11[129201]: segfault at 8 ip 00007fb3928c2875 sp 00007ffd0fbd76a0 error 4 in ld-2.28.so[7fb3928a1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[138833]: segfault at 8 ip 00007f5c8c9ba875 sp 00007ffcb4c2d470 error 4 in ld-2.28.so[7f5c8c999000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[139126]: segfault at 8 ip 00007f1a9e50b875 sp 00007ffdedea8820 error 4 in ld-2.28.so[7f1a9e4ea000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 139126:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a3a3f5800: inode [0x200000403:0x10c0:0x0] mdc close failed: rc = -13
3[139364]: segfault at 8 ip 00007f4173fc8875 sp 00007fff4ae56390 error 4 in ld-2.28.so[7f4173fa7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[139552]: segfault at 8 ip 00007f359c282875 sp 00007ffe3b657390 error 4 in ld-2.28.so[7f359c261000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 0[143034] trap invalid opcode ip:5579086aa996 sp:7ffcba3b70c8 error:0 in 0[5579086a5000+7000]
16[143918]: segfault at 8 ip 00007fd199e6b875 sp 00007ffc5714b830 error 4 in ld-2.28.so[7fd199e4a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 11362:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0001: opcode 7: before 515 < left 615, rollback = 7
Lustre: 11362:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 179 previous similar messages
Lustre: 11362:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 11362:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 179 previous similar messages
Lustre: 11362:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 11362:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 179 previous similar messages
traps: 16[149328] trap invalid opcode ip:55ea43641c28 sp:7ffe11bd6e18 error:0 in 16[55ea4363b000+7000]
traps: 16[149931] trap invalid opcode ip:56499d33dc28 sp:7ffeceb9fb58 error:0 in 16[56499d337000+7000]
5[152360]: segfault at 8 ip 00007fa6d5429875 sp 00007fff8fae67f0 error 4 in ld-2.28.so[7fa6d5408000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 96309:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 96309:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 757 previous similar messages
ODEBUG: object 0000000018002dc4 is on stack 000000003ad0d9fd, but NOT annotated.
WARNING: CPU: 9 PID: 96215 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: loop zfs(O) spl(O) lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_ldiskfs(O) ldiskfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) dm_flakey libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 9 PID: 96215 Comm: mdt00_061 Kdump: loaded Tainted: G O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: de ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffb0ce55023428 EFLAGS: 00010002
RAX: 0000000000000050 RBX: ffffb0ce55023560 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8d3bb245e5a8 RDI: ffff8d3bb245e5a8
RBP: ffffb0ce55023478 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffb0ce550231f8 R12: ffffffffad89c8a8
R13: ffffffffac505aa0 R14: 00000000000174e0 R15: ffff8d39b94e9f00
FS: 0000000000000000(0000) GS:ffff8d3bb2440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffed4123d80 CR3: 000000022261f000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? __irq_work_queue_local+0xc7/0x310
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? __debug_object_init.cold.5+0x74/0x158
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x1164/0x2560 [lod]
? _raw_spin_unlock+0x16/0x30
? do_raw_spin_unlock+0x79/0x190
lod_comp_prep_create+0x1758/0x1ae0 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
lod_declare_xattr_set+0x27c/0x1100 [lod]
dt_declare_xattr_set+0x79/0x230 [obdclass]
mdd_create_data+0x5b5/0x930 [mdd]
mdt_mfd_open+0x146d/0x16b0 [mdt]
mdt_finish_open+0x831/0xe00 [mdt]
mdt_open_by_fid_lock+0x9de/0x1180 [mdt]
mdt_reint_open+0x943/0x3c90 [mdt]
? lu_ucred+0x29/0x40 [obdclass]
? mdt_ucred+0x1d/0x30 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6a0/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.47+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 4b50db97eea640db ]---
Lustre: 48923:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 48923:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 189 previous similar messages
Lustre: 48923:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 48923:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 189 previous similar messages
Lustre: 48923:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 48923:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 189 previous similar messages
0[163003]: segfault at 0 ip 000055bab4008200 sp 00007ffc5f07f808 error 6 in 8[55bab4006000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8[162975]: segfault at 0 ip 0000564ddf770200 sp 00007ffc95123508 error 6 in 8[564ddf76e000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
10[165489]: segfault at 8 ip 00007f36abfba875 sp 00007fff588a03f0 error 4 in ld-2.28.so[7f36abf99000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
6[166922]: segfault at 8 ip 00007f8d361a5875 sp 00007ffd7ff1e3d0 error 4 in ld-2.28.so[7f8d36184000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
17[169023]: segfault at 8 ip 00007f2a85123875 sp 00007ffd3dbe67c0 error 4 in ld-2.28.so[7f2a85102000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[173108]: segfault at 8 ip 00007fe8db208875 sp 00007ffff6c732e0 error 4 in ld-2.28.so[7fe8db1e7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[178409]: segfault at 8 ip 00007f82cb2c2875 sp 00007ffffb276df0 error 4 in ld-2.28.so[7f82cb2a1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 179693:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x200000403:0x1625:0x0] mdc close failed: rc = -13
LustreError: 179693:0:(file.c:251:ll_close_inode_openhandle()) Skipped 2 previous similar messages
traps: 18[180818] general protection fault ip:558ca56dc8fb sp:7ffde75ba738 error:0 in 18[558ca56d6000+7000]
5[185763]: segfault at 8 ip 00007f5b0d005875 sp 00007ffe19815b50 error 4 in ld-2.28.so[7f5b0cfe4000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[185968]: segfault at 8 ip 00007f29aea54875 sp 00007ffed50f78e0 error 4 in ld-2.28.so[7f29aea33000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[187747]: segfault at 8 ip 00007fbd26bcb875 sp 00007ffecc1da4f0 error 4 in ld-2.28.so[7fbd26baa000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[195967]: segfault at 8 ip 00007fba365d5875 sp 00007ffd018dcf50 error 4 in ld-2.28.so[7fba365b4000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
9[210208]: segfault at 8 ip 00007fd2efe8f875 sp 00007fff7e2f44b0 error 4 in ld-2.28.so[7fd2efe6e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 6781:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 514 < left 520, rollback = 2
2[213920]: segfault at 8 ip 00007f8779ed1875 sp 00007ffde831e950 error 4 in ld-2.28.so[7f8779eb0000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[220326]: segfault at 0 ip 0000556936ede200 sp 00007ffe8b0d7318 error 6 in 13[556936edc000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
18[224207]: segfault at 0 ip 000055e171e89200 sp 00007fffd1b09ed8 error 6 in 11[55e171e87000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
15[236402]: segfault at 8 ip 00007fa0f8fff875 sp 00007ffe4696d190 error 4 in ld-2.28.so[7fa0f8fde000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[236786]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[236786]: Failed to get EXE, ignoring: No such process
traps: 12[245228] trap invalid opcode ip:5627309c7b94 sp:7ffd174d5b88 error:0 in 12[5627309c2000+7000]
LustreError: 245066:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x200000404:0x2029:0x0] mdc close failed: rc = -13
LustreError: 245066:0:(file.c:251:ll_close_inode_openhandle()) Skipped 4 previous similar messages
8[248030]: segfault at 8 ip 00007f353cd4d875 sp 00007ffd4ae31dc0 error 4 in ld-2.28.so[7f353cd2c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[254813]: segfault at 8 ip 00007f4fee55b875 sp 00007fff5fabe0f0 error 4 in ld-2.28.so[7f4fee53a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[266052]: segfault at 8 ip 00007fd4d674e875 sp 00007ffd69849fa0 error 4 in ld-2.28.so[7fd4d672d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[266537]: segfault at 8 ip 00007f7a5358b875 sp 00007ffc23a1c8c0 error 4 in ld-2.28.so[7f7a5356a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[266757]: segfault at 8 ip 00007f5f2f43c875 sp 00007ffc209679d0 error 4 in ld-2.28.so[7f5f2f41b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
17[266237]: segfault at 561bdf8bf000 ip 0000561bdf8bf000 sp 00007fffe15ca498 error 14 in 17[561bdfabf000+1000]
17[266183]: segfault at 559579c88000 ip 0000559579c88000 sp 00007fff3c1fed28 error 14
Code: Unable to access opcode bytes at RIP 0x561bdf8befd6.
in 17[559579e88000+1000]
Code: Unable to access opcode bytes at RIP 0x559579c87fd6.
systemd-coredump[267162]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[267162]: Failed to get EXE, ignoring: No such process
systemd-coredump[267161]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[267161]: Failed to get EXE, ignoring: No such process
19[268373]: segfault at 8 ip 00007f7460c1f875 sp 00007ffdddc94180 error 4 in ld-2.28.so[7f7460bfe000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[269119]: segfault at 8 ip 00007fa6109e9875 sp 00007ffc00aa98a0 error 4 in ld-2.28.so[7fa6109c8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[270545]: segfault at 8 ip 00007ff67f23e875 sp 00007ffd6eee7980 error 4 in ld-2.28.so[7ff67f21d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 13[273158] trap int3 ip:5626e43e4861 sp:7ffdc9647530 error:0 in 13[5626e43e2000+7000]
4[274768]: segfault at 8 ip 00007f2ef447d875 sp 00007ffde11a28d0 error 4 in ld-2.28.so[7f2ef445c000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[281477]: segfault at 0 ip 000055c82c6a0b47 sp 00007ffc5c5acbe0 error 6 in 14[55c82c69c000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
11[285261]: segfault at 8 ip 00007f8591ac2875 sp 00007ffc37505e30 error 4 in ld-2.28.so[7f8591aa1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[286593]: segfault at 0 ip 000055634481cb47 sp 00007ffdefffe070 error 6 in 12[556344818000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9[290270]: segfault at 555858ca2000 ip 0000555858ca2000 sp 00007fffa629ca78 error 14 in 9[555858ea2000+1000]
Code: Unable to access opcode bytes at RIP 0x555858ca1fd6.
LustreError: 96341:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 96341:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 1929 previous similar messages
traps: 13[301576] trap invalid opcode ip:56149ca6b792 sp:7ffcaac95be8 error:0 in 13[56149ca66000+7000]
Lustre: 10176:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 7: before 516 < left 615, rollback = 7
Lustre: 10176:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 459 previous similar messages
Lustre: 10176:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 10176:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 461 previous similar messages
Lustre: 10176:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 2/15/0
Lustre: 10176:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 461 previous similar messages
Lustre: 10176:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/615/0, punch: 0/0/0, quota 4/150/0
Lustre: 10176:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 437 previous similar messages
Lustre: 10176:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 10176:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 437 previous similar messages
Lustre: 10176:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 10176:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 437 previous similar messages
systemd-coredump[302453]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[302453]: Failed to get EXE, ignoring: No such process
7[304668]: segfault at 0 ip 000055cb166c5c10 sp 00007ffea0dfaa48 error 6 in 7[55cb166c4000+7000]
Code: 48 89 e5 74 0c 48 8d 3d 6e 5f 20 00 e8 39 fc ff ff e8 64 ff ff ff c6 05 7d 64 20 00 01 5d c3 0f 1f 00 c3 0f 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0[306052]: segfault at 8 ip 00007f883f156875 sp 00007ffdfbac8600 error 4 in ld-2.28.so[7f883f135000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
15[314909]: segfault at 8 ip 00007f3f5a4b8875 sp 00007ffdb5a3b680 error 4 in ld-2.28.so[7f3f5a497000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[316224]: segfault at 8 ip 00007f2c8bc14875 sp 00007fffe33f7270 error 4 in ld-2.28.so[7f2c8bbf3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[322498]: segfault at 0 ip 000055f19a6fdb47 sp 00007ffddb0dd360 error 6 in 3[55f19a6f9000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3[322830]: segfault at 8 ip 00007f4d0acd2875 sp 00007fffabfb6690 error 4 in ld-2.28.so[7f4d0acb1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[323865]: segfault at e ip 0000559c413cdad8 sp 00007ffeb96dbc58 error 6 in 7[559c413c7000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 4b 0e 10 61 0a 0e 08 41 0b 00 00 24 00 00 00 20 0a 00 00 54 dd
LustreError: 323409:0:(mdc_request.c:1507:mdc_read_page()) lustre-MDT0001-mdc-ffff8d3a24b89000: [0x240000403:0x1:0x0] lock enqueue fails: rc = -4
LustreError: 323409:0:(statahead.c:1814:is_first_dirent()) lustre: reading dir [0x240000403:0x1:0x0] at 0 stat_pid = 325383 : rc = -4
5[330833]: segfault at 8 ip 00007f6fdbab7875 sp 00007ffe4f6bba00 error 4 in ld-2.28.so[7f6fdba96000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
11[337041]: segfault at 0 ip 00005574b75c8c10 sp 00007ffe5209bcb8 error 6 in 11[5574b75c7000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
15[338037]: segfault at 8 ip 00007f6db6dec875 sp 00007ffd4b99c620 error 4 in ld-2.28.so[7f6db6dcb000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
6[342393]: segfault at 8 ip 00007ff10c383875 sp 00007ffe5ed55a80 error 4 in ld-2.28.so[7ff10c362000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
6[342491]: segfault at 8 ip 00007f0e90abf875 sp 00007ffff1d78040 error 4 in ld-2.28.so[7f0e90a9e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 346299:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a3a3f5800: inode [0x280000404:0x2162:0x0] mdc close failed: rc = -13
LustreError: 346299:0:(file.c:251:ll_close_inode_openhandle()) Skipped 8 previous similar messages
14[347212]: segfault at 0 ip 000055a707392b47 sp 00007ffd4552e580 error 6 in 14[55a70738e000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
16[346950]: segfault at 8 ip 00007f46584e2875 sp 00007ffdbcbefdc0 error 4 in ld-2.28.so[7f46584c1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[348391]: segfault at 8 ip 00007fb129de6875 sp 00007ffff44deea0 error 4 in ld-2.28.so[7fb129dc5000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[348803]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[348803]: Failed to get EXE, ignoring: No such process
12[356433]: segfault at 0 ip 0000562263915b47 sp 00007ffdba6dec70 error 6 in 12[562263911000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1[360274]: segfault at 8 ip 00007fb97ad12875 sp 00007fff19481a60 error 4 in ld-2.28.so[7fb97acf1000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
4[361639]: segfault at 8 ip 00007fd38b7ef875 sp 00007ffef4810410 error 4 in ld-2.28.so[7fd38b7ce000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[362978]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[362978]: Failed to get EXE, ignoring: No such process
0[363499]: segfault at 8 ip 00007fe226f89875 sp 00007ffd7080afe0 error 4 in ld-2.28.so[7fe226f68000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[364113]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[364113]: Failed to get EXE, ignoring: No such process
15[364753]: segfault at 8 ip 00007fe54b52f875 sp 00007ffdaa0202e0 error 4 in ld-2.28.so[7fe54b50e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
15[364707]: segfault at 8 ip 00007f1e677c4875 sp 00007fff8f338430 error 4 in ld-2.28.so[7f1e677a3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[368204]: segfault at 0 ip 00005608d82b265b sp 00007fffe01b7a68 error 6 in 3[5608d82ac000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 7a 52 00 01 78 10 <01> 1b 0c 07 08 90 01 0e 50 10 00 00 00 1c 00 00 00 ee b1 ff ff 05
1[377338]: segfault at 55aee7e58000 ip 000055aee7e58000 sp 00007ffe7280f7c8 error 14 in 1[55aee8058000+1000]
Code: Unable to access opcode bytes at RIP 0x55aee7e57fd6.
0[384758]: segfault at 8 ip 00007f06a024b875 sp 00007ffe5f61ede0 error 4 in ld-2.28.so[7f06a022a000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
16[385122]: segfault at 8 ip 00007ff3ec0bf875 sp 00007fff13c031b0 error 4 in ld-2.28.so[7ff3ec09e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 6782:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0002: opcode 2: before 513 < left 520, rollback = 2
Lustre: 6782:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
1[396678]: segfault at 8 ip 00007fd249efb875 sp 00007fff4a7fdc30 error 4 in ld-2.28.so[7fd249eda000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
5[404895]: segfault at 8 ip 00007f5e206cc875 sp 00007ffe0d0016d0 error 4 in ld-2.28.so[7f5e206ab000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[404839]: segfault at 8 ip 00007fd2d00ff875 sp 00007fff99de36f0 error 4 in ld-2.28.so[7fd2d00de000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 15[413910] trap int3 ip:55efa5c46861 sp:7fffdfc8e0e0 error:0 in 11[55efa5c44000+7000]
9[415195]: segfault at 8 ip 00007f60ca617875 sp 00007ffc7674abd0 error 4 in ld-2.28.so[7f60ca5f6000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[424508]: segfault at 8 ip 00007f636e623875 sp 00007ffe199992a0 error 4 in ld-2.28.so[7f636e602000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 14012:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 515 < left 520, rollback = 2
2[434994]: segfault at 0 ip 000055effeeacb47 sp 00007ffc0fa58710 error 6 in 2[55effeea8000+7000]
Code: Unable to access opcode bytes at RIP 0x55effeeacb1d.
Lustre: 70228:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 514 < left 520, rollback = 2
Lustre: 70228:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
0[440654]: segfault at 8 ip 00007f5b9d5ff875 sp 00007fff0679e270 error 4 in ld-2.28.so[7f5b9d5de000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[448204]: segfault at 8 ip 00007f51083b7875 sp 00007ffd1747e550 error 4 in ld-2.28.so[7f5108396000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[457886]: segfault at 0 ip 000055a741e09b47 sp 00007fff2d82f7b0 error 6 in 2[55a741e05000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
18[463176]: segfault at 0 ip 0000557283cb4b47 sp 00007ffec7fb27a0 error 6 in 18[557283cb0000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5[469113]: segfault at 8 ip 00007f7d6f7a6875 sp 00007fffddcf2fe0 error 4 in ld-2.28.so[7f7d6f785000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 13830:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 13830:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 1852 previous similar messages
14[470542]: segfault at 8 ip 00007f26ddfaa875 sp 00007fffb6971040 error 4 in ld-2.28.so[7f26ddf89000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 7567:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0003: opcode 7: before 515 < left 531, rollback = 7
Lustre: 7567:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 377 previous similar messages
Lustre: 7567:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 7567:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 382 previous similar messages
Lustre: 7567:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 7567:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 382 previous similar messages
Lustre: 7567:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/531/0, punch: 0/0/0, quota 1/3/0
Lustre: 7567:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 382 previous similar messages
Lustre: 7567:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 7567:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 382 previous similar messages
Lustre: 7567:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 7567:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 382 previous similar messages
9[478784]: segfault at 0 ip 000055b64688b200 sp 00007ffe5441b9d8 error 6 in 9[55b646889000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4[479057]: segfault at 8 ip 00007fa186478875 sp 00007ffde1f2e940 error 4 in ld-2.28.so[7fa186457000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 10949:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 515 < left 520, rollback = 2
Lustre: 10949:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
8[490905]: segfault at 6242e8e9 ip 00005618c78a2105 sp 00007ffd3a2e1e88 error 4 in 8[5618c789c000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e0 b1 ff ff c0 <02> 00 00 00 0e 10 46 0e 18 4a 0f 0b 77 08 80 00 3f 1a 3a 2a 33 24
LustreError: 495446:0:(file.c:251:ll_close_inode_openhandle()) lustre-clilmv-ffff8d3a24b89000: inode [0x280000404:0x32c0:0x0] mdc close failed: rc = -13
0[495296]: segfault at 8 ip 00007fc5c7778875 sp 00007fff4451f800 error 4
LustreError: 495446:0:(file.c:251:ll_close_inode_openhandle()) Skipped 7 previous similar messages
in ld-2.28.so[7fc5c7757000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
12[498393]: segfault at 8 ip 00007fdeae89f875 sp 00007ffd25c32110 error 4 in ld-2.28.so[7fdeae87e000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[498206]: segfault at 8 ip 00007fda614ae875 sp 00007ffd65e3e3f0 error 4 in ld-2.28.so[7fda6148d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[499107]: Failed to get EXE, ignoring: No such process
6[499661]: segfault at 8 ip 00007ffa4721c875 sp 00007ffca68189e0 error 4 in ld-2.28.so[7ffa471fb000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[500876]: segfault at 8 ip 00007ff2d08fa875 sp 00007fff8bcc3a00 error 4 in ld-2.28.so[7ff2d08d9000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Lustre: 6782:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 513 < left 520, rollback = 2
Lustre: 6782:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message
4[520394]: segfault at 8 ip 00007f82b1170875 sp 00007ffc9c807fe0 error 4 in ld-2.28.so[7f82b114f000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[520938]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[520938]: Failed to get EXE, ignoring: No such process
Lustre: 6781:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 516 < left 520, rollback = 2
LustreError: 527325:0:(file.c:6224:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000404:0x4a68:0x0] error: rc = -4
11[531364]: segfault at 8 ip 00007f21ae647875 sp 00007ffdcf2d5d20 error 4 in ld-2.28.so[7f21ae626000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 16[533132] general protection fault ip:5562297eeaa4 sp:7ffd8de3dd08 error:0 in 16[5562297e9000+7000]
3[533133]: segfault at 8 ip 00007efeb45f8875 sp 00007fffa8019cc0 error 4 in ld-2.28.so[7efeb45d7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 16[533358] general protection fault ip:557ca7ffaaa4 sp:7fff71e8b0a8 error:0 in 16[557ca7ff5000+7000]
1[538411]: segfault at 563503a4b000 ip 0000563503a4b000 sp 00007ffe8b993b88 error 14 in 1[563503c4b000+1000]
Code: Unable to access opcode bytes at RIP 0x563503a4afd6.
systemd-coredump[539186]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[539186]: Failed to get EXE, ignoring: No such process
7[543430]: segfault at 8 ip 00007f9bf2f7c875 sp 00007ffde508b000 error 4 in ld-2.28.so[7f9bf2f5b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
18[545059]: segfault at 8 ip 00007f86598cb875 sp 00007fff314fb360 error 4 in ld-2.28.so[7f86598aa000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[547070]: segfault at 8 ip 00007fc09b231875 sp 00007ffdea9b6f60 error 4 in ld-2.28.so[7fc09b210000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 9[550882] trap invalid opcode ip:555f21175592 sp:7fff1001cf28 error:0 in 9[555f21170000+7000]
19[561399]: segfault at 8 ip 00007f8b64bd8875 sp 00007fff06601200 error 4 in ld-2.28.so[7f8b64bb7000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[568093]: segfault at 0 ip 000055d301258b47 sp 00007ffd5b53a630 error 6 in 1 (deleted)[55d301254000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
12[569064]: segfault at 8 ip 00007f5a2cd5a875 sp 00007fffcb8348a0 error 4 in ld-2.28.so[7f5a2cd39000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
traps: 16[571991] general protection fault ip:55b9be86a2b0 sp:7ffd6eb85510 error:0 in 16[55b9be865000+7000]
traps: 4[573185] trap invalid opcode ip:5594c44980f1 sp:7ffcaa2d10c8 error:0 in 4[5594c4492000+7000]
traps: 4[573638] trap invalid opcode ip:559afd8b20f1 sp:7fff71116b98 error:0 in 4[559afd8ac000+7000]
1[575181]: segfault at 8 ip 00007fefeff68875 sp 00007ffc71c15bb0 error 4 in ld-2.28.so[7fefeff47000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[579708]: segfault at 8 ip 00007f25b142a875 sp 00007ffe7ff1c330 error 4 in ld-2.28.so[7f25b1409000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[581007]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[581007]: Failed to get EXE, ignoring: No such process
traps: 14[581136] trap invalid opcode ip:563caa5b5902 sp:7ffd2c530d68 error:0 in 14 (deleted)[563caa5b0000+7000]
2[581395]: segfault at 8 ip 00007f22438ac875 sp 00007ffda7b05930 error 4 in ld-2.28.so[7f224388b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
2[583025]: segfault at 8 ip 00007fc74eacb875 sp 00007ffcc2e2c1c0 error 4 in ld-2.28.so[7fc74eaaa000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
18[591321]: segfault at 8 ip 00007f0e637ed875 sp 00007ffe3af12f10 error 4 in ld-2.28.so[7f0e637cc000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
10[595426]: segfault at 8 ip 00007effffe74875 sp 00007ffede475130 error 4 in ld-2.28.so[7effffe53000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
16[596524]: segfault at 8 ip 00007fb6274fe875 sp 00007ffc627e3f80 error 4 in ld-2.28.so[7fb6274dd000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[597306]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[597306]: Failed to get EXE, ignoring: No such process
2[608298]: segfault at 8 ip 00007fc966299875 sp 00007ffdba1fd8e0 error 4 in ld-2.28.so[7fc966278000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[608781]: segfault at 8 ip 00007f04fe119875 sp 00007ffe046e7140 error 4 in ld-2.28.so[7f04fe0f8000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[608933]: segfault at 8 ip 00007fec7158a875 sp 00007ffeacf003f0 error 4 in ld-2.28.so[7fec71569000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
3[611076]: segfault at 8 ip 00007fb752047875 sp 00007ffcf8d327a0 error 4 in ld-2.28.so[7fb752026000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
LustreError: 15007:0:(mdt_open.c:1324:mdt_cross_open()) lustre-MDT0001: [0x240000404:0xe8:0x0] doesn't exist!: rc = -14
LustreError: 15007:0:(mdt_open.c:1324:mdt_cross_open()) Skipped 2070 previous similar messages
Lustre: 178319:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0002: opcode 7: before 515 < left 543, rollback = 7
Lustre: 178319:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 463 previous similar messages
Lustre: 178319:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0
Lustre: 178319:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 467 previous similar messages
Lustre: 178319:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/1, xattr_set: 2/15/0
Lustre: 178319:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 467 previous similar messages
Lustre: 178319:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/543/0, punch: 0/0/0, quota 4/150/0
Lustre: 178319:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 467 previous similar messages
Lustre: 178319:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 0/0/0
Lustre: 178319:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 467 previous similar messages
Lustre: 178319:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0
Lustre: 178319:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 467 previous similar messages
7[637354]: segfault at 8 ip 00007f9e68780875 sp 00007ffeb9591f90 error 4 in ld-2.28.so[7f9e6875f000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
19[637967]: segfault at 8 ip 00007f2d375c4875 sp 00007fffb48995b0 error 4 in ld-2.28.so[7f2d375a3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
11[640052]: segfault at 8 ip 00007f396f942875 sp 00007fffd5c039a0 error 4 in ld-2.28.so[7f396f921000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[648445]: segfault at 0 ip 000055a94d1a4b47 sp 00007ffd6ebb8bc0 error 6 in 8[55a94d1a0000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ODEBUG: object 00000000c14ba016 is on stack 00000000cafe4d44, but NOT annotated.
WARNING: CPU: 8 PID: 54222 at lib/debugobjects.c:368 __debug_object_init.cold.5+0x35/0x158
Modules linked in: loop zfs(O) spl(O) lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) osd_ldiskfs(O) ldiskfs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) crc32_generic lnet(O) dm_flakey libcfs(O) virtio_balloon i2c_piix4 pcspkr rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver ata_generic ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic
CPU: 8 PID: 54222 Comm: mdt00_055 Kdump: loaded Tainted: G W O -------- - - 4.18.0rocky8.10-debug #2
Hardware name: Red Hat KVM, BIOS 1.16.0-4.module+el8.9.0+1408+7b966129 04/01/2014
RIP: 0010:__debug_object_init.cold.5+0x35/0x158
Code: de ab 48 83 05 66 63 e3 02 01 89 05 9c 6b e3 02 65 48 8b 04 25 00 dd 01 00 48 8b 50 18 e8 74 4c 97 ff 48 83 05 5e 63 e3 02 01 <0f> 0b 48 83 05 5c 63 e3 02 01 48 83 05 5c 63 e3 02 01 e9 6a ee ff
RSP: 0018:ffffb0ce5cb13338 EFLAGS: 00010002
RAX: 0000000000000050 RBX: ffffb0ce5cb13470 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff8d3bb241e5a8 RDI: ffff8d3bb241e5a8
RBP: ffffb0ce5cb13388 R08: 0000000000000000 R09: c0000000ffff7fff
R10: 0000000000000001 R11: ffffb0ce5cb13108 R12: ffffffffad8ba888
R13: ffffffffac505aa0 R14: 00000000000354c0 R15: ffff8d3a0c8dc0f0
FS: 0000000000000000(0000) GS:ffff8d3bb2400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000556b4258d008 CR3: 00000001889ac000 CR4: 00000000000006e0
Call Trace:
? show_regs.cold.10+0x22/0x2f
? __warn+0xd0/0x160
? __debug_object_init.cold.5+0x35/0x158
? report_bug+0x120/0x150
? __irq_work_queue_local+0xc7/0x310
? do_error_trap+0xc3/0x140
? do_invalid_op+0x4b/0x60
? __debug_object_init.cold.5+0x35/0x158
? invalid_op+0x14/0x20
? __debug_object_init.cold.5+0x35/0x158
? __debug_object_init.cold.5+0x74/0x158
? lod_comp_is_ost_used+0x90/0x90 [lod]
debug_object_init+0x26/0x30
init_timer_key+0x2d/0x120
lod_ost_alloc_qos+0x1164/0x2560 [lod]
? vsnprintf+0x304/0x7f0
lod_comp_prep_create+0x1758/0x1ae0 [lod]
lod_prepare_create+0x202/0x450 [lod]
lod_declare_striped_create+0x270/0xd40 [lod]
? lod_sub_declare_create+0xcc/0x150 [lod]
lod_declare_create+0x237/0xc30 [lod]
dt_declare_create+0x79/0x230 [obdclass]
mdd_declare_create_object_internal+0xb6/0x2f0 [mdd]
mdd_declare_create_object.isra.20+0x5a/0xed0 [mdd]
? lod_get_default_lov_striping+0x11d/0xad0 [lod]
mdd_declare_create+0x5a/0x680 [mdd]
mdd_create+0x71a/0x2590 [mdd]
mdt_reint_open+0x35e4/0x3c90 [mdt]
? old_init_ucred_common+0x1a1/0x830 [mdt]
mdt_reint_rec+0x144/0x2c0 [mdt]
mdt_reint_internal+0x6a0/0xdd0 [mdt]
mdt_intent_open+0x187/0x5c0 [mdt]
mdt_intent_opc.constprop.47+0x153/0xfa0 [mdt]
? mdt_intent_fixup_resent+0x2e0/0x2e0 [mdt]
mdt_intent_policy+0x151/0x680 [mdt]
ldlm_lock_enqueue+0x449/0xcf0 [ptlrpc]
? cfs_hash_rw_unlock+0x15/0x40 [obdclass]
? cfs_hash_add+0xf6/0x2a0 [obdclass]
ldlm_handle_enqueue+0xcac/0x2260 [ptlrpc]
? lustre_swab_ldlm_lock_desc+0x90/0x90 [ptlrpc]
tgt_enqueue+0xd7/0x320 [ptlrpc]
tgt_handle_request0+0x139/0xb00 [ptlrpc]
tgt_request_handle+0x59b/0x2010 [ptlrpc]
ptlrpc_server_handle_request+0x443/0x13c0 [ptlrpc]
ptlrpc_main+0xcfc/0x1410 [ptlrpc]
? ptlrpc_wait_event+0x6d0/0x6d0 [ptlrpc]
kthread+0x1de/0x210
? set_kthread_struct+0x70/0x70
ret_from_fork+0x1f/0x30
---[ end trace 4b50db97eea640dc ]---
2[653273]: segfault at 0 ip 0000555d9b76fa88 sp 00007ffdff79e750 error 6 in 2[555d9b76e000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
11[653793]: segfault at 8 ip 00007fa84340a875 sp 00007ffec1bb77b0 error 4 in ld-2.28.so[7fa8433e9000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
13[658077]: segfault at 8 ip 00007fa419191875 sp 00007ffe9b932100 error 4 in ld-2.28.so[7fa419170000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[658871]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[658871]: Failed to get EXE, ignoring: No such process
10[659664]: segfault at 8 ip 00007f5a2546c875 sp 00007ffe3c602310 error 4 in ld-2.28.so[7f5a2544b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
9[664451]: segfault at 0 ip 0000560f739bfa88 sp 00007ffe9d8bc5a0 error 6 in 12[560f739be000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8[666356]: segfault at 8 ip 00007fefe421a875 sp 00007ffe6f2edbd0 error 4 in ld-2.28.so[7fefe41f9000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[665964]: segfault at 8 ip 00007ff35b13c875 sp 00007ffeaf5c9cf0 error 4 in ld-2.28.so[7ff35b11b000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
8[666635]: segfault at 8 ip 00007f3f1b0b5875 sp 00007ffdb1a4f760 error 4 in ld-2.28.so[7f3f1b094000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
systemd-coredump[666713]: Failed to get COMM, falling back to the command line: No such process
systemd-coredump[666713]: Failed to get EXE, ignoring: No such process
9[668582]: segfault at 8 ip 00007f3cf1c67875 sp 00007ffd97eeb130 error 4 in ld-2.28.so[7f3cf1c46000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
10[671184]: segfault at 0 ip 000055ca03039b47 sp 00007fff4c687180 error 6 in 10[55ca03035000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5[678242]: segfault at 8 ip 00007f20b431d875 sp 00007ffc41905630 error 4 in ld-2.28.so[7f20b42fc000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
7[697461]: segfault at 558c6bb03000 ip 0000558c6bb03000 sp 00007fffd3c65958 error 14 in 7[558c6bd03000+1000]
Code: Unable to access opcode bytes at RIP 0x558c6bb02fd6.
0[700256]: segfault at 8 ip 00007f135176a875 sp 00007ffef251fc70 error 4 in ld-2.28.so[7f1351749000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
12[701478]: segfault at 0 ip 0000561f40684b47 sp 00007ffda370b920 error 6 in 12[561f40680000+7000]
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9[702848]: segfault at 8 ip 00007f000b23e875 sp 00007ffe2d6c3bd0 error 4 in ld-2.28.so[7f000b21d000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[711172]: segfault at 8 ip 00007fa41b8a5875 sp 00007ffc72990ce0 error 4 in ld-2.28.so[7fa41b884000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[715334]: segfault at 8 ip 00007f3525ec6875 sp 00007fff7903fe60 error 4 in ld-2.28.so[7f3525ea5000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
1[715580]: segfault at 8 ip 00007f7dc30e4875 sp 00007ffe9801e010 error 4 in ld-2.28.so[7f7dc30c3000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
9[717401]: segfault at 8 ip 00007f1b77a06875 sp 00007ffe24189bb0 error 4 in ld-2.28.so[7f1b779e5000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
14[718444]: segfault at 8 ip 00007f9ee4020875 sp 00007ffc22d19010 error 4 in ld-2.28.so[7f9ee3fff000+2f000]
Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 24 f6 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48
Link to test (2026-09-10 08:06)
Return to new crashes list