Edit new crash report

Reason	Crashing Function	Where to cut Backtrace	Reports Count
ASSERTION( ext->oe_state == OES_CACHE ) failed	osc_extent_holes_prune	osc_extent_holes_prune osc_lock_flush osc_dlm_blocking_ast0 osc_ldlm_blocking_ast ldlm_cancel_callback ldlm_cli_cancel_local ldlm_cli_cancel osc_ldlm_blocking_ast ldlm_handle_bl_callback ldlm_bl_thread_blwi ldlm_bl_thread_main kthread ret_from_fork	3

Failing Test	Full Crash	Messages before crash	Comment
racer test 1: racer on clients: trevis-155vm29.trevis.whamcloud.com,trevis-155vm30 DURATION=300	LustreError: 62161:0:(osc_cache.c:1410:osc_extent_holes_prune()) ASSERTION( ext->oe_state == OES_CACHE ) failed: LustreError: 62161:0:(osc_cache.c:1410:osc_extent_holes_prune()) LBUG CPU: 0 PID: 62161 Comm: ldlm_bl_04 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osc_extent_holes_prune+0x8e7/0x8f0 [osc] osc_lock_flush+0x9e/0x320 [osc] osc_dlm_blocking_ast0.constprop.0+0xdf/0x310 [osc] osc_ldlm_blocking_ast+0x6f/0x190 [osc] ldlm_cancel_callback+0x7a/0x2b0 [ptlrpc] ldlm_cli_cancel_local+0xa6/0x490 [ptlrpc] ldlm_cli_cancel+0xab/0x540 [ptlrpc] osc_ldlm_blocking_ast+0x14b/0x190 [osc] ldlm_handle_bl_callback+0x10a/0x3f0 [ptlrpc] ldlm_bl_thread_blwi.constprop.0+0x7c/0x320 [ptlrpc] ldlm_bl_thread_main+0x4fb/0x5b0 [ptlrpc] ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_ldlm_bl_thread_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK>	Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA LustreError: 62955:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521a4913800: inode [0x240000bee:0x6:0x0] mdc close failed: rc = -2 Lustre: dir [0x2c0000bec:0xc7:0x0] stripe 3 readdir failed: -2, directory is partially accessed! LustreError: 64436:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff4be521a4913800: dir page locate: [0x280000bd1:0x7b:0x0] at 0: rc -5 17[65353]: segfault at 8 ip 00007fe6f69f8480 sp 00007ffe8923f458 error 4 in ld-linux-x86-64.so.2[7fe6f69dc000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 64130:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be8:0x72:0x0]: rc = -5 LustreError: 64130:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 Lustre: dir [0x240000bed:0xb7:0x0] stripe 4 readdir failed: -2, directory is partially accessed! Lustre: Skipped 20 previous similar messages 10[434845]: segfault at 8 ip 00007f50182a6480 sp 00007ffe533c0f78 error 4 in ld-linux-x86-64.so.2[7f501828a000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 70549:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521a4913800: inode [0x240000bed:0x60:0x0] mdc close failed: rc = -2 LustreError: 435327:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be6:0x96:0x0]: rc = -5 LustreError: 435327:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 435327:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 435327:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 2 previous similar messages LustreError: 436373:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bef:0x2d:0x0]: rc = -5 LustreError: 436373:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 1 previous similar message LustreError: 436373:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 436373:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 1 previous similar message Lustre: dir [0x2c0000bea:0x1b0:0x0] stripe 1 readdir failed: -2, directory is partially accessed! LustreError: 437031:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be5:0xf8:0x0]: rc = -5 LustreError: 437031:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 34 previous similar messages LustreError: 437265:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521a4913800: inode [0x2c0000bec:0x236:0x0] mdc close failed: rc = -2 LustreError: 437475:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 437475:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 36 previous similar messages Lustre: dir [0x280000be7:0xc1:0x0] stripe 6 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages LustreError: 437459:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0003-mdc-ff4be521dca87800: dir page locate: [0x2c0000bd2:0x27:0x0] at 0: rc -5 LustreError: 437459:0:(mdc_request.c:1510:mdc_read_page()) Skipped 14 previous similar messages LustreError: 441900:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bef:0x1b:0x0]: rc = -5 LustreError: 441900:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 5 previous similar messages LustreError: 441900:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 441900:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 3 previous similar messages LustreError: 443057:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521dca87800: inode [0x2c0000bea:0x1bd:0x0] mdc close failed: rc = -2 LustreError: 443057:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message 5[445217]: segfault at 8 ip 00007f84da8d1480 sp 00007ffebc7e3058 error 4 in ld-linux-x86-64.so.2[7f84da8b5000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 446407:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521dca87800: inode [0x200000bea:0x24f:0x0] mdc close failed: rc = -2 LustreError: 446569:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bed:0x27d:0x0]: rc = -5 LustreError: 446569:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 28 previous similar messages LustreError: 446569:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 446569:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 28 previous similar messages LustreError: 445981:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521a4913800: cannot apply new layout on [0x280000be5:0xf8:0x0] : rc = -5 LustreError: 445981:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be5:0xf8:0x0] error -5. Lustre: dir [0x280000be6:0x2dd:0x0] stripe 4 readdir failed: -2, directory is partially accessed! Lustre: Skipped 14 previous similar messages LustreError: 452891:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521dca87800: cannot apply new layout on [0x2c0000bec:0x41d:0x0] : rc = -5 LustreError: 452891:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x2c0000bec:0x41d:0x0] error -5. LustreError: 452186:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521a4913800: inode [0x200000beb:0x2ca:0x0] mdc close failed: rc = -2 LustreError: 452186:0:(file.c:248:ll_close_inode_openhandle()) Skipped 3 previous similar messages LustreError: 55486:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 11 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 LustreError: 456366:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bed:0x27d:0x0]: rc = -5 LustreError: 456366:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 153 previous similar messages LustreError: 456366:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 456366:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 153 previous similar messages 8[457523]: segfault at 8 ip 00007f4db2c1b480 sp 00007fff62814bb8 error 4 in ld-linux-x86-64.so.2[7f4db2bff000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 1[459888]: segfault at 8 ip 00007fa718cb2480 sp 00007ffd5f66ba88 error 4 in ld-linux-x86-64.so.2[7fa718c96000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 Lustre: dir [0x280000be8:0x4e9:0x0] stripe 4 readdir failed: -2, directory is partially accessed! 19[460775]: segfault at 1c ip 000055c8cd10db90 sp 00007fff464bf030 error 6 in 19[55c8cd10d000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 19[460779]: segfault at 0 ip 000055d8dc41ca00 sp 00007ffc2ec05920 error 6 in 19[55d8dc41c000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 LustreError: 462178:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0001-mdc-ff4be521a4913800: dir page locate: [0x240000bd3:0x5f:0x0] at 0: rc -5 LustreError: 462178:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message LustreError: 464207:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0001-mdc-ff4be521dca87800: dir page locate: [0x240000bd3:0x5f:0x0] at 0: rc -5 LustreError: 464207:0:(mdc_request.c:1510:mdc_read_page()) Skipped 40 previous similar messages 17[464989]: segfault at 7f5830fbccd0 ip 000055af5fe310c0 sp 00007ffd177e6248 error 7 in 17[55af5fe2e000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 LustreError: 463724:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521dca87800: inode [0x280000be5:0x258:0x0] mdc close failed: rc = -2 LustreError: 463724:0:(file.c:248:ll_close_inode_openhandle()) Skipped 2 previous similar messages LustreError: 466410:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0001-mdc-ff4be521a4913800: dir page locate: [0x240000bd3:0x5f:0x0] at 0: rc -5 LustreError: 466410:0:(mdc_request.c:1510:mdc_read_page()) Skipped 19 previous similar messages Lustre: dir [0x200000beb:0x505:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 128 previous similar messages LustreError: 468421:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521dca87800: cannot apply new layout on [0x2c0000bec:0x603:0x0] : rc = -5 LustreError: 468421:0:(lov_object.c:1346:lov_layout_change()) Skipped 2 previous similar messages LustreError: 468421:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x2c0000bec:0x603:0x0] error -5. LustreError: 471199:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521a4913800: cannot apply new layout on [0x240000bef:0x4c4:0x0] : rc = -5 LustreError: 471199:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bef:0x4c4:0x0] error -5. LustreError: 471891:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x240000bee:0x5c5:0x0] get parent: rc = -116 LustreError: 473132:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521a4913800: cannot apply new layout on [0x200000beb:0x68d:0x0] : rc = -5 LustreError: 473132:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x200000beb:0x68d:0x0] error -5. LustreError: 475902:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000bec:0x603:0x0]: rc = -5 LustreError: 475902:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 73 previous similar messages LustreError: 475902:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 475902:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 73 previous similar messages LustreError: 481678:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x2c0000beb:0x663:0x0] get parent: rc = -116 12[482392]: segfault at 2250 ip 0000000000002250 sp 00007fffa3f8caa8 error 14 in 12[563e9366b000+2000] likely on CPU 0 (core 0, socket 0) Code: Unable to access opcode bytes at 0x2226. LustreError: 483835:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521dca87800: cannot apply new layout on [0x2c0000beb:0x7c0:0x0] : rc = -5 LustreError: 483835:0:(lov_object.c:1346:lov_layout_change()) Skipped 3 previous similar messages LustreError: 485270:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff4be521a4913800: inode [0x200000bea:0x7b5:0x0] mdc close failed: rc = -2 LustreError: 485270:0:(file.c:248:ll_close_inode_openhandle()) Skipped 12 previous similar messages 15[487985]: segfault at 1c ip 0000563264404b90 sp 00007ffe1d72c750 error 6 in 15[563264404000+4000] likely on CPU 0 (core 0, socket 0) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Lustre: dir [0x240000bed:0xa5f:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 74 previous similar messages LustreError: 497584:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff4be521dca87800: cannot apply new layout on [0x280000be5:0x9e0:0x0] : rc = -5 LustreError: 497584:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be5:0x9e0:0x0] error -5. LustreError: 497584:0:(vvp_io.c:1931:vvp_io_init()) Skipped 2 previous similar messages 0[498528]: segfault at 8 ip 00007f9a6f2e9480 sp 00007ffca9717368 error 4 in ld-linux-x86-64.so.2[7f9a6f2cd000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 12[499137]: segfault at 8 ip 00007f338f15b480 sp 00007fff61344a78 error 4 in ld-linux-x86-64.so.2[7f338f13f000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 2[500755]: segfault at 0 ip 000055de81833a00 sp 00007ffc656f8910 error 6 in 2[55de81833000+4000] likely on CPU 0 (core 0, socket 0) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 LustreError: 55426:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 7[511855]: segfault at 1c ip 0000555e510fab90 sp 00007ffff6912de0 error 6 in 7[555e510fa000+4000] likely on CPU 0 (core 0, socket 0) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13[513496]: segfault at 8 ip 00007f72e5122480 sp 00007ffc434be818 error 4 in ld-linux-x86-64.so.2[7f72e5106000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 515904:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bed:0xd0b:0x0]: rc = -5 LustreError: 515904:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 52 previous similar messages LustreError: 515904:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 515904:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 52 previous similar messages LustreError: 515635:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0001-mdc-ff4be521a4913800: dir page locate: [0x240000bed:0xc5a:0x0] at 0: rc -5 LustreError: 515635:0:(mdc_request.c:1510:mdc_read_page()) Skipped 88 previous similar messages LustreError: 62161:0:(osc_cache.c:1410:osc_extent_holes_prune()) extent ff4be521acebddc0@{[2048 -> 3471/3471], [3\|1\|-\|active\|wio\|ff4be521ecaf06b8], [0\|0\|-\|-\|0000000000000000\|0\|0000000000000000]} LustreError: 62161:0:(osc_cache.c:1268:osc_extent_tree_dump0()) Dump object ff4be521ecaf06b8 extents at osc_extent_holes_prune:1410, mppr: 1024. LustreError: 62161:0:(osc_cache.c:1274:osc_extent_tree_dump0()) extent ff4be521acebddc0@{[2048 -> 3471/3471], [3\|1\|-\|active\|wio\|ff4be521ecaf06b8], [0\|0\|-\|-\|0000000000000000\|0\|0000000000000000]} in tree 1.	Link to test
racer test 1: racer on clients: onyx-155vm40.onyx.whamcloud.com,onyx-155vm41 DURATION=300	LustreError: 58927:0:(osc_cache.c:1410:osc_extent_holes_prune()) ASSERTION( ext->oe_state == OES_CACHE ) failed: LustreError: 58927:0:(osc_cache.c:1410:osc_extent_holes_prune()) LBUG CPU: 1 PID: 58927 Comm: ldlm_bl_01 Kdump: loaded Tainted: G OE ------- --- 5.14.0-570.62.1.el9_6.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osc_extent_holes_prune+0x8e7/0x8f0 [osc] osc_lock_flush+0x9e/0x320 [osc] osc_dlm_blocking_ast0.constprop.0+0xdf/0x310 [osc] osc_ldlm_blocking_ast+0x6f/0x190 [osc] ldlm_cancel_callback+0x7a/0x2b0 [ptlrpc] ldlm_cli_cancel_local+0xa6/0x490 [ptlrpc] ldlm_cli_cancel+0xab/0x540 [ptlrpc] ? asm_common_interrupt+0x22/0x40 osc_ldlm_blocking_ast+0x14b/0x190 [osc] ldlm_handle_bl_callback+0x10a/0x3f0 [ptlrpc] ldlm_bl_thread_blwi.constprop.0+0x7c/0x320 [ptlrpc] ldlm_bl_thread_main+0x4fb/0x5b0 [ptlrpc] ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_ldlm_bl_thread_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK>	Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA 10[66635]: segfault at 8 ip 00007f9e365ec480 sp 00007ffef63eb618 error 4 in ld-linux-x86-64.so.2[7f9e365d0000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 57765:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x33:0x0]: rc = -5 LustreError: 57765:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 5 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 LustreError: 68786:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x240000bec:0xa5:0x0] mdc close failed: rc = -13 LustreError: 68985:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x200000bec:0x182:0x0] mdc close failed: rc = -2 LustreError: 72252:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x2c0000be8:0x38:0x0] mdc close failed: rc = -2 LustreError: 72311:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be8:0x59:0x0]: rc = -5 LustreError: 72311:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 Lustre: dir [0x280000be7:0xbc:0x0] stripe 4 readdir failed: -2, directory is partially accessed! LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bec:0x108:0x0]: rc = -5 LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 2 previous similar messages Lustre: dir [0x200000bea:0x182:0x0] stripe 4 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x240000bee:0x81:0x0] stripe 4 readdir failed: -2, directory is partially accessed! LustreError: 74105:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0003-mdc-ff1de51f427a2800: dir page locate: [0x2c0000be8:0x9:0x0] at 0: rc -5 LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x96:0x0]: rc = -5 LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 2 previous similar messages LustreError: 76371:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x280000be8:0x4c:0x0] mdc close failed: rc = -2 LustreError: 77479:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bee:0xc5:0x0]: rc = -5 LustreError: 77479:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 1 previous similar message LustreError: 77479:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 77479:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 1 previous similar message 0[79314]: segfault at 8 ip 00007f5837473480 sp 00007ffed22eda98 error 4 in ld-linux-x86-64.so.2[7f5837457000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 78341:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x280000be7:0x5e:0x0] mdc close failed: rc = -2 LustreError: 78341:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message Lustre: dir [0x280000be8:0x16d:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages LustreError: 83055:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bed:0x190:0x0]: rc = -5 LustreError: 83055:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 5 previous similar messages LustreError: 83055:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 83055:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 5 previous similar messages LustreError: 83105:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x240000bef:0x18d:0x0] : rc = -5 LustreError: 83105:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bef:0x18d:0x0] error -5. LustreError: 83830:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x200000bed:0x1a8:0x0] mdc close failed: rc = -2 LustreError: 83830:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message LustreError: 85817:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f427a2800: cannot apply new layout on [0x280000be8:0x59:0x0] : rc = -5 LustreError: 85817:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be8:0x59:0x0] error -5. LustreError: 87033:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f427a2800: cannot apply new layout on [0x280000be8:0x59:0x0] : rc = -5 LustreError: 87033:0:(lov_object.c:1346:lov_layout_change()) Skipped 5 previous similar messages 10[86657]: segfault at 8 ip 00007fa9e0b8f480 sp 00007ffd719e1978 error 4 in ld-linux-x86-64.so.2[7fa9e0b73000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 19[87940]: segfault at 561f0942c040 ip 0000561f0942c040 sp 00007ffd4a5e9358 error 7 in 19[561f09429000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Lustre: dir [0x2c0000be9:0x312:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message 14[88870]: segfault at 8 ip 00007fa51041c480 sp 00007ffcdbac9648 error 4 in ld-linux-x86-64.so.2[7fa510400000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 5[90533]: segfault at 8 ip 00007fdb072c7480 sp 00007ffeecf63bd8 error 4 in ld-linux-x86-64.so.2[7fdb072ab000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 90988:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x200000bed:0x186:0x0] : rc = -5 LustreError: 90988:0:(lov_object.c:1346:lov_layout_change()) Skipped 3 previous similar messages LustreError: 90988:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x200000bed:0x186:0x0] error -5. LustreError: 93343:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x30b:0x0]: rc = -5 LustreError: 93343:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 53 previous similar messages LustreError: 93343:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 93343:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 53 previous similar messages LustreError: 91523:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x200000beb:0x27d:0x0] get parent: rc = -2 10[96463]: segfault at 1c ip 000055bff9593b90 sp 00007ffdbad0feb0 error 6 in 10[55bff9593000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 LustreError: 94668:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x2c0000bea:0x309:0x0] mdc close failed: rc = -2 LustreError: 94668:0:(file.c:248:ll_close_inode_openhandle()) Skipped 9 previous similar messages Lustre: dir [0x200000bec:0x3ec:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 6 previous similar messages LustreError: 98921:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0000-mdc-ff1de51f5ec16800: dir page locate: [0x200000bd0:0x53:0x0] at 0: rc -5 LustreError: 98921:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message LustreError: 103738:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x2c0000bea:0x4b4:0x0] : rc = -5 LustreError: 103738:0:(lov_object.c:1346:lov_layout_change()) Skipped 1 previous similar message LustreError: 103738:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x2c0000bea:0x4b4:0x0] error -5. LustreError: 103738:0:(vvp_io.c:1931:vvp_io_init()) Skipped 1 previous similar message LustreError: 104978:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0001-mdc-ff1de51f5ec16800: dir page locate: [0x240000bef:0x4e1:0x0] at 0: rc -5 LustreError: 104978:0:(mdc_request.c:1510:mdc_read_page()) Skipped 33 previous similar messages traps: 4[105688] trap invalid opcode ip:55c17ebaa66b sp:7ffe848c42c8 error:0 in 4[55c17ebaa000+4000] 10[107288]: segfault at 8 ip 00007ff85799f480 sp 00007ffd62468cf8 error 4 in ld-linux-x86-64.so.2[7ff857983000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 7[107557]: segfault at 8 ip 00007f61471a2480 sp 00007ffceac52ae8 error 4 in ld-linux-x86-64.so.2[7f6147186000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 93304:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 6[111446]: segfault at 0 ip 0000000000000000 sp 00007fff4304c7c8 error 14 in 8[55914ceaa000+2000] likely on CPU 1 (core 0, socket 1) Code: Unable to access opcode bytes at 0xffffffffffffffd6. 13[111798]: segfault at 0 ip 0000000000000000 sp 00007ffd8c3bb428 error 14 in 8[5555af340000+2000] likely on CPU 1 (core 0, socket 1) Code: Unable to access opcode bytes at 0xffffffffffffffd6. Lustre: dir [0x240000bed:0x826:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 45 previous similar messages LustreError: 111980:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x48e:0x0]: rc = -5 LustreError: 111980:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 39 previous similar messages LustreError: 111980:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 111980:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 39 previous similar messages LustreError: 115244:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x280000be8:0x688:0x0] : rc = -5 LustreError: 115244:0:(lov_object.c:1346:lov_layout_change()) Skipped 2 previous similar messages LustreError: 115244:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be8:0x688:0x0] error -5. LustreError: 115244:0:(vvp_io.c:1931:vvp_io_init()) Skipped 1 previous similar message 12[116604]: segfault at 8 ip 00007fdcc371e480 sp 00007ffd3f37c618 error 4 in ld-linux-x86-64.so.2[7fdcc3702000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 57728:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 10 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 LustreError: 116636:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x280000be6:0x601:0x0] mdc close failed: rc = -2 LustreError: 116636:0:(file.c:248:ll_close_inode_openhandle()) Skipped 13 previous similar messages 15[118472]: segfault at 8 ip 00007fc7d5503480 sp 00007ffc2854af68 error 4 in ld-linux-x86-64.so.2[7fc7d54e7000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 10 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 2 previous similar messages LustreError: 117795:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bec:0x72e:0x0] error -5. 9[124524]: segfault at 0 ip 0000558ec9ca1a00 sp 00007ffc264fda40 error 6 in 9[558ec9ca1000+4000] likely on CPU 0 (core 0, socket 0) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 11 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 2 previous similar messages 9[125966]: segfault at 8 ip 00007fc448c90480 sp 00007ffc2df2d588 error 4 in ld-linux-x86-64.so.2[7fc448c74000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 124989:0:(llite_lib.c:1896:ll_update_lsm_md()) lustre: [0x200000bed:0x8af:0x0] dir layout mismatch: LustreError: 124989:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=4 count=4 index=0 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool= LustreError: 124989:0:(lustre_lmv.h:166:lmv_stripe_object_dump()) stripe[0] [0x200000bd0:0x93:0x0] LustreError: 124989:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=1 count=5 index=0 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=4 migrate_hash=fnv_1a_64:2 pool= LustreError: 124995:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=4 count=4 index=0 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool= LustreError: 124995:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=1 count=5 index=0 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=4 migrate_hash=fnv_1a_64:2 pool= LustreError: 127026:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x280000be9:0x7ce:0x0] : rc = -5 LustreError: 127026:0:(lov_object.c:1346:lov_layout_change()) Skipped 6 previous similar messages 18[128509]: segfault at 2b90 ip 0000000000002b90 sp 00007ffcc467d9c0 error 14 likely on CPU 0 (core 0, socket 0) Code: Unable to access opcode bytes at 0x2b66. Lustre: dir [0x200000bec:0xb11:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages 12[134065]: segfault at 8 ip 00007fe5ef8b7480 sp 00007fff5bd4d048 error 4 in ld-linux-x86-64.so.2[7fe5ef89b000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 0[134249]: segfault at 0 ip 000055d64c4d0a00 sp 00007ffca8fd6990 error 6 in 0[55d64c4d0000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 18[138805]: segfault at 8 ip 00007fd88718f480 sp 00007ffdf8ccd738 error 4 in ld-linux-x86-64.so.2[7fd887173000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 137242:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bef:0xb20:0x0] error -5. LustreError: 137242:0:(vvp_io.c:1931:vvp_io_init()) Skipped 2 previous similar messages LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 4 previous similar messages LustreError: 153344:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000beb:0xc9d:0x0]: rc = -5 LustreError: 153344:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 284 previous similar messages LustreError: 153344:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 153344:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 284 previous similar messages LustreError: 154935:0:(statahead.c:2408:start_statahead_thread()) lustre: invalid pattern 0X0. LustreError: 157055:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x200000bea:0x1051:0x0] mdc close failed: rc = -2 LustreError: 157055:0:(file.c:248:ll_close_inode_openhandle()) Skipped 18 previous similar messages LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 2 previous similar messages LustreError: 163675:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x200000bed:0xedf:0x0] get parent: rc = -2 LustreError: 163675:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) Skipped 19 previous similar messages LustreError: 162815:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff1de51f427a2800: dir page locate: [0x280000be9:0xcab:0x0] at 0: rc -5 LustreError: 162815:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message 12[166687]: segfault at 8 ip 00007fcba37c5480 sp 00007ffe0dc4a138 error 4 in ld-linux-x86-64.so.2[7fcba37a9000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 1[167954]: segfault at 1c ip 00005645b500bb90 sp 00007ffe306b0d60 error 6 in 4[5645b500b000+4000] likely on CPU 0 (core 0, socket 0) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1[167731]: segfault at 1c ip 0000560a94585b90 sp 00007ffdf2398c40 error 6 in 4[560a94585000+4000] likely on CPU 1 (core 0, socket 1) Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 5[172878]: segfault at 8 ip 00007f15d5ba2480 sp 00007ffd89524ec8 error 4 in ld-linux-x86-64.so.2[7f15d5b86000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 172368:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f427a2800: cannot apply new layout on [0x280000be6:0xd8e:0x0] : rc = -5 LustreError: 172368:0:(lov_object.c:1346:lov_layout_change()) Skipped 2 previous similar messages LustreError: 172368:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be6:0xd8e:0x0] error -5. LustreError: 172368:0:(vvp_io.c:1931:vvp_io_init()) Skipped 1 previous similar message LustreError: 174192:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x280000be6:0xe68:0x0] get parent: rc = -116 LustreError: 174192:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) Skipped 3 previous similar messages Lustre: dir [0x2c0000beb:0xf70:0x0] stripe 4 readdir failed: -2, directory is partially accessed! Lustre: Skipped 14 previous similar messages LustreError: 174460:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff1de51f5ec16800: dir page locate: [0x280000be8:0xe25:0x0] at 0: rc -5 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 1 previous similar message 7[180070]: segfault at 8 ip 00007f2c263cf480 sp 00007ffed1ac4a78 error 4 in ld-linux-x86-64.so.2[7f2c263b3000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 16[180390]: segfault at 8 ip 00007f8180bf2480 sp 00007ffc6e76e1b8 error 4 in ld-linux-x86-64.so.2[7f8180bd6000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 10[181774]: segfault at 8 ip 00007f247252e480 sp 00007ffd7c39d4a8 error 4 in ld-linux-x86-64.so.2[7f2472512000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 183741:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0000-mdc-ff1de51f5ec16800: dir page locate: [0x200000bd0:0xde:0x0] at 0: rc -5 LustreError: 183741:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message 15[190026]: segfault at 8 ip 00007f1236949480 sp 00007ffd32ed3208 error 4 in ld-linux-x86-64.so.2[7f123692d000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 16[192585]: segfault at 8 ip 00007fa752e20480 sp 00007ffcbe7887e8 error 4 in ld-linux-x86-64.so.2[7fa752e04000+27000] likely on CPU 0 (core 0, socket 0) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 189586:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x240000bef:0x1430:0x0] get parent: rc = -116 LustreError: 197800:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff1de51f427a2800: dir page locate: [0x280000be6:0x10b8:0x0] at 0: rc -5 LustreError: 197800:0:(mdc_request.c:1510:mdc_read_page()) Skipped 53 previous similar messages 13[201215]: segfault at 8 ip 00007f68449d9480 sp 00007fffc206a088 error 4 in ld-linux-x86-64.so.2[7f68449bd000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 16[207302]: segfault at 8 ip 00007f0c94089480 sp 00007fffe65ef988 error 4 in ld-linux-x86-64.so.2[7f0c9406d000+27000] likely on CPU 1 (core 0, socket 1) Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48 LustreError: 207621:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x240000bef:0x13f2:0x0] get parent: rc = -116 LustreError: 207621:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) Skipped 29 previous similar messages LustreError: 211824:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x240000bec:0x15cc:0x0] : rc = -5 LustreError: 211824:0:(lov_object.c:1346:lov_layout_change()) Skipped 17 previous similar messages LustreError: 211824:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bec:0x15cc:0x0] error -5. LustreError: 211824:0:(vvp_io.c:1931:vvp_io_init()) Skipped 7 previous similar messages LustreError: 213410:0:(llite_lib.c:1896:ll_update_lsm_md()) lustre: [0x240000bee:0x1892:0x0] dir layout mismatch: LustreError: 213410:0:(llite_lib.c:1896:ll_update_lsm_md()) Skipped 1 previous similar message LustreError: 213410:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=2 count=3 index=1 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool= LustreError: 213410:0:(lustre_lmv.h:166:lmv_stripe_object_dump()) stripe[0] [0x240000bd1:0x11a:0x0] LustreError: 213410:0:(lustre_lmv.h:166:lmv_stripe_object_dump()) Skipped 17 previous similar messages LustreError: 213410:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=1 count=4 index=1 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=3 migrate_hash=fnv_1a_64:2 pool= LustreError: 215088:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0000-mdc-ff1de51f427a2800: dir page locate: [0x200000bed:0x194e:0x0] at 0: rc -5 LustreError: 215088:0:(mdc_request.c:1510:mdc_read_page()) Skipped 22 previous similar messages LustreError: 58927:0:(osc_cache.c:1410:osc_extent_holes_prune()) extent ff1de51fb0548d10@{[0 -> 1023/1023], [2\|0\|-\|rpc\|wio\|ff1de51fb35eeec8], [0\|0\|-\|-\|0000000000000000\|0\|ff1de51fb8b48000]} LustreError: 58927:0:(osc_cache.c:1268:osc_extent_tree_dump0()) Dump object ff1de51fb35eeec8 extents at osc_extent_holes_prune:1410, mppr: 1024. LustreError: 58927:0:(osc_cache.c:1274:osc_extent_tree_dump0()) extent ff1de51fb0548d10@{[0 -> 1023/1023], [2\|0\|-\|rpc\|wio\|ff1de51fb35eeec8], [0\|0\|-\|-\|0000000000000000\|0\|ff1de51fb8b48000]} in tree 1.	Link to test
sanity test 398b: DIO and buffer IO race	LustreError: 1120642:0:(osc_cache.c:1407:osc_extent_holes_prune()) ASSERTION( ext->oe_state == OES_CACHE ) failed: LustreError: 1120642:0:(osc_cache.c:1407:osc_extent_holes_prune()) LBUG CPU: 0 PID: 1120642 Comm: ldlm_bl_03 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osc_extent_holes_prune+0x5b3/0x620 [osc] osc_lock_flush+0x9e/0x320 [osc] osc_dlm_blocking_ast0.constprop.0+0xdf/0x310 [osc] osc_ldlm_blocking_ast+0x6f/0x190 [osc] ldlm_cancel_callback+0x7a/0x2b0 [ptlrpc] ldlm_cli_cancel_local+0xa6/0x490 [ptlrpc] ldlm_cli_cancel+0xab/0x540 [ptlrpc] osc_ldlm_blocking_ast+0x14b/0x190 [osc] ldlm_handle_bl_callback+0x10a/0x3f0 [ptlrpc] ldlm_bl_thread_blwi.constprop.0+0x7c/0x320 [ptlrpc] ldlm_bl_thread_main+0x4fb/0x5b0 [ptlrpc] ? __pfx_autoremove_wake_function+0x10/0x10 ? __pfx_ldlm_bl_thread_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK>		Link to test

Failing Test

Full Crash

Messages before crash

Comment

racer test 1: racer on clients: trevis-155vm29.trevis.whamcloud.com,trevis-155vm30 DURATION=300

LustreError: 62161:0:(osc_cache.c:1410:osc_extent_holes_prune()) ASSERTION( ext->oe_state == OES_CACHE ) failed:
LustreError: 62161:0:(osc_cache.c:1410:osc_extent_holes_prune()) LBUG
CPU: 0 PID: 62161 Comm: ldlm_bl_04 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x34/0x48
lbug_with_loc.cold+0x5/0x43 [libcfs]
osc_extent_holes_prune+0x8e7/0x8f0 [osc]
osc_lock_flush+0x9e/0x320 [osc]
osc_dlm_blocking_ast0.constprop.0+0xdf/0x310 [osc]
osc_ldlm_blocking_ast+0x6f/0x190 [osc]
ldlm_cancel_callback+0x7a/0x2b0 [ptlrpc]
ldlm_cli_cancel_local+0xa6/0x490 [ptlrpc]
ldlm_cli_cancel+0xab/0x540 [ptlrpc]
osc_ldlm_blocking_ast+0x14b/0x190 [osc]
ldlm_handle_bl_callback+0x10a/0x3f0 [ptlrpc]
ldlm_bl_thread_blwi.constprop.0+0x7c/0x320 [ptlrpc]
ldlm_bl_thread_main+0x4fb/0x5b0 [ptlrpc]
? __pfx_autoremove_wake_function+0x10/0x10
? __pfx_ldlm_bl_thread_main+0x10/0x10 [ptlrpc]
kthread+0xdd/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x29/0x50
</TASK>

Link to test

racer test 1: racer on clients: onyx-155vm40.onyx.whamcloud.com,onyx-155vm41 DURATION=300

LustreError: 58927:0:(osc_cache.c:1410:osc_extent_holes_prune()) ASSERTION( ext->oe_state == OES_CACHE ) failed:
LustreError: 58927:0:(osc_cache.c:1410:osc_extent_holes_prune()) LBUG
CPU: 1 PID: 58927 Comm: ldlm_bl_01 Kdump: loaded Tainted: G OE ------- --- 5.14.0-570.62.1.el9_6.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x34/0x48
lbug_with_loc.cold+0x5/0x43 [libcfs]
osc_extent_holes_prune+0x8e7/0x8f0 [osc]
osc_lock_flush+0x9e/0x320 [osc]
osc_dlm_blocking_ast0.constprop.0+0xdf/0x310 [osc]
osc_ldlm_blocking_ast+0x6f/0x190 [osc]
ldlm_cancel_callback+0x7a/0x2b0 [ptlrpc]
ldlm_cli_cancel_local+0xa6/0x490 [ptlrpc]
ldlm_cli_cancel+0xab/0x540 [ptlrpc]
? asm_common_interrupt+0x22/0x40
osc_ldlm_blocking_ast+0x14b/0x190 [osc]
ldlm_handle_bl_callback+0x10a/0x3f0 [ptlrpc]
ldlm_bl_thread_blwi.constprop.0+0x7c/0x320 [ptlrpc]
ldlm_bl_thread_main+0x4fb/0x5b0 [ptlrpc]
? __pfx_autoremove_wake_function+0x10/0x10
? __pfx_ldlm_bl_thread_main+0x10/0x10 [ptlrpc]
kthread+0xdd/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x29/0x50
</TASK>

Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA
10[66635]: segfault at 8 ip 00007f9e365ec480 sp 00007ffef63eb618 error 4 in ld-linux-x86-64.so.2[7f9e365d0000+27000] likely on CPU 0 (core 0, socket 0)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 57765:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x33:0x0]: rc = -5
LustreError: 57765:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 5 [0x0:0x0:0x0] inode@0000000000000000: rc = -5
LustreError: 68786:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x240000bec:0xa5:0x0] mdc close failed: rc = -13
LustreError: 68985:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x200000bec:0x182:0x0] mdc close failed: rc = -2
LustreError: 72252:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x2c0000be8:0x38:0x0] mdc close failed: rc = -2
LustreError: 72311:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be8:0x59:0x0]: rc = -5
LustreError: 72311:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
Lustre: dir [0x280000be7:0xbc:0x0] stripe 4 readdir failed: -2, directory is partially accessed!
LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bec:0x108:0x0]: rc = -5
LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 2 previous similar messages
LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 2 previous similar messages
Lustre: dir [0x200000bea:0x182:0x0] stripe 4 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
Lustre: dir [0x240000bee:0x81:0x0] stripe 4 readdir failed: -2, directory is partially accessed!
LustreError: 74105:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0003-mdc-ff1de51f427a2800: dir page locate: [0x2c0000be8:0x9:0x0] at 0: rc -5
LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x96:0x0]: rc = -5
LustreError: 73290:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 2 previous similar messages
LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 73290:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 2 previous similar messages
LustreError: 76371:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x280000be8:0x4c:0x0] mdc close failed: rc = -2
LustreError: 77479:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000bee:0xc5:0x0]: rc = -5
LustreError: 77479:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 1 previous similar message
LustreError: 77479:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 77479:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 1 previous similar message
0[79314]: segfault at 8 ip 00007f5837473480 sp 00007ffed22eda98 error 4 in ld-linux-x86-64.so.2[7f5837457000+27000] likely on CPU 0 (core 0, socket 0)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 78341:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x280000be7:0x5e:0x0] mdc close failed: rc = -2
LustreError: 78341:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message
Lustre: dir [0x280000be8:0x16d:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 3 previous similar messages
LustreError: 83055:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bed:0x190:0x0]: rc = -5
LustreError: 83055:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 5 previous similar messages
LustreError: 83055:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 83055:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 5 previous similar messages
LustreError: 83105:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x240000bef:0x18d:0x0] : rc = -5
LustreError: 83105:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bef:0x18d:0x0] error -5.
LustreError: 83830:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x200000bed:0x1a8:0x0] mdc close failed: rc = -2
LustreError: 83830:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message
LustreError: 85817:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f427a2800: cannot apply new layout on [0x280000be8:0x59:0x0] : rc = -5
LustreError: 85817:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be8:0x59:0x0] error -5.
LustreError: 87033:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f427a2800: cannot apply new layout on [0x280000be8:0x59:0x0] : rc = -5
LustreError: 87033:0:(lov_object.c:1346:lov_layout_change()) Skipped 5 previous similar messages
10[86657]: segfault at 8 ip 00007fa9e0b8f480 sp 00007ffd719e1978 error 4 in ld-linux-x86-64.so.2[7fa9e0b73000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
19[87940]: segfault at 561f0942c040 ip 0000561f0942c040 sp 00007ffd4a5e9358 error 7 in 19[561f09429000+4000] likely on CPU 1 (core 0, socket 1)
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Lustre: dir [0x2c0000be9:0x312:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 1 previous similar message
14[88870]: segfault at 8 ip 00007fa51041c480 sp 00007ffcdbac9648 error 4 in ld-linux-x86-64.so.2[7fa510400000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
5[90533]: segfault at 8 ip 00007fdb072c7480 sp 00007ffeecf63bd8 error 4 in ld-linux-x86-64.so.2[7fdb072ab000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 90988:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x200000bed:0x186:0x0] : rc = -5
LustreError: 90988:0:(lov_object.c:1346:lov_layout_change()) Skipped 3 previous similar messages
LustreError: 90988:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x200000bed:0x186:0x0] error -5.
LustreError: 93343:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x30b:0x0]: rc = -5
LustreError: 93343:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 53 previous similar messages
LustreError: 93343:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 93343:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 53 previous similar messages
LustreError: 91523:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x200000beb:0x27d:0x0] get parent: rc = -2
10[96463]: segfault at 1c ip 000055bff9593b90 sp 00007ffdbad0feb0 error 6 in 10[55bff9593000+4000] likely on CPU 1 (core 0, socket 1)
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
LustreError: 94668:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f427a2800: inode [0x2c0000bea:0x309:0x0] mdc close failed: rc = -2
LustreError: 94668:0:(file.c:248:ll_close_inode_openhandle()) Skipped 9 previous similar messages
Lustre: dir [0x200000bec:0x3ec:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 6 previous similar messages
LustreError: 98921:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0000-mdc-ff1de51f5ec16800: dir page locate: [0x200000bd0:0x53:0x0] at 0: rc -5
LustreError: 98921:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message
LustreError: 103738:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x2c0000bea:0x4b4:0x0] : rc = -5
LustreError: 103738:0:(lov_object.c:1346:lov_layout_change()) Skipped 1 previous similar message
LustreError: 103738:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x2c0000bea:0x4b4:0x0] error -5.
LustreError: 103738:0:(vvp_io.c:1931:vvp_io_init()) Skipped 1 previous similar message
LustreError: 104978:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0001-mdc-ff1de51f5ec16800: dir page locate: [0x240000bef:0x4e1:0x0] at 0: rc -5
LustreError: 104978:0:(mdc_request.c:1510:mdc_read_page()) Skipped 33 previous similar messages
traps: 4[105688] trap invalid opcode ip:55c17ebaa66b sp:7ffe848c42c8 error:0 in 4[55c17ebaa000+4000]
10[107288]: segfault at 8 ip 00007ff85799f480 sp 00007ffd62468cf8 error 4 in ld-linux-x86-64.so.2[7ff857983000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
7[107557]: segfault at 8 ip 00007f61471a2480 sp 00007ffceac52ae8 error 4 in ld-linux-x86-64.so.2[7f6147186000+27000] likely on CPU 0 (core 0, socket 0)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 93304:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1
6[111446]: segfault at 0 ip 0000000000000000 sp 00007fff4304c7c8 error 14 in 8[55914ceaa000+2000] likely on CPU 1 (core 0, socket 1)
Code: Unable to access opcode bytes at 0xffffffffffffffd6.
13[111798]: segfault at 0 ip 0000000000000000 sp 00007ffd8c3bb428 error 14 in 8[5555af340000+2000] likely on CPU 1 (core 0, socket 1)
Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Lustre: dir [0x240000bed:0x826:0x0] stripe 1 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 45 previous similar messages
LustreError: 111980:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x48e:0x0]: rc = -5
LustreError: 111980:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 39 previous similar messages
LustreError: 111980:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 111980:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 39 previous similar messages
LustreError: 115244:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x280000be8:0x688:0x0] : rc = -5
LustreError: 115244:0:(lov_object.c:1346:lov_layout_change()) Skipped 2 previous similar messages
LustreError: 115244:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be8:0x688:0x0] error -5.
LustreError: 115244:0:(vvp_io.c:1931:vvp_io_init()) Skipped 1 previous similar message
12[116604]: segfault at 8 ip 00007fdcc371e480 sp 00007ffd3f37c618 error 4 in ld-linux-x86-64.so.2[7fdcc3702000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 57728:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 10 [0x0:0x0:0x0] inode@0000000000000000: rc = -5
LustreError: 116636:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x280000be6:0x601:0x0] mdc close failed: rc = -2
LustreError: 116636:0:(file.c:248:ll_close_inode_openhandle()) Skipped 13 previous similar messages
15[118472]: segfault at 8 ip 00007fc7d5503480 sp 00007ffc2854af68 error 4 in ld-linux-x86-64.so.2[7fc7d54e7000+27000] likely on CPU 0 (core 0, socket 0)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 10 [0x0:0x0:0x0] inode@0000000000000000: rc = -5
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 2 previous similar messages
LustreError: 117795:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bec:0x72e:0x0] error -5.
9[124524]: segfault at 0 ip 0000558ec9ca1a00 sp 00007ffc264fda40 error 6 in 9[558ec9ca1000+4000] likely on CPU 0 (core 0, socket 0)
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 11 [0x0:0x0:0x0] inode@0000000000000000: rc = -5
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 2 previous similar messages
9[125966]: segfault at 8 ip 00007fc448c90480 sp 00007ffc2df2d588 error 4 in ld-linux-x86-64.so.2[7fc448c74000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 124989:0:(llite_lib.c:1896:ll_update_lsm_md()) lustre: [0x200000bed:0x8af:0x0] dir layout mismatch:
LustreError: 124989:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=4 count=4 index=0 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool=
LustreError: 124989:0:(lustre_lmv.h:166:lmv_stripe_object_dump()) stripe[0] [0x200000bd0:0x93:0x0]
LustreError: 124989:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=1 count=5 index=0 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=4 migrate_hash=fnv_1a_64:2 pool=
LustreError: 124995:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=4 count=4 index=0 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool=
LustreError: 124995:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=1 count=5 index=0 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=4 migrate_hash=fnv_1a_64:2 pool=
LustreError: 127026:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x280000be9:0x7ce:0x0] : rc = -5
LustreError: 127026:0:(lov_object.c:1346:lov_layout_change()) Skipped 6 previous similar messages
18[128509]: segfault at 2b90 ip 0000000000002b90 sp 00007ffcc467d9c0 error 14 likely on CPU 0 (core 0, socket 0)
Code: Unable to access opcode bytes at 0x2b66.
Lustre: dir [0x200000bec:0xb11:0x0] stripe 2 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 2 previous similar messages
12[134065]: segfault at 8 ip 00007fe5ef8b7480 sp 00007fff5bd4d048 error 4 in ld-linux-x86-64.so.2[7fe5ef89b000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
0[134249]: segfault at 0 ip 000055d64c4d0a00 sp 00007ffca8fd6990 error 6 in 0[55d64c4d0000+4000] likely on CPU 1 (core 0, socket 1)
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
18[138805]: segfault at 8 ip 00007fd88718f480 sp 00007ffdf8ccd738 error 4 in ld-linux-x86-64.so.2[7fd887173000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 137242:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bef:0xb20:0x0] error -5.
LustreError: 137242:0:(vvp_io.c:1931:vvp_io_init()) Skipped 2 previous similar messages
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 4 previous similar messages
LustreError: 153344:0:(lcommon_cl.c:186:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000beb:0xc9d:0x0]: rc = -5
LustreError: 153344:0:(lcommon_cl.c:186:cl_file_inode_init()) Skipped 284 previous similar messages
LustreError: 153344:0:(llite_lib.c:3781:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5
LustreError: 153344:0:(llite_lib.c:3781:ll_prep_inode()) Skipped 284 previous similar messages
LustreError: 154935:0:(statahead.c:2408:start_statahead_thread()) lustre: invalid pattern 0X0.
LustreError: 157055:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ff1de51f5ec16800: inode [0x200000bea:0x1051:0x0] mdc close failed: rc = -2
LustreError: 157055:0:(file.c:248:ll_close_inode_openhandle()) Skipped 18 previous similar messages
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 2 previous similar messages
LustreError: 163675:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x200000bed:0xedf:0x0] get parent: rc = -2
LustreError: 163675:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) Skipped 19 previous similar messages
LustreError: 162815:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff1de51f427a2800: dir page locate: [0x280000be9:0xcab:0x0] at 0: rc -5
LustreError: 162815:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message
12[166687]: segfault at 8 ip 00007fcba37c5480 sp 00007ffe0dc4a138 error 4 in ld-linux-x86-64.so.2[7fcba37a9000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
1[167954]: segfault at 1c ip 00005645b500bb90 sp 00007ffe306b0d60 error 6 in 4[5645b500b000+4000] likely on CPU 0 (core 0, socket 0)
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1[167731]: segfault at 1c ip 0000560a94585b90 sp 00007ffdf2398c40 error 6 in 4[560a94585000+4000] likely on CPU 1 (core 0, socket 1)
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5[172878]: segfault at 8 ip 00007f15d5ba2480 sp 00007ffd89524ec8 error 4 in ld-linux-x86-64.so.2[7f15d5b86000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 172368:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f427a2800: cannot apply new layout on [0x280000be6:0xd8e:0x0] : rc = -5
LustreError: 172368:0:(lov_object.c:1346:lov_layout_change()) Skipped 2 previous similar messages
LustreError: 172368:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x280000be6:0xd8e:0x0] error -5.
LustreError: 172368:0:(vvp_io.c:1931:vvp_io_init()) Skipped 1 previous similar message
LustreError: 174192:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x280000be6:0xe68:0x0] get parent: rc = -116
LustreError: 174192:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) Skipped 3 previous similar messages
Lustre: dir [0x2c0000beb:0xf70:0x0] stripe 4 readdir failed: -2, directory is partially accessed!
Lustre: Skipped 14 previous similar messages
LustreError: 174460:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff1de51f5ec16800: dir page locate: [0x280000be8:0xe25:0x0] at 0: rc -5
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) lustre: failed to prep 12 [0x0:0x0:0x0] inode@0000000000000000: rc = -1
LustreError: 57765:0:(statahead.c:822:ll_statahead_interpret_work()) Skipped 1 previous similar message
7[180070]: segfault at 8 ip 00007f2c263cf480 sp 00007ffed1ac4a78 error 4 in ld-linux-x86-64.so.2[7f2c263b3000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
16[180390]: segfault at 8 ip 00007f8180bf2480 sp 00007ffc6e76e1b8 error 4 in ld-linux-x86-64.so.2[7f8180bd6000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
10[181774]: segfault at 8 ip 00007f247252e480 sp 00007ffd7c39d4a8 error 4 in ld-linux-x86-64.so.2[7f2472512000+27000] likely on CPU 0 (core 0, socket 0)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 183741:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0000-mdc-ff1de51f5ec16800: dir page locate: [0x200000bd0:0xde:0x0] at 0: rc -5
LustreError: 183741:0:(mdc_request.c:1510:mdc_read_page()) Skipped 1 previous similar message
15[190026]: segfault at 8 ip 00007f1236949480 sp 00007ffd32ed3208 error 4 in ld-linux-x86-64.so.2[7f123692d000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
16[192585]: segfault at 8 ip 00007fa752e20480 sp 00007ffcbe7887e8 error 4 in ld-linux-x86-64.so.2[7fa752e04000+27000] likely on CPU 0 (core 0, socket 0)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 189586:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x240000bef:0x1430:0x0] get parent: rc = -116
LustreError: 197800:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0002-mdc-ff1de51f427a2800: dir page locate: [0x280000be6:0x10b8:0x0] at 0: rc -5
LustreError: 197800:0:(mdc_request.c:1510:mdc_read_page()) Skipped 53 previous similar messages
13[201215]: segfault at 8 ip 00007f68449d9480 sp 00007fffc206a088 error 4 in ld-linux-x86-64.so.2[7f68449bd000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
16[207302]: segfault at 8 ip 00007f0c94089480 sp 00007fffe65ef988 error 4 in ld-linux-x86-64.so.2[7f0c9406d000+27000] likely on CPU 1 (core 0, socket 1)
Code: 5b 4c 89 f0 5d 41 5c 41 5d 41 5e c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 41 ff ff 6f 29 d0 48 8b 54 c6 40 48 8b 46 68 <48> 8b 40 08 f6 86 1e 03 00 00 20 74 03 48 03 06 48 85 d2 74 2b 48
LustreError: 207621:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) lustre: failure inode [0x240000bef:0x13f2:0x0] get parent: rc = -116
LustreError: 207621:0:(llite_nfs.c:413:ll_dir_get_parent_fid()) Skipped 29 previous similar messages
LustreError: 211824:0:(lov_object.c:1346:lov_layout_change()) lustre-clilov-ff1de51f5ec16800: cannot apply new layout on [0x240000bec:0x15cc:0x0] : rc = -5
LustreError: 211824:0:(lov_object.c:1346:lov_layout_change()) Skipped 17 previous similar messages
LustreError: 211824:0:(vvp_io.c:1931:vvp_io_init()) lustre: refresh file layout [0x240000bec:0x15cc:0x0] error -5.
LustreError: 211824:0:(vvp_io.c:1931:vvp_io_init()) Skipped 7 previous similar messages
LustreError: 213410:0:(llite_lib.c:1896:ll_update_lsm_md()) lustre: [0x240000bee:0x1892:0x0] dir layout mismatch:
LustreError: 213410:0:(llite_lib.c:1896:ll_update_lsm_md()) Skipped 1 previous similar message
LustreError: 213410:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=2 count=3 index=1 hash=crush:0x2000003 max_inherit=0 max_inherit_rr=0 version=2 migrate_offset=0 migrate_hash=invalid:0 pool=
LustreError: 213410:0:(lustre_lmv.h:166:lmv_stripe_object_dump()) stripe[0] [0x240000bd1:0x11a:0x0]
LustreError: 213410:0:(lustre_lmv.h:166:lmv_stripe_object_dump()) Skipped 17 previous similar messages
LustreError: 213410:0:(lustre_lmv.h:148:lmv_stripe_object_dump()) dump LMV: magic=0xcd20cd0 refs=1 count=4 index=1 hash=crush:0x82000003 max_inherit=0 max_inherit_rr=0 version=1 migrate_offset=3 migrate_hash=fnv_1a_64:2 pool=
LustreError: 215088:0:(mdc_request.c:1510:mdc_read_page()) lustre-MDT0000-mdc-ff1de51f427a2800: dir page locate: [0x200000bed:0x194e:0x0] at 0: rc -5
LustreError: 215088:0:(mdc_request.c:1510:mdc_read_page()) Skipped 22 previous similar messages
LustreError: 58927:0:(osc_cache.c:1410:osc_extent_holes_prune()) extent ff1de51fb0548d10@{[0 -> 1023/1023], [2|0|-|rpc|wio|ff1de51fb35eeec8], [0|0|-|-|0000000000000000|0|ff1de51fb8b48000]}
LustreError: 58927:0:(osc_cache.c:1268:osc_extent_tree_dump0()) Dump object ff1de51fb35eeec8 extents at osc_extent_holes_prune:1410, mppr: 1024.
LustreError: 58927:0:(osc_cache.c:1274:osc_extent_tree_dump0()) extent ff1de51fb0548d10@{[0 -> 1023/1023], [2|0|-|rpc|wio|ff1de51fb35eeec8], [0|0|-|-|0000000000000000|0|ff1de51fb8b48000]} in tree 1.

Link to test

sanity test 398b: DIO and buffer IO race

LustreError: 1120642:0:(osc_cache.c:1407:osc_extent_holes_prune()) ASSERTION( ext->oe_state == OES_CACHE ) failed:
LustreError: 1120642:0:(osc_cache.c:1407:osc_extent_holes_prune()) LBUG
CPU: 0 PID: 1120642 Comm: ldlm_bl_03 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x34/0x48
lbug_with_loc.cold+0x5/0x43 [libcfs]
osc_extent_holes_prune+0x5b3/0x620 [osc]
osc_lock_flush+0x9e/0x320 [osc]
osc_dlm_blocking_ast0.constprop.0+0xdf/0x310 [osc]
osc_ldlm_blocking_ast+0x6f/0x190 [osc]
ldlm_cancel_callback+0x7a/0x2b0 [ptlrpc]
ldlm_cli_cancel_local+0xa6/0x490 [ptlrpc]
ldlm_cli_cancel+0xab/0x540 [ptlrpc]
osc_ldlm_blocking_ast+0x14b/0x190 [osc]
ldlm_handle_bl_callback+0x10a/0x3f0 [ptlrpc]
ldlm_bl_thread_blwi.constprop.0+0x7c/0x320 [ptlrpc]
ldlm_bl_thread_main+0x4fb/0x5b0 [ptlrpc]
? __pfx_autoremove_wake_function+0x10/0x10
? __pfx_ldlm_bl_thread_main+0x10/0x10 [ptlrpc]
kthread+0xdd/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x29/0x50
</TASK>

Link to test

Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below):
Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below):
Limit to a test: (Copy from below "Failing text"):
Delete these reports as invalid (real bug in review or some such)
Bug or comment:
Extra info:

Editing crashreport #74477

Added fields:

Failures list (last 100):