Editing crashreport #75146

ReasonCrashing FunctionWhere to cut BacktraceReports Count
ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed__osc_lru_del__osc_lru_del
osc_page_cache_add
osc_io_commit_async
cl_io_commit_async
lov_io_commit_async
cl_io_commit_async
vvp_io_write_commit
vvp_io_write_start
cl_io_start
cl_io_loop
ll_file_io_generic
do_file_write_iter
ll_file_write_iter
vfs_write
ksys_write
__x64_sys_write
x64_sys_call
do_syscall_64
entry_SYSCALL_64_after_hwframe
5

Added fields:

Match messages in logs
(every line would be required to be present in log output
Copy from "Messages before crash" column below):
Match messages in full crash
(every line would be required to be present in crash log output
Copy from "Full Crash" column below):
Limit to a test:
(Copy from below "Failing text"):
Delete these reports as invalid (real bug in review or some such)
Bug or comment:
Extra info:

Failures list (last 100):

Failing TestFull CrashMessages before crashLink and date
sanity-flr test 70a: flr mode fsx test
LustreError: 360807:0:(osc_page.c:492:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed:
LustreError: 360807:0:(osc_page.c:492:__osc_lru_del()) LBUG
CPU: 1 UID: 0 PID: 360807 Comm: fsx Kdump: loaded Tainted: G OE 7.0.0-22-generic #22-Ubuntu PREEMPT(lazy)
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
show_stack+0x49/0x60
dump_stack_lvl+0x5f/0x90
dump_stack+0x10/0x18
lbug_with_loc.cold+0x5/0x50 [libcfs]
__osc_lru_del+0xd1/0xe0 [osc]
osc_page_cache_add+0x11d/0x1f0 [osc]
? cl_page_list_del+0x170/0x1d0 [obdclass]
osc_io_commit_async+0x1a1/0x5b0 [osc]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
lov_io_commit_async+0x388/0x650 [lov]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
vvp_io_write_commit+0x1ed/0x540 [lustre]
? srso_alias_return_thunk+0x5/0xfbef5
vvp_io_write_start+0x703/0xea0 [lustre]
cl_io_start+0x84/0x190 [obdclass]
cl_io_loop+0xe6/0x240 [obdclass]
ll_file_io_generic+0x8f9/0x1670 [lustre]
do_file_write_iter+0x640/0xda0 [lustre]
ll_file_write_iter+0x4b/0x120 [lustre]
? security_file_permission+0x5b/0x170
? srso_alias_return_thunk+0x5/0xfbef5
? rw_verify_area+0x57/0x180
vfs_write+0x25b/0x490
ksys_write+0x71/0xf0
__x64_sys_write+0x19/0x30
x64_sys_call+0x22f/0x2390
do_syscall_64+0x115/0x5a0
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? put_dec+0x1b/0xc0
? srso_alias_return_thunk+0x5/0xfbef5
? put_dec+0x1b/0xc0
? srso_alias_return_thunk+0x5/0xfbef5
? number+0x4c4/0x690
? string+0x64/0x120
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? string+0x64/0x120
? srso_alias_return_thunk+0x5/0xfbef5
? vsnprintf+0x21f/0x630
? srso_alias_return_thunk+0x5/0xfbef5
? snprintf+0x4e/0x80
? srso_alias_return_thunk+0x5/0xfbef5
? _raw_spin_unlock+0xe/0x40
? srso_alias_return_thunk+0x5/0xfbef5
? libcfs_debug_msg+0xb98/0x12d0 [libcfs]
? update_curr+0x115/0x130
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? ll_file_seek+0x1c1/0xa70 [lustre]
? _raw_spin_unlock+0xe/0x40
? srso_alias_return_thunk+0x5/0xfbef5
? raw_spin_rq_unlock+0x14/0x50
? srso_alias_return_thunk+0x5/0xfbef5
? ksys_lseek+0x50/0xd0
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? sysvec_apic_timer_interrupt+0x54/0xd0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7a8be7ea06c6
Lustre: 268198:0:(client.c:1668:after_reply()) @@@ resending request on EINPROGRESS req@ff3dd7d7ab6f70c0 x1879732874300160/t0(0) o10->lustre-OST0006-osc-ff3dd7d789259800@10.240.38.235@tcp:6/4 lens 440/432 e 0 to 0 dl 1792656244 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0
Link to test (2026-10-22 09:05)
sanity-flr test 70a: flr mode fsx test
LustreError: 256644:0:(osc_page.c:492:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed:
LustreError: 256644:0:(osc_page.c:492:__osc_lru_del()) LBUG
CPU: 0 UID: 0 PID: 256644 Comm: fsx Kdump: loaded Tainted: G OE 7.0.0-22-generic #22-Ubuntu PREEMPT(lazy)
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
show_stack+0x49/0x60
dump_stack_lvl+0x5f/0x90
dump_stack+0x10/0x18
lbug_with_loc.cold+0x5/0x50 [libcfs]
__osc_lru_del+0xd1/0xe0 [osc]
osc_page_cache_add+0x11d/0x1f0 [osc]
? cl_page_list_del+0x170/0x1d0 [obdclass]
osc_io_commit_async+0x32e/0x620 [osc]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
lov_io_commit_async+0x388/0x650 [lov]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
vvp_io_write_commit+0x1ed/0x5e0 [lustre]
? srso_alias_return_thunk+0x5/0xfbef5
vvp_io_write_start+0x703/0xea0 [lustre]
cl_io_start+0x84/0x190 [obdclass]
cl_io_loop+0xe6/0x240 [obdclass]
ll_file_io_generic+0x911/0x14e0 [lustre]
do_file_write_iter+0x640/0xda0 [lustre]
ll_file_write_iter+0x4b/0x120 [lustre]
? security_file_permission+0x5b/0x170
? srso_alias_return_thunk+0x5/0xfbef5
? rw_verify_area+0x57/0x180
vfs_write+0x25b/0x490
ksys_write+0x71/0xf0
__x64_sys_write+0x19/0x30
x64_sys_call+0x22f/0x2390
do_syscall_64+0x115/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? ll_file_seek+0x1c1/0x9a0 [lustre]
? ksys_lseek+0x50/0xd0
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? ksys_lseek+0x50/0xd0
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? arch_exit_to_user_mode_prepare.isra.0+0xc5/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? exc_page_fault+0x94/0x1e0
? common_interrupt+0x61/0xe0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7f027f0a06c6
Lustre: 137518:0:(client.c:1651:after_reply()) @@@ resending request on EINPROGRESS req@ff3589600ae72a40 x1876772253484544/t0(0) o10->lustre-OST0006-osc-ff3589600ae0d000@10.240.40.204@tcp:6/4 lens 440/432 e 0 to 0 dl 1789833460 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0
Link to test (2026-09-19 16:25)
sanity-flr test 70a: flr mode fsx test
LustreError: 272737:0:(osc_page.c:492:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed:
LustreError: 272737:0:(osc_page.c:492:__osc_lru_del()) LBUG
CPU: 1 UID: 0 PID: 272737 Comm: fsx Kdump: loaded Tainted: G OE 7.0.0-22-generic #22-Ubuntu PREEMPT(lazy)
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
show_stack+0x49/0x60
dump_stack_lvl+0x5f/0x90
dump_stack+0x10/0x18
lbug_with_loc.cold+0x5/0x50 [libcfs]
__osc_lru_del+0xd1/0xe0 [osc]
osc_page_cache_add+0x11d/0x1f0 [osc]
? cl_page_list_del+0x170/0x1d0 [obdclass]
osc_io_commit_async+0x32e/0x620 [osc]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
lov_io_commit_async+0x388/0x650 [lov]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
vvp_io_write_commit+0x1ed/0x5e0 [lustre]
? srso_alias_return_thunk+0x5/0xfbef5
vvp_io_write_start+0x703/0xea0 [lustre]
cl_io_start+0x84/0x190 [obdclass]
cl_io_loop+0xe6/0x240 [obdclass]
ll_file_io_generic+0x5d5/0x1330 [lustre]
do_file_write_iter+0x640/0xda0 [lustre]
ll_file_write_iter+0x4b/0x120 [lustre]
? security_file_permission+0x5b/0x170
? srso_alias_return_thunk+0x5/0xfbef5
? rw_verify_area+0x57/0x180
vfs_write+0x25b/0x490
ksys_write+0x71/0xf0
__x64_sys_write+0x19/0x30
x64_sys_call+0x22f/0x2390
do_syscall_64+0x115/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? common_interrupt+0x61/0xe0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x72885c8a06c6
Link to test (2026-08-25 18:44)
sanity-flr test 70a: flr mode fsx test
LustreError: 151546:0:(osc_page.c:492:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed:
LustreError: 151546:0:(osc_page.c:492:__osc_lru_del()) LBUG
CPU: 1 UID: 0 PID: 151546 Comm: fsx Kdump: loaded Tainted: G OE 7.0.0-22-generic #22-Ubuntu PREEMPT(lazy)
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
show_stack+0x49/0x60
dump_stack_lvl+0x5f/0x90
dump_stack+0x10/0x18
lbug_with_loc.cold+0x5/0x50 [libcfs]
__osc_lru_del+0xd1/0xe0 [osc]
osc_page_cache_add+0x11d/0x1f0 [osc]
? cl_page_list_del+0x170/0x1d0 [obdclass]
osc_io_commit_async+0x32e/0x620 [osc]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
lov_io_commit_async+0x388/0x650 [lov]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
vvp_io_write_commit+0x1ed/0x5e0 [lustre]
? srso_alias_return_thunk+0x5/0xfbef5
vvp_io_write_start+0x703/0xea0 [lustre]
cl_io_start+0x84/0x190 [obdclass]
cl_io_loop+0xe6/0x240 [obdclass]
ll_file_io_generic+0x5d5/0x1330 [lustre]
do_file_write_iter+0x640/0xda0 [lustre]
ll_file_write_iter+0x4b/0x120 [lustre]
? security_file_permission+0x5b/0x170
? srso_alias_return_thunk+0x5/0xfbef5
? rw_verify_area+0x57/0x180
vfs_write+0x25b/0x490
ksys_write+0x71/0xf0
__x64_sys_write+0x19/0x30
x64_sys_call+0x22f/0x2390
do_syscall_64+0x115/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7ba7952a06c6
Link to test (2026-08-21 08:55)
sanity-flr test 70a: flr mode fsx test
LustreError: 140733:0:(osc_page.c:492:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed:
LustreError: 140733:0:(osc_page.c:492:__osc_lru_del()) LBUG
CPU: 0 UID: 0 PID: 140733 Comm: fsx Kdump: loaded Tainted: G OE 7.0.0-22-generic #22-Ubuntu PREEMPT(lazy)
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
show_stack+0x49/0x60
dump_stack_lvl+0x5f/0x90
dump_stack+0x10/0x18
lbug_with_loc.cold+0x5/0x50 [libcfs]
__osc_lru_del+0xd1/0xe0 [osc]
osc_page_cache_add+0x11d/0x1f0 [osc]
? cl_page_list_del+0x170/0x1d0 [obdclass]
osc_io_commit_async+0x32e/0x620 [osc]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
lov_io_commit_async+0x388/0x650 [lov]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
cl_io_commit_async+0x8f/0x170 [obdclass]
? __pfx_write_commit_callback+0x10/0x10 [lustre]
vvp_io_write_commit+0x1ed/0x5e0 [lustre]
? srso_alias_return_thunk+0x5/0xfbef5
vvp_io_write_start+0x703/0xea0 [lustre]
cl_io_start+0x84/0x190 [obdclass]
cl_io_loop+0xe6/0x240 [obdclass]
ll_file_io_generic+0x5d5/0x1330 [lustre]
do_file_write_iter+0x643/0xda0 [lustre]
ll_file_write_iter+0x4b/0x120 [lustre]
? security_file_permission+0x5b/0x170
? srso_alias_return_thunk+0x5/0xfbef5
? rw_verify_area+0x57/0x180
vfs_write+0x25b/0x490
ksys_write+0x71/0xf0
__x64_sys_write+0x19/0x30
x64_sys_call+0x22f/0x2390
do_syscall_64+0x115/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? arch_exit_to_user_mode_prepare.isra.0+0xd/0xe0
? srso_alias_return_thunk+0x5/0xfbef5
? do_syscall_64+0x150/0x5a0
? srso_alias_return_thunk+0x5/0xfbef5
? exc_page_fault+0x94/0x1e0
? common_interrupt+0x61/0xe0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7b4468ea06c6
Lustre: 3435:0:(client.c:1638:after_reply()) @@@ resending request on EINPROGRESS req@ff1c8359adbcc9c0 x1871607468517504/t0(0) o10->lustre-OST0001-osc-ff1c83594a37d800@10.240.28.175@tcp:6/4 lens 440/432 e 0 to 0 dl 1784908050 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0
Link to test (2026-07-24 15:56)
Return to new crashes list