| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| sanity-flr test 70a: flr mode fsx test | LustreError: 1482:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 1482:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 1482 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150700.53.22-default #1 SLE15-SP7 4ec3f6cf5343bc56be8643f30aa0915ec99eb2ef Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs 24a0962790080ad6d7d98b66fc9963d32129d35c] __osc_lru_del+0x8c/0x90 [osc e1d6e8fbadf0f806cbc488eb2e7e6ec513b36d35] osc_lru_use.part.20+0x41/0x70 [osc e1d6e8fbadf0f806cbc488eb2e7e6ec513b36d35] osc_page_cache_add+0x166/0x170 [osc e1d6e8fbadf0f806cbc488eb2e7e6ec513b36d35] osc_io_commit_async+0x34c/0x510 [osc e1d6e8fbadf0f806cbc488eb2e7e6ec513b36d35] ? __pfx_write_commit_callback+0x10/0x10 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] ? __pfx_write_commit_callback+0x10/0x10 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] cl_io_commit_async+0x91/0x170 [obdclass 2cf835210718d176857f40cd8bfd4442a8248402] lov_io_commit_async+0xf6/0x5d0 [lov 216a5ac1e6c475e2f6c09dc4e66a14f63002917f] ? __pfx_write_commit_callback+0x10/0x10 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] ? __pfx_write_commit_callback+0x10/0x10 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] cl_io_commit_async+0x91/0x170 [obdclass 2cf835210718d176857f40cd8bfd4442a8248402] vvp_io_write_commit+0xc1/0x5d0 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] vvp_io_write_start+0x757/0xf50 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] cl_io_start+0x70/0x140 [obdclass 2cf835210718d176857f40cd8bfd4442a8248402] cl_io_loop+0x9e/0x230 [obdclass 2cf835210718d176857f40cd8bfd4442a8248402] ? ll_cl_add+0x95/0x100 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] ll_file_io_generic+0xab0/0x1210 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] do_file_write_iter+0x1fe/0xb90 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] vfs_write+0x2e6/0x380 ksys_write+0xa5/0xe0 do_syscall_64+0x5b/0x80 ? ll_file_seek+0x325/0x880 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] ? ll_file_seek+0x325/0x880 [lustre 95ede9841b3d18d6b886ebcdfeed81fd1668523b] ? file_update_time+0x9f/0xb0 ? ksys_lseek+0x76/0xa0 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? common_file_perm+0x74/0x1a0 ? do_syscall_64+0x67/0x80 ? vfs_write+0x2e6/0x380 ? ksys_write+0x50/0xe0 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? arch_do_signal_or_restart+0x2a/0x260 ? recalc_sigpending+0x17/0x50 ? exit_to_user_mode_prepare+0x197/0x240 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? do_syscall_64+0x67/0x80 ? ksys_lseek+0x76/0xa0 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7fd221f21050 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 199898:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 199898:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 199898 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d4/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x2cb/0x410 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xf0 ? lov_lsm_addref+0x8e/0x220 [lov] ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x16b/0x5a0 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? exc_page_fault+0x62/0x150 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7f40a3efdb57 | Lustre: 3218:0:(client.c:1636:after_reply()) @@@ resending request on EINPROGRESS req@ff4dbef9583389c0 x1855771582888832/t0(0) o10->lustre-OST0001-osc-ff4dbef943262000@10.240.40.2@tcp:6/4 lens 440/432 e 0 to 0 dl 1769805504 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Autotest: Test running for 65 minutes (lustre-master-next_full-part-1_948.190) | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 30135:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 30135:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 30135 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150700.53.22-default #1 SLE15-SP7 4ec3f6cf5343bc56be8643f30aa0915ec99eb2ef Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs f905d3a25fe8f0da12533d0dc0f6d527c39851fc] __osc_lru_del+0x8c/0x90 [osc ca45b0afca591a578d27c4e170c455914b23cca8] osc_lru_use.part.20+0x41/0x70 [osc ca45b0afca591a578d27c4e170c455914b23cca8] osc_page_cache_add+0x166/0x170 [osc ca45b0afca591a578d27c4e170c455914b23cca8] osc_io_commit_async+0x34c/0x510 [osc ca45b0afca591a578d27c4e170c455914b23cca8] ? __pfx_write_commit_callback+0x10/0x10 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] ? __pfx_write_commit_callback+0x10/0x10 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] cl_io_commit_async+0x8e/0x170 [obdclass 84944707313279ea088d32dab8276d5b51d622f4] lov_io_commit_async+0xf6/0x5d0 [lov 2ffbd9b21626159ab4fa5d503a1423e94d395633] ? __pfx_write_commit_callback+0x10/0x10 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] ? __pfx_write_commit_callback+0x10/0x10 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] cl_io_commit_async+0x8e/0x170 [obdclass 84944707313279ea088d32dab8276d5b51d622f4] vvp_io_write_commit+0xc1/0x5d0 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] vvp_io_write_start+0x757/0xf50 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] cl_io_start+0x6d/0x140 [obdclass 84944707313279ea088d32dab8276d5b51d622f4] cl_io_loop+0x9e/0x230 [obdclass 84944707313279ea088d32dab8276d5b51d622f4] ? ll_cl_add+0x95/0x100 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] ll_file_io_generic+0xab0/0x1210 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] do_file_write_iter+0x1fe/0xb90 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 do_syscall_64+0x58/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x325/0x880 [lustre cd9b93ce5fd6346d9afdba7ce6d26420c0de771f] ? __wake_up_common+0x71/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_lseek+0x73/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? pipe_write+0x3ae/0x630 ? common_file_perm+0x74/0x1a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? vfs_write+0x2e3/0x380 ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_write+0x50/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? exit_to_user_mode_prepare+0x15f/0x240 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_lseek+0x73/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f6873d21050 | Lustre: 6570:0:(client.c:1637:after_reply()) @@@ resending request on EINPROGRESS req@ff33749e6b10ba80 x1855707527728384/t0(0) o10->lustre-OST0006-osc-ff33749e54a80800@10.240.38.132@tcp:6/4 lens 440/432 e 0 to 0 dl 1769744504 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 647301:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 647301:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 647301 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-570.62.1.el9_6.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d1/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] ? __entry_text_end+0x102739/0x10273d do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x33a/0x480 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xe0 ? lov_lsm_addref+0x8e/0x220 [lov] ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x16b/0x5a0 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? update_load_avg+0x7e/0x7a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? update_load_avg+0x7e/0x7a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? set_next_entity+0xda/0x150 ? srso_alias_return_thunk+0x5/0xfbef5 ? pick_next_task_fair+0x1dc/0x510 ? srso_alias_return_thunk+0x5/0xfbef5 ? pick_next_task+0x56/0xaf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? rseq_get_rseq_cs+0x1d/0x240 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? rseq_ip_fixup+0x6e/0x1a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? __rseq_handle_notify_resume+0x26/0xc0 ? srso_alias_return_thunk+0x5/0xfbef5 ? fpregs_restore_userregs+0x47/0xd0 ? srso_alias_return_thunk+0x5/0xfbef5 ? arch_exit_to_user_mode_prepare.constprop.0+0x6c/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? do_syscall_64+0x6b/0xe0 ? sysvec_apic_timer_interrupt+0x3c/0x90 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7f30970fde97 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 13101:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 13101:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 13101 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150600.23.78-default #1 SLE15-SP6 05608db11af775a2aeb6efab3a046d19e27eab89 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs a7d560608c8e11ec0acdb7caedf677183c5a025b] __osc_lru_del+0x8c/0x90 [osc 649728b4c776eb0867465d21a8397d45f7896927] osc_lru_use.part.20+0x41/0x70 [osc 649728b4c776eb0867465d21a8397d45f7896927] osc_page_cache_add+0x166/0x170 [osc 649728b4c776eb0867465d21a8397d45f7896927] osc_io_commit_async+0x34c/0x510 [osc 649728b4c776eb0867465d21a8397d45f7896927] ? __pfx_write_commit_callback+0x10/0x10 [lustre fa954fc00516f91d7337251dcf35f304fa586130] ? __pfx_write_commit_callback+0x10/0x10 [lustre fa954fc00516f91d7337251dcf35f304fa586130] cl_io_commit_async+0x8e/0x170 [obdclass 8846631593c7dcffbf439053d2ee9e05be79e203] lov_io_commit_async+0xf6/0x5d0 [lov d03afc55ad537a79822249667f9b5638f69d9638] ? __pfx_write_commit_callback+0x10/0x10 [lustre fa954fc00516f91d7337251dcf35f304fa586130] ? __pfx_write_commit_callback+0x10/0x10 [lustre fa954fc00516f91d7337251dcf35f304fa586130] cl_io_commit_async+0x8e/0x170 [obdclass 8846631593c7dcffbf439053d2ee9e05be79e203] vvp_io_write_commit+0xc1/0x5d0 [lustre fa954fc00516f91d7337251dcf35f304fa586130] vvp_io_write_start+0x757/0xf50 [lustre fa954fc00516f91d7337251dcf35f304fa586130] cl_io_start+0x6d/0x140 [obdclass 8846631593c7dcffbf439053d2ee9e05be79e203] cl_io_loop+0x9e/0x230 [obdclass 8846631593c7dcffbf439053d2ee9e05be79e203] ? ll_cl_add+0x95/0x100 [lustre fa954fc00516f91d7337251dcf35f304fa586130] ll_file_io_generic+0xab0/0x1210 [lustre fa954fc00516f91d7337251dcf35f304fa586130] do_file_write_iter+0x1fe/0xb90 [lustre fa954fc00516f91d7337251dcf35f304fa586130] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 do_syscall_64+0x58/0x80 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f4dff121770 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 631:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 631:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 631 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150700.53.22-default #1 SLE15-SP7 4ec3f6cf5343bc56be8643f30aa0915ec99eb2ef Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs 6d9092fb69242d31ac87931bbcdf1ebec0a9ddc4] __osc_lru_del+0x8c/0x90 [osc 4f6ecaf89fb86e46a8c9b2a23facb6bac79f0a8b] osc_lru_use.part.20+0x41/0x70 [osc 4f6ecaf89fb86e46a8c9b2a23facb6bac79f0a8b] osc_page_cache_add+0x166/0x170 [osc 4f6ecaf89fb86e46a8c9b2a23facb6bac79f0a8b] osc_io_commit_async+0x34c/0x510 [osc 4f6ecaf89fb86e46a8c9b2a23facb6bac79f0a8b] ? __pfx_write_commit_callback+0x10/0x10 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] ? __pfx_write_commit_callback+0x10/0x10 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] cl_io_commit_async+0x8e/0x170 [obdclass 855f324934c5779c868a9a0ba4c9c08d91214e7c] lov_io_commit_async+0xf6/0x5d0 [lov ed1295851f85b4dc351ed78c5687cbdaa5f03ef7] ? __pfx_write_commit_callback+0x10/0x10 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] ? __pfx_write_commit_callback+0x10/0x10 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] cl_io_commit_async+0x8e/0x170 [obdclass 855f324934c5779c868a9a0ba4c9c08d91214e7c] vvp_io_write_commit+0xc1/0x5d0 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] vvp_io_write_start+0x757/0xf50 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] cl_io_start+0x6d/0x140 [obdclass 855f324934c5779c868a9a0ba4c9c08d91214e7c] cl_io_loop+0x9e/0x230 [obdclass 855f324934c5779c868a9a0ba4c9c08d91214e7c] ? ll_cl_add+0x95/0x100 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] ll_file_io_generic+0xab0/0x1210 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] do_file_write_iter+0x1fe/0xb90 [lustre d569d2c69669a4daca0bc15e89690ab82e940ff3] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 ? file_update_time+0x70/0xb0 do_syscall_64+0x58/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? pipe_write+0x3ae/0x630 ? common_file_perm+0x74/0x1a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? vfs_write+0x2e3/0x380 ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_write+0x50/0xe0 ? dequeue_signal+0xce/0x1a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? get_signal+0x744/0x910 ? srso_alias_return_thunk+0x5/0xfbef5 ? arch_do_signal_or_restart+0x2a/0x260 ? do_sigaction+0x122/0x270 ? srso_alias_return_thunk+0x5/0xfbef5 ? recalc_sigpending+0x17/0x50 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? exit_to_user_mode_prepare+0x197/0x240 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7fe484321050 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 647937:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 647937:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 647937 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-570.62.1.el9_6.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d1/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x33a/0x480 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? libcfs_debug_msg+0x7e6/0xb00 [libcfs] ? srso_alias_return_thunk+0x5/0xfbef5 ? check_preempt_wakeup_fair+0x136/0x2d0 ? ttwu_do_activate+0x72/0x1f0 ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x16b/0x5a0 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? __entry_text_end+0x102739/0x10273d ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? vfs_write+0x3da/0x480 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? exc_page_fault+0x62/0x150 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7f97eb8fde97 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 635182:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 635182:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 635182 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d4/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x2cb/0x410 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? rseq_get_rseq_cs+0x1d/0x240 ? srso_alias_return_thunk+0x5/0xfbef5 ? rseq_ip_fixup+0x6e/0x1a0 ? kmem_cache_free+0x15/0x360 ? srso_alias_return_thunk+0x5/0xfbef5 ? __rseq_handle_notify_resume+0x26/0xb0 ? srso_alias_return_thunk+0x5/0xfbef5 ? exit_to_user_mode_loop+0xd9/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? exit_to_user_mode_prepare+0xef/0x100 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xf0 ? do_syscall_64+0x6b/0xf0 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7fbfd34fdb57 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 648130:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 648130:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 648130 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-570.62.1.el9_6.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d1/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x33a/0x480 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xe0 ? lov_lsm_addref+0x8e/0x220 [lov] ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x16b/0x5a0 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? libcfs_debug_msg+0x7e6/0xb00 [libcfs] ? cl_env_put+0x1ec/0x200 [obdclass] ? srso_alias_return_thunk+0x5/0xfbef5 ? rseq_get_rseq_cs+0x1d/0x240 ? srso_alias_return_thunk+0x5/0xfbef5 ? rseq_ip_fixup+0x6e/0x1a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? __rseq_handle_notify_resume+0x26/0xc0 ? srso_alias_return_thunk+0x5/0xfbef5 ? exit_to_user_mode_loop+0xd9/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? arch_exit_to_user_mode_prepare.constprop.0+0x6c/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xe0 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7f4c81efde97 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 757958:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 757958:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 UID: 0 PID: 757958 Comm: fsx Kdump: loaded Tainted: G OE ------ --- 6.12.0-124.20.1.el10_1.x86_64 #1 PREEMPT(voluntary) Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0xe3/0x190 [osc] osc_io_commit_async+0x2ce/0x5e0 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x86/0x160 [obdclass] lov_io_commit_async+0x318/0x670 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x86/0x160 [obdclass] vvp_io_write_commit+0x1dc/0x5c0 [lustre] vvp_io_write_start+0x6c0/0xe40 [lustre] cl_io_start+0x62/0x130 [obdclass] cl_io_loop+0x10b/0x210 [obdclass] ll_file_io_generic+0x5ad/0x11c0 [lustre] do_file_write_iter+0x5d4/0x9b0 [lustre] vfs_write+0x28d/0x450 ksys_write+0x6d/0xf0 do_syscall_64+0x7d/0x160 ? srso_alias_return_thunk+0x5/0xfbef5 ? vsnprintf+0x257/0x700 ? srso_alias_return_thunk+0x5/0xfbef5 ? snprintf+0x53/0x70 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? libcfs_debug_msg+0x815/0xbc0 [libcfs] ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x16f/0x640 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0xf3/0x120 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x32/0x190 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x89/0x160 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 entry_SYSCALL_64_after_hwframe+0x76/0x7e RIP: 0033:0x7f3c35ca9cd4 | Lustre: 375250:0:(client.c:1636:after_reply()) @@@ resending request on EINPROGRESS req@ff449f18f3686700 x1852365588556928/t0(0) o10->lustre-OST0001-osc-ff449f18c1c04800@10.240.28.95@tcp:6/4 lens 440/432 e 0 to 0 dl 1766562215 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Lustre: 375249:0:(client.c:1636:after_reply()) @@@ resending request on EINPROGRESS req@ff449f18c64d49c0 x1852365588903936/t0(0) o10->lustre-OST0004-osc-ff449f18c1c04800@10.240.28.95@tcp:6/4 lens 440/432 e 0 to 0 dl 1766562223 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Lustre: 375249:0:(client.c:1636:after_reply()) Skipped 7 previous similar messages | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 2654:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 2654:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 2654 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150700.53.22-default #1 SLE15-SP7 4ec3f6cf5343bc56be8643f30aa0915ec99eb2ef Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs fe92a5316e5256797d11e6c93afc4010c8132012] __osc_lru_del+0x8c/0x90 [osc 90977176ccf3d4f250b8e9b37f7b4dde48b06301] osc_lru_use.part.20+0x41/0x70 [osc 90977176ccf3d4f250b8e9b37f7b4dde48b06301] osc_page_cache_add+0x166/0x170 [osc 90977176ccf3d4f250b8e9b37f7b4dde48b06301] osc_io_commit_async+0x34c/0x510 [osc 90977176ccf3d4f250b8e9b37f7b4dde48b06301] ? __pfx_write_commit_callback+0x10/0x10 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] ? __pfx_write_commit_callback+0x10/0x10 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] cl_io_commit_async+0x8e/0x170 [obdclass 02e64c195a64b3455e6867903bdc49113663ff13] lov_io_commit_async+0xf6/0x5d0 [lov d32e76bb76f72cc9b06eeb079e68846830d09b5f] ? __pfx_write_commit_callback+0x10/0x10 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] ? __pfx_write_commit_callback+0x10/0x10 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] cl_io_commit_async+0x8e/0x170 [obdclass 02e64c195a64b3455e6867903bdc49113663ff13] vvp_io_write_commit+0xc1/0x5d0 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] vvp_io_write_start+0x757/0xf50 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] cl_io_start+0x6d/0x140 [obdclass 02e64c195a64b3455e6867903bdc49113663ff13] cl_io_loop+0x9e/0x230 [obdclass 02e64c195a64b3455e6867903bdc49113663ff13] ? ll_cl_add+0x95/0x100 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] ll_file_io_generic+0xab0/0x1210 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] do_file_write_iter+0x1fe/0xb90 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 ? string+0x44/0x60 do_syscall_64+0x58/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? snprintf+0x45/0x70 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? libcfs_debug_msg+0x589/0xd40 [libcfs fe92a5316e5256797d11e6c93afc4010c8132012] ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x325/0x880 [lustre bfd37c10d88185268755c7a4a87edcfbc6258599] ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_lseek+0x73/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? exc_page_fault+0x6c/0x160 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f44b0721050 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 2813:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 2813:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 2813 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150600.23.78-default #1 SLE15-SP6 05608db11af775a2aeb6efab3a046d19e27eab89 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs a9c763b0ee54dc3562da13b0fe77408c0120aaba] __osc_lru_del+0x8c/0x90 [osc f1b8bb7c15c490d1294120253f49495db8267bbf] osc_lru_use.part.20+0x41/0x70 [osc f1b8bb7c15c490d1294120253f49495db8267bbf] osc_page_cache_add+0x166/0x170 [osc f1b8bb7c15c490d1294120253f49495db8267bbf] osc_io_commit_async+0x34c/0x510 [osc f1b8bb7c15c490d1294120253f49495db8267bbf] ? __pfx_write_commit_callback+0x10/0x10 [lustre 498a1a09752481349c369652b8a728853df0cdb2] ? __pfx_write_commit_callback+0x10/0x10 [lustre 498a1a09752481349c369652b8a728853df0cdb2] cl_io_commit_async+0x8e/0x170 [obdclass cc9fcbe2061238074425661b525a3d114438544f] lov_io_commit_async+0xf6/0x5d0 [lov 647ac97e71fc49a8f6036274a8317b49984ad491] ? __pfx_write_commit_callback+0x10/0x10 [lustre 498a1a09752481349c369652b8a728853df0cdb2] ? __pfx_write_commit_callback+0x10/0x10 [lustre 498a1a09752481349c369652b8a728853df0cdb2] cl_io_commit_async+0x8e/0x170 [obdclass cc9fcbe2061238074425661b525a3d114438544f] vvp_io_write_commit+0xc1/0x5d0 [lustre 498a1a09752481349c369652b8a728853df0cdb2] vvp_io_write_start+0x757/0xf50 [lustre 498a1a09752481349c369652b8a728853df0cdb2] cl_io_start+0x6d/0x140 [obdclass cc9fcbe2061238074425661b525a3d114438544f] cl_io_loop+0x9e/0x230 [obdclass cc9fcbe2061238074425661b525a3d114438544f] ? ll_cl_add+0x95/0x100 [lustre 498a1a09752481349c369652b8a728853df0cdb2] ll_file_io_generic+0xab0/0x1210 [lustre 498a1a09752481349c369652b8a728853df0cdb2] do_file_write_iter+0x1fe/0xb90 [lustre 498a1a09752481349c369652b8a728853df0cdb2] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 do_syscall_64+0x58/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_lseek+0x73/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? exc_page_fault+0x69/0x150 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f8a36521770 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 634870:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 634870:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 634870 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d4/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x2cb/0x410 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? cl_env_put+0x1ec/0x200 [obdclass] ? lov_lsm_addref+0x8e/0x220 [lov] ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x16b/0x5a0 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xf0 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7f8d3d8fdb57 | Lustre: 310821:0:(client.c:1636:after_reply()) @@@ resending request on EINPROGRESS req@ff127d0f85a6da00 x1851661142741504/t0(0) o10->lustre-OST0001-osc-ff127d0f833ee800@10.240.40.21@tcp:6/4 lens 440/432 e 0 to 0 dl 1765891875 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Lustre: 310821:0:(client.c:1636:after_reply()) Skipped 4 previous similar messages | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 8493:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 8493:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 8493 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150700.53.22-default #1 SLE15-SP7 4ec3f6cf5343bc56be8643f30aa0915ec99eb2ef Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [lnet b8db23c7d859ace54e3588a28ec332b8005ee8ef] __osc_lru_del+0x8c/0x90 [osc 7903b3379f4bc20e720a0f5578245d556cc11e0e] osc_lru_use.part.20+0x41/0x70 [osc 7903b3379f4bc20e720a0f5578245d556cc11e0e] osc_page_cache_add+0x166/0x170 [osc 7903b3379f4bc20e720a0f5578245d556cc11e0e] osc_io_commit_async+0x34c/0x510 [osc 7903b3379f4bc20e720a0f5578245d556cc11e0e] ? __pfx_write_commit_callback+0x10/0x10 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] ? __pfx_write_commit_callback+0x10/0x10 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] cl_io_commit_async+0x8e/0x170 [obdclass 674f914b2a459bf46b7377cd522cca1aae9f3833] lov_io_commit_async+0xf6/0x5d0 [lov 807175ee3de28b4c6fcd1ea3b9499e0ad5388035] ? __pfx_write_commit_callback+0x10/0x10 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] ? __pfx_write_commit_callback+0x10/0x10 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] cl_io_commit_async+0x8e/0x170 [obdclass 674f914b2a459bf46b7377cd522cca1aae9f3833] vvp_io_write_commit+0xc1/0x5d0 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] vvp_io_write_start+0x757/0xf50 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] cl_io_start+0x6d/0x140 [obdclass 674f914b2a459bf46b7377cd522cca1aae9f3833] cl_io_loop+0x9e/0x230 [obdclass 674f914b2a459bf46b7377cd522cca1aae9f3833] ? ll_cl_add+0x95/0x100 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] ll_file_io_generic+0xab0/0x1210 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] do_file_write_iter+0x1fe/0xb90 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 ? switch_fpu_return+0x4c/0xd0 do_syscall_64+0x58/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? ll_file_seek+0x325/0x880 [lustre 1d6322ce6b0e5aee0d443eb0f86476982c0df8b8] ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? ksys_lseek+0x73/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? irq_exit_rcu+0x46/0x100 ? srso_alias_return_thunk+0x5/0xfbef5 ? common_interrupt+0x8b/0xa0 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7ff2dff21050 | Lustre: 3644:0:(client.c:1637:after_reply()) @@@ resending request on EINPROGRESS req@ff2cb19543f72a40 x1849808076181376/t0(0) o10->lustre-OST0003-osc-ff2cb19563799800@10.240.46.173@tcp:6/4 lens 440/432 e 0 to 0 dl 1764117976 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Autotest: Test running for 60 minutes (lustre-master-next_full-part-1_941.154) | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 645481:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 645481:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 645481 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [lnet] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d4/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x7c0/0xed0 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x593/0x11f0 [lustre] do_file_write_iter+0x558/0x890 [lustre] ? srso_alias_return_thunk+0x5/0xfbef5 vfs_write+0x2cb/0x410 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? __irq_exit_rcu+0x46/0xc0 ? srso_alias_return_thunk+0x5/0xfbef5 ? common_interrupt+0x43/0xa0 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7fa952cfdb57 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 1017:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 1017:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 1017 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150600.23.73-default #1 SLE15-SP6 650906754755f72aca25a23b90b6957d6e63cd93 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [lnet 0f107b0e31097c107d8da9363ca52bc2dabd1c01] __osc_lru_del+0x8c/0x90 [osc 7adf1ee5cdf91e5ce49c1606283657a40d1c7c99] osc_lru_use.part.20+0x41/0x70 [osc 7adf1ee5cdf91e5ce49c1606283657a40d1c7c99] osc_page_cache_add+0x166/0x170 [osc 7adf1ee5cdf91e5ce49c1606283657a40d1c7c99] osc_io_commit_async+0x34c/0x510 [osc 7adf1ee5cdf91e5ce49c1606283657a40d1c7c99] ? __pfx_write_commit_callback+0x10/0x10 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] ? __pfx_write_commit_callback+0x10/0x10 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] cl_io_commit_async+0x8e/0x170 [obdclass e145789c473bb48862a34e9e1602ba58fd6f86e2] lov_io_commit_async+0xf6/0x5d0 [lov 864bcc4dcfbdc9eaba846cd0a50334570352573d] ? __pfx_write_commit_callback+0x10/0x10 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] ? __pfx_write_commit_callback+0x10/0x10 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] cl_io_commit_async+0x8e/0x170 [obdclass e145789c473bb48862a34e9e1602ba58fd6f86e2] vvp_io_write_commit+0xc1/0x5d0 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] vvp_io_write_start+0x757/0xf50 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] cl_io_start+0x6d/0x140 [obdclass e145789c473bb48862a34e9e1602ba58fd6f86e2] cl_io_loop+0x9e/0x230 [obdclass e145789c473bb48862a34e9e1602ba58fd6f86e2] ? ll_cl_add+0x95/0x100 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] ll_file_io_generic+0xab0/0x1210 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] do_file_write_iter+0x1fe/0xb90 [lustre bd788379fd8a45b8efe7a14f2d21fc65bb7c43f4] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 ? syscall_exit_to_user_mode+0x1e/0x40 do_syscall_64+0x58/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? do_syscall_64+0x67/0x80 ? irq_exit_rcu+0x40/0xc0 ? srso_alias_return_thunk+0x5/0xfbef5 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f8be9921770 | Lustre: 11115:0:(client.c:1637:after_reply()) @@@ resending request on EINPROGRESS req@ff48354fc32c1a00 x1849778790811776/t0(0) o10->lustre-OST0003-osc-ff48354fe57ab000@10.240.26.239@tcp:6/4 lens 440/432 e 0 to 0 dl 1764096244 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 26359:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 26359:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 0 PID: 26359 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150700.53.16-default #1 SLE15-SP7 8ac8baf2d3053d7a3310b265bd97774b04cf5857 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs 41c72b90fadc5bcc6ac769a91fbc4db76e7453ba] __osc_lru_del+0x8c/0x90 [osc f0504d401483c2ec769e7f1acd51dcccfd23557e] osc_lru_use.part.20+0x41/0x70 [osc f0504d401483c2ec769e7f1acd51dcccfd23557e] osc_page_cache_add+0x166/0x170 [osc f0504d401483c2ec769e7f1acd51dcccfd23557e] osc_io_commit_async+0x34c/0x510 [osc f0504d401483c2ec769e7f1acd51dcccfd23557e] ? __pfx_write_commit_callback+0x10/0x10 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] ? __pfx_write_commit_callback+0x10/0x10 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] cl_io_commit_async+0x91/0x170 [obdclass 9513d4a245445564acd3e2d47541b7d9bce66adc] lov_io_commit_async+0xf6/0x5d0 [lov f6c688a0db76699b78efa5be18f67a80f756e857] ? __pfx_write_commit_callback+0x10/0x10 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] ? __pfx_write_commit_callback+0x10/0x10 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] cl_io_commit_async+0x91/0x170 [obdclass 9513d4a245445564acd3e2d47541b7d9bce66adc] vvp_io_write_commit+0xc1/0x5d0 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] vvp_io_write_start+0x729/0xe20 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] ? cl_lock_enqueue+0x73/0x150 [obdclass 9513d4a245445564acd3e2d47541b7d9bce66adc] cl_io_start+0x70/0x140 [obdclass 9513d4a245445564acd3e2d47541b7d9bce66adc] cl_io_loop+0x9e/0x230 [obdclass 9513d4a245445564acd3e2d47541b7d9bce66adc] ? ll_cl_add+0x95/0x100 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] ll_file_io_generic+0xa20/0x10a0 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] do_file_write_iter+0x1fe/0xb90 [lustre a3626e7ee3f9b27a51e4484685287fda1bc6dc2c] vfs_write+0x2e6/0x380 ksys_write+0xa5/0xe0 do_syscall_64+0x5b/0x80 ? ksys_lseek+0x76/0xa0 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? ksys_write+0x50/0xe0 ? syscall_exit_to_user_mode+0x1e/0x40 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? syscall_exit_to_user_mode+0x1e/0x40 ? do_syscall_64+0x67/0x80 ? do_syscall_64+0x67/0x80 ? exit_to_user_mode_prepare+0x142/0x220 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f8593521050 | Lustre: 3444:0:(client.c:1637:after_reply()) @@@ resending request on EINPROGRESS req@ffff889025971d40 x1849767969157888/t0(0) o10->lustre-OST0005-osc-ffff88913369a800@10.240.42.158@tcp:6/4 lens 440/432 e 0 to 0 dl 1764080223 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Lustre: 3445:0:(client.c:1637:after_reply()) @@@ resending request on EINPROGRESS req@ffff889101c84340 x1849767969244032/t0(0) o10->lustre-OST0005-osc-ffff88913369a800@10.240.42.158@tcp:6/4 lens 440/432 e 0 to 0 dl 1764080234 ref 1 fl Rpc:RQU/602/0 rc 0/-115 job:'fsx.0' uid:0 gid:0 projid:0 Lustre: 3445:0:(client.c:1637:after_reply()) Skipped 7 previous similar messages | Link to test |
| sanity-flr test 70a: flr mode fsx test | LustreError: 198314:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 198314:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 198314 Comm: fsx Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1.el9_5.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> ? __pfx_write_commit_callback+0x10/0x10 [lustre] dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] __osc_lru_del+0xaa/0xb0 [osc] osc_lru_use.part.0+0x41/0x70 [osc] osc_page_cache_add+0x88/0x180 [osc] osc_io_commit_async+0x2d4/0x520 [osc] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] ? __pfx_write_commit_callback+0x10/0x10 [lustre] lov_io_commit_async+0x149/0x5b0 [lov] ? __pfx_write_commit_callback+0x10/0x10 [lustre] ? __pfx_write_commit_callback+0x10/0x10 [lustre] cl_io_commit_async+0x7e/0x160 [obdclass] vvp_io_write_commit+0xe8/0x5d0 [lustre] vvp_io_write_start+0x6a9/0xe20 [lustre] cl_io_start+0x5e/0x130 [obdclass] cl_io_loop+0x99/0x220 [obdclass] ll_file_io_generic+0x585/0x1090 [lustre] do_file_write_iter+0x558/0x890 [lustre] vfs_write+0x2cb/0x410 ksys_write+0x5f/0xe0 do_syscall_64+0x5c/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_work+0x103/0x130 ? srso_alias_return_thunk+0x5/0xfbef5 ? syscall_exit_to_user_mode+0x19/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x6b/0xf0 ? srso_alias_return_thunk+0x5/0xfbef5 ? __do_sys_newfstat+0x57/0x60 ? srso_alias_return_thunk+0x5/0xfbef5 ? __update_load_avg_cfs_rq+0x289/0x2f0 ? srso_alias_return_thunk+0x5/0xfbef5 ? update_load_avg+0x7e/0x7a0 ? srso_alias_return_thunk+0x5/0xfbef5 ? update_curr+0x24/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? reweight_entity+0x14d/0x160 ? srso_alias_return_thunk+0x5/0xfbef5 ? task_tick_fair+0x30b/0x3d0 ? srso_alias_return_thunk+0x5/0xfbef5 ? perf_event_task_tick+0x56/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? nohz_balancer_kick+0x31/0x240 ? srso_alias_return_thunk+0x5/0xfbef5 ? next_expiry_recalc+0x9e/0x110 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? __run_timers.part.0+0x105/0x280 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? enqueue_hrtimer+0x2f/0x80 ? srso_alias_return_thunk+0x5/0xfbef5 ? __hrtimer_run_queues+0x141/0x2b0 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? ktime_get+0x35/0xa0 ? srso_alias_return_thunk+0x5/0xfbef5 ? __do_softirq+0x169/0x2a8 ? srso_alias_return_thunk+0x5/0xfbef5 ? __irq_exit_rcu+0x46/0xc0 ? srso_alias_return_thunk+0x5/0xfbef5 ? sysvec_apic_timer_interrupt+0x3c/0x90 entry_SYSCALL_64_after_hwframe+0x78/0x80 RIP: 0033:0x7f0c460fdb57 | Link to test | |
| sanity-flr test 70a: flr mode fsx test | LustreError: 4304:0:(osc_page.c:470:__osc_lru_del()) ASSERTION( atomic_long_read(&cli->cl_lru_in_list) >= 0 ) failed: LustreError: 4304:0:(osc_page.c:470:__osc_lru_del()) LBUG CPU: 1 PID: 4304 Comm: fsx Kdump: loaded Tainted: G OE n 6.4.0-150600.23.70-default #1 SLE15-SP6 2bde4fa135f054a155dd920aca05c271b0f6003d Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x80 lbug_with_loc+0x3a/0x70 [libcfs 465605ba0eacb0048ba48575dea685aa3aeafd42] __osc_lru_del+0x8c/0x90 [osc ac1b935f2d2d2774f2f2b35a73248b38c8a8dcf3] osc_lru_use.part.20+0x41/0x70 [osc ac1b935f2d2d2774f2f2b35a73248b38c8a8dcf3] osc_page_cache_add+0x166/0x170 [osc ac1b935f2d2d2774f2f2b35a73248b38c8a8dcf3] osc_io_commit_async+0x34c/0x510 [osc ac1b935f2d2d2774f2f2b35a73248b38c8a8dcf3] ? __pfx_write_commit_callback+0x10/0x10 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] ? __pfx_write_commit_callback+0x10/0x10 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] cl_io_commit_async+0x8e/0x170 [obdclass 7655f2cfa58cbaeac82bbd88ef55c4ca9659a820] lov_io_commit_async+0xf6/0x5d0 [lov ef475865d18129aef8095f4449f94349d8234d17] ? __pfx_write_commit_callback+0x10/0x10 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] ? __pfx_write_commit_callback+0x10/0x10 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] cl_io_commit_async+0x8e/0x170 [obdclass 7655f2cfa58cbaeac82bbd88ef55c4ca9659a820] vvp_io_write_commit+0xc1/0x5d0 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] vvp_io_write_start+0x729/0xe20 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] ? srso_alias_return_thunk+0x5/0xfbef5 ? cl_lock_enqueue+0x70/0x150 [obdclass 7655f2cfa58cbaeac82bbd88ef55c4ca9659a820] cl_io_start+0x6d/0x140 [obdclass 7655f2cfa58cbaeac82bbd88ef55c4ca9659a820] cl_io_loop+0x9e/0x230 [obdclass 7655f2cfa58cbaeac82bbd88ef55c4ca9659a820] ? ll_cl_add+0x95/0x100 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] ll_file_io_generic+0xa20/0x10a0 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] do_file_write_iter+0x1fe/0xb90 [lustre c18b2cce7f5308794da1fcf2d7a732f7112c1808] vfs_write+0x2e3/0x380 ksys_write+0xa5/0xe0 ? srso_alias_return_thunk+0x5/0xfbef5 do_syscall_64+0x58/0x80 ? do_syscall_64+0x67/0x80 ? syscall_exit_to_user_mode+0x1e/0x40 ? srso_alias_return_thunk+0x5/0xfbef5 ? do_syscall_64+0x67/0x80 ? exc_page_fault+0x69/0x150 entry_SYSCALL_64_after_hwframe+0x7c/0xe6 RIP: 0033:0x7f4c65d21770 | Link to test |