Edit new crash report

Reason	Crashing Function	Where to cut Backtrace	Reports Count
LBUG	tgt_grant_incoming	tgt_grant_incoming tgt_grant_prepare_write ofd_preprw_write ofd_preprw obd_preprw tgt_brw_write tgt_handle_request0 tgt_request_handle ptlrpc_server_handle_request ptlrpc_main kthread ret_from_fork	23

Reason

Crashing Function

Where to cut Backtrace

Reports Count

LBUG

tgt_grant_incoming

tgt_grant_incoming
tgt_grant_prepare_write
ofd_preprw_write
ofd_preprw
obd_preprw
tgt_brw_write
tgt_handle_request0
tgt_request_handle
ptlrpc_server_handle_request
ptlrpc_main
kthread
ret_from_fork

Added fields:

Match messages in logs
(every line would be required to be present in log output
Copy from "Messages before crash" column below):

Match messages in full crash
(every line would be required to be present in crash log output
Copy from "Full Crash" column below):

Limit to a test:
(Copy from below "Failing text"):

Delete these reports as invalid (real bug in review or some such)

Bug or comment:

Extra info:

Failures list (last 100):

Failing Test	Full Crash	Messages before crash	Link and date
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 961931:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 961931 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ? libcfs_debug_msg+0x946/0xd00 [libcfs] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1eb0 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 675 minutes (lustre-reviews_review-dne-zfs-part-4_134247.57) \| 2026-10-24 02:43:28 Lustre: ll_ost_io00_000: service thread pid 919136 was inactive for 40.753 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: task:ll_ost_io00_000 state:D stack:0 pid:919136 tgid:919136 ppid:2 task_flags:0x208040 flags:0x00004000 Call Trace: <TASK> __schedule+0x2aa/0x660 schedule+0x27/0xa0 io_schedule+0x46/0x70 cv_wait_common+0xac/0x130 [spl] ? __pfx_autoremove_wake_function+0x10/0x10 txg_wait_synced_flags+0xd0/0x140 [zfs] txg_wait_synced+0x10/0x40 [zfs] osd_sync+0x156/0x2c0 [osd_zfs] qsd_acquire+0x117/0x2d0 [lquota] qsd_op_begin0+0x231/0x7d0 [lquota] ? __pfx_woken_wake_function+0x10/0x10 qsd_op_begin+0x426/0x590 [lquota] osd_declare_quota+0x21e/0x600 [osd_zfs] osd_declare_write_commit+0x45f/0x830 [osd_zfs] ofd_commitrw_write+0x5e3/0x1610 [ofd] ofd_commitrw+0x761/0x1050 [ofd] obd_commitrw+0x15d/0x290 [ptlrpc] tgt_brw_write+0xca7/0x1eb0 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> task:ll_ost_io00_011 state:D stack:0 pid:961937 tgid:961937 ppid:2 task_flags:0x208040 flags:0x00004000 Call Trace: <TASK> __schedule+0x2aa/0x660 schedule+0x27/0xa0 io_schedule+0x46/0x70 cv_wait_common+0xac/0x130 [spl] ? __pfx_autoremove_wake_function+0x10/0x10 txg_wait_synced_flags+0xd0/0x140 [zfs] txg_wait_synced+0x10/0x40 [zfs] osd_sync+0x156/0x2c0 [osd_zfs] qsd_acquire+0x117/0x2d0 [lquota] qsd_op_begin0+0x231/0x7d0 [lquota] ? __pfx_woken_wake_function+0x10/0x10 qsd_op_begin+0x426/0x590 [lquota] osd_declare_quota+0x21e/0x600 [osd_zfs] osd_declare_write_commit+0x45f/0x830 [osd_zfs] ofd_commitrw_write+0x5e3/0x1610 [ofd] ofd_commitrw+0x761/0x1050 [ofd] obd_commitrw+0x15d/0x290 [ptlrpc] tgt_brw_write+0xca7/0x1eb0 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> Lustre: ll_ost_io00_011: service thread pid 961937 completed after 52.605s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources). Lustre: ll_ost_io00_000: service thread pid 919136 completed after 58.794s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources). LustreError: 961931:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0006: cli 495653ad-7db9-4fac-8581-b44fd5e8eb66/ffff8e24d0096000 dirty 5111808 pend 5111808 grant -1703936	Link to test (2026-10-23 22:54)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 798379:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 798379 Comm: ll_ost_io00_000 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1e90 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 655 minutes (lustre-reviews_review-dne-zfs-part-4_134174.57) \| 2026-10-23 07:34:31 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 827105:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0004: cli defc324f-1e77-4b0e-9576-333ff11a26b6 claims 1703936 GRANT, real grant 0 LustreError: 798379:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0004: cli defc324f-1e77-4b0e-9576-333ff11a26b6/ffff8dc7b59fb800 dirty 3407872 pend 0 grant -3407872	Link to test (2026-10-23 03:50)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 484664:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 0 PID: 484664 Comm: ll_ost_io00_003 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f90 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 565 minutes (lustre-reviews_review-dne-zfs-part-4_134187.41) \| 2026-10-23 07:17:50 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 570 minutes (lustre-reviews_review-dne-zfs-part-4_134187.41) \| 2026-10-23 07:22:51 LustreError: 484673:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0007: cli e62a040b-7516-4115-a57f-699e48c20fd9 claims 1703936 GRANT, real grant 0 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 575 minutes (lustre-reviews_review-dne-zfs-part-4_134187.41) \| 2026-10-23 07:27:51 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 Autotest: Test running for 580 minutes (lustre-reviews_review-dne-zfs-part-4_134187.41) \| 2026-10-23 07:32:53 LustreError: 484664:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0000: cli e62a040b-7516-4115-a57f-699e48c20fd9/ffff888236420c00 dirty 0 pend 0 grant -5111808	Link to test (2026-10-23 03:45)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 567835:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 0 PID: 567835 Comm: ll_ost_io00_007 Kdump: loaded Tainted: P OE ------ --- 5.14.0-611.55.1.el9_7_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.constprop.0+0x569/0x570 [ptlrpc] tgt_grant_prepare_write+0x300/0x560 [ptlrpc] ofd_preprw_write.constprop.0+0xd1/0x1340 [ofd] ofd_preprw+0x545/0x930 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xdb8/0x1be0 [ptlrpc] tgt_handle_request0+0x149/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc] ptlrpc_main+0x9be/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x104/0x110 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 395 minutes (lustre-reviews_custom_134095.1002) \| 2026-10-21 11:25:41 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 400 minutes (lustre-reviews_custom_134095.1002) \| 2026-10-21 11:30:42 LustreError: 567833:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0005: cli ab163165-b691-40ae-988b-a112a7f222bb claims 1703936 GRANT, real grant 0 LustreError: 567835:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0005: cli ab163165-b691-40ae-988b-a112a7f222bb/ffff95b7c6e7d400 dirty 8519680 pend 8519680 grant -3407872	Link to test (2026-10-21 07:45)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 827088:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 1 PID: 827088 Comm: ll_ost_io00_001 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 455 minutes (lustre-reviews_review-dne-zfs-part-4_134078.41) \| 2026-10-21 08:12:04 LustreError: 827088:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0003: cli eb7d092f-6ddf-4c25-8ef5-baeed4faf63d/ffff89b3280e4400 dirty 0 pend 0 grant -3407872	Link to test (2026-10-21 04:24)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 849531:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 1 PID: 849531 Comm: ll_ost_io00_019 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 520 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) \| 2026-10-20 15:16:19 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 525 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) \| 2026-10-20 15:21:20 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 530 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) \| 2026-10-20 15:26:21 LustreError: 849529:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0000: cli 05072d91-2963-4c59-84d9-8ac7fdde1249 claims 1703936 GRANT, real grant 0 LustreError: 849520:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0000: cli 05072d91-2963-4c59-84d9-8ac7fdde1249 claims 1703936 GRANT, real grant 0 LustreError: 849520:0:(tgt_grant.c:800:tgt_grant_check()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 535 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) \| 2026-10-20 15:31:21 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 540 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) \| 2026-10-20 15:36:22 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 LustreError: 849519:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0002: cli 05072d91-2963-4c59-84d9-8ac7fdde1249 claims 1703936 GRANT, real grant 0 LustreError: 849519:0:(tgt_grant.c:800:tgt_grant_check()) Skipped 6 previous similar messages LustreError: 849531:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0002: cli 05072d91-2963-4c59-84d9-8ac7fdde1249/ffff8c3cfa63a000 dirty 3407872 pend 0 grant -1703936	Link to test (2026-10-20 11:50)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 849247:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 1 PID: 849247 Comm: ll_ost_io00_008 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled LustreError: 810922:0:(fld_handler.c:219:fld_local_lookup()) srv-lustre-OST0006: FLD cache range [0x0000000480000400-0x00000004c0000400]:6:ost does not match requested flag 0: rc = -5 LustreError: 810922:0:(fld_handler.c:219:fld_local_lookup()) Skipped 72 previous similar messages Autotest: Test running for 455 minutes (lustre-reviews_review-dne-zfs-part-4_133938.41) \| 2026-10-19 07:09:40 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 849247:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0007: cli be03d994-7409-4f17-af87-c456febfe8b5/ffff898c5f0fd800 dirty 3407872 pend 1703936 grant -1703936	Link to test (2026-10-19 03:24)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1219610:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 0 UID: 0 PID: 1219610 Comm: ll_ost_io00_004 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ? libcfs_debug_msg+0x946/0xd00 [libcfs] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1e90 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 685 minutes (lustre-reviews_review-dne-zfs-part-4_133706.57) \| 2026-10-17 15:51:27 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 1219611:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0003: cli eb231b3c-301c-4695-8246-037af87b5fe2 claims 1703936 GRANT, real grant 0 LustreError: 1219611:0:(tgt_grant.c:800:tgt_grant_check()) Skipped 1 previous similar message LustreError: 1219610:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0003: cli eb231b3c-301c-4695-8246-037af87b5fe2/ffff8a6a7011d800 dirty 10223616 pend 10223616 grant -3407872	Link to test (2026-10-17 12:05)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1216346:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 1216346 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ? libcfs_debug_msg+0x946/0xd00 [libcfs] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1e90 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 565 minutes (lustre-reviews_review-dne-zfs-part-4_133725.57) \| 2026-10-17 15:19:23 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 570 minutes (lustre-reviews_review-dne-zfs-part-4_133725.57) \| 2026-10-17 15:24:25 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 575 minutes (lustre-reviews_review-dne-zfs-part-4_133725.57) \| 2026-10-17 15:29:25 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 LustreError: 1216346:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0002: cli 6d002035-4761-427d-b076-8351c03b3667/ffff8892f335e400 dirty 1703936 pend 0 grant -1703936	Link to test (2026-10-17 11:45)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1214578:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG CPU: 0 UID: 0 PID: 1214578 Comm: ll_ost_io00_007 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c2/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1d80 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 490 minutes (lustre-reviews_review-dne-zfs-part-4_133370.57) \| 2026-10-11 11:03:56 LustreError: 1214576:0:(tgt_grant.c:744:tgt_grant_check()) lustre-OST0000: cli fc55e168-9013-417f-80ac-7e990352aa76 claims 1703936 GRANT, real grant 0 LustreError: 1214578:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0000: cli fc55e168-9013-417f-80ac-7e990352aa76/ffff8c145ad10c00 dirty 6815744 pend 0 grant -3407872	Link to test (2026-10-11 07:17)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1149908:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 0 UID: 0 PID: 1149908 Comm: ll_ost_io00_002 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1e90 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 1178527:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0007: cli fc08883d-c194-4fcf-907d-ce77a89b6bf8 claims 1703936 GRANT, real grant 0 LustreError: 1178527:0:(tgt_grant.c:793:tgt_grant_check()) Skipped 1 previous similar message Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 540 minutes (lustre-reviews_review-dne-zfs-part-4_133293.57) \| 2026-10-10 14:27:24 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 LustreError: 1203528:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0001: cli fc08883d-c194-4fcf-907d-ce77a89b6bf8 claims 1703936 GRANT, real grant 0 LustreError: 1203528:0:(tgt_grant.c:793:tgt_grant_check()) Skipped 2 previous similar messages LustreError: 1149908:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0001: cli fc08883d-c194-4fcf-907d-ce77a89b6bf8/ffff8b4f6fb54c00 dirty 3407872 pend 0 grant -1703936	Link to test (2026-10-10 10:38)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 867196:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 0 PID: 867196 Comm: ll_ost_io00_004 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ? _raw_spin_unlock+0xa/0x30 ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 Autotest: Test running for 675 minutes (lustre-reviews_review-dne-zfs-part-4_133191.41) \| 2026-10-09 18:20:04 LustreError: 867199:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0001: cli c476eca6-d926-4e3c-b08d-0a85e4918b2a claims 1703936 GRANT, real grant 0 LustreError: 867196:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0001: cli c476eca6-d926-4e3c-b08d-0a85e4918b2a/ffff8d5bdb542400 dirty 0 pend 0 grant -1703936	Link to test (2026-10-09 14:33)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1213662:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 1213662 Comm: ll_ost_io00_007 Kdump: loaded Tainted: P OE ------ --- 6.12.0-211.47.1.el10_2_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x14c0 [ofd] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1e90 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 480 minutes (lustre-reviews_review-dne-zfs-part-4_132714.73) \| 2026-10-02 14:57:30 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 1213662:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0005: cli 9159682b-602e-4276-952e-072c36ad3309/ffff88a3ec326800 dirty 1703936 pend 0 grant -3407872	Link to test (2026-10-02 11:14)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 848096:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 0 PID: 848096 Comm: ll_ost_io00_010 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ? _raw_spin_unlock+0xa/0x30 ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 480 minutes (lustre-reviews_review-dne-zfs-part-4_132518.41) \| 2026-09-30 07:24:56 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 485 minutes (lustre-reviews_review-dne-zfs-part-4_132518.41) \| 2026-09-30 07:29:57 LustreError: 848091:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0007: cli 01edaecc-55f1-41a5-94cb-3e88d6fb319f claims 1703936 GRANT, real grant 0 LustreError: 848096:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0007: cli 01edaecc-55f1-41a5-94cb-3e88d6fb319f/ffff8ed32686c000 dirty 0 pend 0 grant -3407872	Link to test (2026-10-01 23:25)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 848096:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 0 PID: 848096 Comm: ll_ost_io00_010 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ? _raw_spin_unlock+0xa/0x30 ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 480 minutes (lustre-reviews_review-dne-zfs-part-4_132518.41) \| 2026-09-30 07:24:56 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 485 minutes (lustre-reviews_review-dne-zfs-part-4_132518.41) \| 2026-09-30 07:29:57 LustreError: 848091:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0007: cli 01edaecc-55f1-41a5-94cb-3e88d6fb319f claims 1703936 GRANT, real grant 0 LustreError: 848096:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0007: cli 01edaecc-55f1-41a5-94cb-3e88d6fb319f/ffff8ed32686c000 dirty 0 pend 0 grant -3407872	Link to test (2026-10-01 23:25)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 845785:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG CPU: 0 PID: 845785 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ? _raw_spin_unlock+0xa/0x30 ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb7/0x1f70 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 460 minutes (lustre-reviews_review-dne-zfs-part-4_132480.41) \| 2026-09-29 18:14:02 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 845785:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0005: cli 4198f24b-86d7-411e-914e-a43b828ef4e1/ffff8b363145cc00 dirty 3407872 pend 0 grant -3407872	Link to test (2026-10-01 23:25)
replay-ost-single test 8a: Verify redo io: redo io when get -EINPROGRESS error	LustreError: 161957:0:(tgt_grant.c:570:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 161957 Comm: ll_ost_io00_003 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x27a/0x570 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ? null_alloc_rs+0xe3/0x340 [ptlrpc] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14b/0x280 [ptlrpc] tgt_brw_write+0xb07/0x1fb0 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? libcfs_debug_msg+0x815/0xbc0 [libcfs] ? srso_alias_return_thunk+0x5/0xfbef5 ? vsnprintf+0x257/0x700 tgt_handle_request0.isra.0+0x144/0x880 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x293/0xda0 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ? ptlrpc_server_handle_req_in.isra.0+0x721/0xa30 [ptlrpc] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfa/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x31/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: lctl set_param fail_loc=0x230 Lustre: * cfs_fail_loc=230, val=0* LustreError: 137676:0:(tgt_grant.c:752:tgt_grant_check()) lustre-OST0000: cli 9d3a116d-0f9a-411d-a518-2bb2b3d88f2e claims 1703936 GRANT, real grant 393216 Lustre: * cfs_fail_loc=230, val=0* Lustre: Skipped 769 previous similar messages LustreError: 161957:0:(tgt_grant.c:566:tgt_grant_incoming()) lustre-OST0000: cli 9d3a116d-0f9a-411d-a518-2bb2b3d88f2e/ff33d350ea8c7400 dirty 1703936 pend 0 grant -3407872	Link to test (2026-10-01 23:25)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 863227:0:(tgt_grant.c:570:tgt_grant_incoming()) LBUG CPU: 1 PID: 863227 Comm: ll_ost_io00_009 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc] tgt_grant_prepare_write+0x266/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb1/0x1e20 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Autotest: Test running for 410 minutes (lustre-reviews_review-dne-zfs-part-4_132394.41) \| 2026-09-28 02:40:45 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 863227:0:(tgt_grant.c:566:tgt_grant_incoming()) lustre-OST0000: cli 22091308-f881-47c0-b066-48cf2e1acadb/ffff888b286f4800 dirty 1703936 pend 0 grant -1703936	Link to test (2026-09-27 22:57)
sanity-benchmark test iozone: iozone	LustreError: 51364:0:(tgt_grant.c:570:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 51364 Comm: ll_ost_io00_018 Kdump: loaded Tainted: G W OE ------ --- 6.12.0-211.47.1.el10_2_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc] tgt_grant_prepare_write+0x27a/0x570 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x14c0 [ofd] ? null_alloc_rs+0xe3/0x340 [ptlrpc] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14b/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1d80 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? libcfs_debug_msg+0x815/0xbc0 [libcfs] ? srso_alias_return_thunk+0x5/0xfbef5 ? vsnprintf+0x257/0x700 tgt_handle_request0.isra.0+0x124/0x810 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x293/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfa/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x31/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: /usr/sbin/lctl mark min OST has 7243628kB available, using 7485392kB file size Lustre: DEBUG MARKER: min OST has 7243628kB available, using 7485392kB file size Autotest: Test running for 220 minutes (lustre-master_full-part-3_4758.195) \| 2026-09-19 09:12:46 Lustre: lustre-OST0001: Client aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp) reconnecting LustreError: 51434:0:(ldlm_lib.c:3661:target_bulk_io()) @@@ Reconnect on bulk WRITE req@00000000ec30026e x1876739834721088/t0(0) o4->aa741fc5-047d-4a4b-8002-5dd0855b6e2f@10.240.23.36@tcp:722/0 lens 488/448 e 1 to 0 dl 1789809762 ref 1 fl Interpret:/0/0 rc 0/0 job:'iozone.500' uid:4294967295 gid:4294967295 projid:4294967295 Lustre: lustre-OST0001: Bulk IO write error with aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp), client will retry: rc = -110 LustreError: 51368:0:(tgt_grant.c:752:tgt_grant_check()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f claims 4218880 GRANT, real grant 2613248 LustreError: 118219:0:(tgt_grant.c:752:tgt_grant_check()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f claims 2646016 GRANT, real grant 0 Lustre: lustre-OST0002: Client aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp) reconnecting Lustre: Skipped 7 previous similar messages LustreError: 118352:0:(ldlm_lib.c:3661:target_bulk_io()) @@@ Reconnect on bulk WRITE req@000000007d179b21 x1876739834872576/t0(0) o4->aa741fc5-047d-4a4b-8002-5dd0855b6e2f@10.240.23.36@tcp:0/0 lens 488/448 e 0 to 0 dl 1789809795 ref 1 fl Interpret:/0/0 rc 0/0 job:'iozone.500' uid:4294967295 gid:4294967295 projid:4294967295 LustreError: 118352:0:(ldlm_lib.c:3661:target_bulk_io()) Skipped 4 previous similar messages Lustre: lustre-OST0003: Bulk IO write error with aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp), client will retry: rc = -110 Lustre: Skipped 4 previous similar messages LustreError: 118346:0:(tgt_grant.c:453:tgt_grant_space_left()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f/00000000b7a28c13 left=6328066048 < tot_grant=18446744073707030720 unstable=1597440 pending=5734400 dirty=0 LustreError: 51364:0:(tgt_grant.c:566:tgt_grant_incoming()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f/00000000b7a28c13 dirty 13778944 pend 11051008 grant -13721600	Link to test (2026-09-19 13:04)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1208551:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG CPU: 0 UID: 0 PID: 1208551 Comm: ll_ost_io00_004 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c2/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb06/0x1d80 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 595 minutes (lustre-reviews_review-dne-zfs-part-4_131375.69) \| 2026-09-10 10:58:15 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 600 minutes (lustre-reviews_review-dne-zfs-part-4_131375.69) \| 2026-09-10 11:03:15 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 LustreError: 1208551:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0004: cli c8655ffb-738e-4ae9-807f-6a4b2b3ff77f/ffff8e879d56b800 dirty 0 pend 0 grant -3407872	Link to test (2026-09-10 07:17)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 862676:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG CPU: 1 PID: 862676 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x33/0x50 lbug_with_loc+0x3a/0x80 [libcfs] tgt_grant_incoming.constprop.0+0x42f/0x590 [ptlrpc] tgt_grant_prepare_write+0x262/0x540 [ptlrpc] ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd] ofd_preprw+0x3cf/0x9f0 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xeb1/0x1e20 [ptlrpc] tgt_handle_request0+0x159/0x820 [ptlrpc] tgt_request_handle+0x3b7/0xd40 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? lprocfs_counter_add+0x121/0x180 [obdclass] ptlrpc_main+0x983/0xe50 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x113/0x240 ? __pfx_kthread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Autotest: Test running for 405 minutes (lustre-reviews_review-dne-zfs-part-4_131370.53) \| 2026-09-10 06:25:03 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 Autotest: Test running for 410 minutes (lustre-reviews_review-dne-zfs-part-4_131370.53) \| 2026-09-10 06:30:03 LustreError: 862676:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0001: cli 61cc3f3b-1b47-4e58-9ea3-5d92a9ff66dd/ffff8ac40241f000 dirty 0 pend 0 grant -1703936	Link to test (2026-09-10 02:40)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 812826:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG CPU: 1 PID: 812826 Comm: ll_ost_io00_001 Kdump: loaded Tainted: P OE ------ --- 5.14.0-611.55.1.el9_7_lustre.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.constprop.0+0x561/0x570 [ptlrpc] tgt_grant_prepare_write+0x2fc/0x550 [ptlrpc] ofd_preprw_write.constprop.0+0xd1/0x1340 [ofd] ofd_preprw+0x545/0x930 [ofd] obd_preprw+0x150/0x2a0 [ptlrpc] tgt_brw_write+0xdb8/0x1ad0 [ptlrpc] tgt_handle_request0+0x149/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc] ptlrpc_main+0x9be/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0x104/0x110 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x50 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 785 minutes (lustre-reviews_review-dne-zfs-part-4_131086.37) \| 2026-09-06 13:43:05 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 Autotest: Test running for 790 minutes (lustre-reviews_review-dne-zfs-part-4_131086.37) \| 2026-09-06 13:48:05 LustreError: 812826:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0006: cli 37809623-126e-4656-82e8-1a293bae938a/ffff982076b05400 dirty 1703936 pend 0 grant -3407872	Link to test (2026-09-06 10:33)
sanity-quota test 58: project ID should be kept for new mirrors created by FID	LustreError: 1161720:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG LustreError: 1134636:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG CPU: 1 UID: 0 PID: 1161720 Comm: ll_ost_io00_017 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary) Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x4e/0x70 lbug_with_loc.cold+0x5/0x50 [libcfs] tgt_grant_incoming.isra.0+0x3c2/0x580 [ptlrpc] tgt_grant_prepare_write+0x28a/0x580 [ptlrpc] ofd_preprw_write.isra.0+0xd5/0x1490 [ofd] ofd_preprw+0x3f5/0x9d0 [ofd] obd_preprw+0x14e/0x280 [ptlrpc] tgt_brw_write+0xb13/0x1db0 [ptlrpc] ? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc] tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc] tgt_request_handle+0x232/0xd50 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc] ? lprocfs_counter_add+0x12a/0x190 [obdclass] ptlrpc_main+0xaeb/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xfd/0x240 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x34/0x50 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK>	Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl dl Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 485 minutes (lustre-reviews_review-dne-zfs-part-4_129227.69) Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 LustreError: 1161715:0:(tgt_grant.c:744:tgt_grant_check()) lustre-OST0005: cli ff50c0d7-5e17-41d3-b894-754fb711da98 claims 1703936 GRANT, real grant 0 Lustre: DEBUG MARKER: zpool trim lustre-ost1 Lustre: DEBUG MARKER: zpool trim lustre-ost2 Lustre: DEBUG MARKER: zpool trim lustre-ost4 Lustre: DEBUG MARKER: zpool trim lustre-ost6 Lustre: DEBUG MARKER: zpool trim lustre-ost3 Lustre: DEBUG MARKER: zpool trim lustre-ost8 Lustre: DEBUG MARKER: zpool trim lustre-ost5 Lustre: DEBUG MARKER: zpool trim lustre-ost7 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Autotest: Test running for 490 minutes (lustre-reviews_review-dne-zfs-part-4_129227.69) Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS.force_sync=1 Lustre: DEBUG MARKER: lctl set_param -n osd.OS*.force_sync=1 LustreError: 1161715:0:(tgt_grant.c:744:tgt_grant_check()) lustre-OST0007: cli ff50c0d7-5e17-41d3-b894-754fb711da98 claims 1703936 GRANT, real grant 0 LustreError: 1161715:0:(tgt_grant.c:744:tgt_grant_check()) Skipped 1 previous similar message LustreError: 1161720:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0007: cli ff50c0d7-5e17-41d3-b894-754fb711da98/ffff893c82984000 dirty 3407872 pend 5111808 grant -10223616	Link to test (2026-08-13 21:38)

Failing Test

Full Crash

Messages before crash

Link and date

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 961931:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 961931 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
? libcfs_debug_msg+0x946/0xd00 [libcfs]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1eb0 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 675 minutes (lustre-reviews_review-dne-zfs-part-4_134247.57) | 2026-10-24 02:43:28
Lustre: ll_ost_io00_000: service thread pid 919136 was inactive for 40.753 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
task:ll_ost_io00_000 state:D stack:0 pid:919136 tgid:919136 ppid:2 task_flags:0x208040 flags:0x00004000
Call Trace:
<TASK>
__schedule+0x2aa/0x660
schedule+0x27/0xa0
io_schedule+0x46/0x70
cv_wait_common+0xac/0x130 [spl]
? __pfx_autoremove_wake_function+0x10/0x10
txg_wait_synced_flags+0xd0/0x140 [zfs]
txg_wait_synced+0x10/0x40 [zfs]
osd_sync+0x156/0x2c0 [osd_zfs]
qsd_acquire+0x117/0x2d0 [lquota]
qsd_op_begin0+0x231/0x7d0 [lquota]
? __pfx_woken_wake_function+0x10/0x10
qsd_op_begin+0x426/0x590 [lquota]
osd_declare_quota+0x21e/0x600 [osd_zfs]
osd_declare_write_commit+0x45f/0x830 [osd_zfs]
ofd_commitrw_write+0x5e3/0x1610 [ofd]
ofd_commitrw+0x761/0x1050 [ofd]
obd_commitrw+0x15d/0x290 [ptlrpc]
tgt_brw_write+0xca7/0x1eb0 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>
task:ll_ost_io00_011 state:D stack:0 pid:961937 tgid:961937 ppid:2 task_flags:0x208040 flags:0x00004000
Call Trace:
<TASK>
__schedule+0x2aa/0x660
schedule+0x27/0xa0
io_schedule+0x46/0x70
cv_wait_common+0xac/0x130 [spl]
? __pfx_autoremove_wake_function+0x10/0x10
txg_wait_synced_flags+0xd0/0x140 [zfs]
txg_wait_synced+0x10/0x40 [zfs]
osd_sync+0x156/0x2c0 [osd_zfs]
qsd_acquire+0x117/0x2d0 [lquota]
qsd_op_begin0+0x231/0x7d0 [lquota]
? __pfx_woken_wake_function+0x10/0x10
qsd_op_begin+0x426/0x590 [lquota]
osd_declare_quota+0x21e/0x600 [osd_zfs]
osd_declare_write_commit+0x45f/0x830 [osd_zfs]
ofd_commitrw_write+0x5e3/0x1610 [ofd]
ofd_commitrw+0x761/0x1050 [ofd]
obd_commitrw+0x15d/0x290 [ptlrpc]
tgt_brw_write+0xca7/0x1eb0 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>
Lustre: ll_ost_io00_011: service thread pid 961937 completed after 52.605s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
Lustre: ll_ost_io00_000: service thread pid 919136 completed after 58.794s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources).
LustreError: 961931:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0006: cli 495653ad-7db9-4fac-8581-b44fd5e8eb66/ffff8e24d0096000 dirty 5111808 pend 5111808 grant -1703936

Link to test (2026-10-23 22:54)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 798379:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 798379 Comm: ll_ost_io00_000 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1e90 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Link to test (2026-10-23 03:50)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 484664:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 0 PID: 484664 Comm: ll_ost_io00_003 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f90 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Link to test (2026-10-23 03:45)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 567835:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 0 PID: 567835 Comm: ll_ost_io00_007 Kdump: loaded Tainted: P OE ------ --- 5.14.0-611.55.1.el9_7_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x34/0x48
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.constprop.0+0x569/0x570 [ptlrpc]
tgt_grant_prepare_write+0x300/0x560 [ptlrpc]
ofd_preprw_write.constprop.0+0xd1/0x1340 [ofd]
ofd_preprw+0x545/0x930 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xdb8/0x1be0 [ptlrpc]
tgt_handle_request0+0x149/0x770 [ptlrpc]
tgt_request_handle+0x3fd/0xd00 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9be/0xea0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x104/0x110
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Link to test (2026-10-21 07:45)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 827088:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 1 PID: 827088 Comm: ll_ost_io00_001 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f70 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 455 minutes (lustre-reviews_review-dne-zfs-part-4_134078.41) | 2026-10-21 08:12:04
LustreError: 827088:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0003: cli eb7d092f-6ddf-4c25-8ef5-baeed4faf63d/ffff89b3280e4400 dirty 0 pend 0 grant -3407872

Link to test (2026-10-21 04:24)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 849531:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 1 PID: 849531 Comm: ll_ost_io00_019 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f70 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 520 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) | 2026-10-20 15:16:19
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 525 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) | 2026-10-20 15:21:20
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 530 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) | 2026-10-20 15:26:21
LustreError: 849529:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0000: cli 05072d91-2963-4c59-84d9-8ac7fdde1249 claims 1703936 GRANT, real grant 0
LustreError: 849520:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0000: cli 05072d91-2963-4c59-84d9-8ac7fdde1249 claims 1703936 GRANT, real grant 0
LustreError: 849520:0:(tgt_grant.c:800:tgt_grant_check()) Skipped 6 previous similar messages
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 535 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) | 2026-10-20 15:31:21
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 540 minutes (lustre-reviews_review-dne-zfs-part-4_134018.41) | 2026-10-20 15:36:22
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 849519:0:(tgt_grant.c:800:tgt_grant_check()) lustre-OST0002: cli 05072d91-2963-4c59-84d9-8ac7fdde1249 claims 1703936 GRANT, real grant 0
LustreError: 849519:0:(tgt_grant.c:800:tgt_grant_check()) Skipped 6 previous similar messages
LustreError: 849531:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0002: cli 05072d91-2963-4c59-84d9-8ac7fdde1249/ffff8c3cfa63a000 dirty 3407872 pend 0 grant -1703936

Link to test (2026-10-20 11:50)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 849247:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 1 PID: 849247 Comm: ll_ost_io00_008 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f70 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
LustreError: 810922:0:(fld_handler.c:219:fld_local_lookup()) srv-lustre-OST0006: FLD cache range [0x0000000480000400-0x00000004c0000400]:6:ost does not match requested flag 0: rc = -5
LustreError: 810922:0:(fld_handler.c:219:fld_local_lookup()) Skipped 72 previous similar messages
Autotest: Test running for 455 minutes (lustre-reviews_review-dne-zfs-part-4_133938.41) | 2026-10-19 07:09:40
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 849247:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0007: cli be03d994-7409-4f17-af87-c456febfe8b5/ffff898c5f0fd800 dirty 3407872 pend 1703936 grant -1703936

Link to test (2026-10-19 03:24)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1219610:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 0 UID: 0 PID: 1219610 Comm: ll_ost_io00_004 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
? libcfs_debug_msg+0x946/0xd00 [libcfs]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1e90 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Link to test (2026-10-17 12:05)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1216346:0:(tgt_grant.c:618:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 1216346 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
? libcfs_debug_msg+0x946/0xd00 [libcfs]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1e90 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 565 minutes (lustre-reviews_review-dne-zfs-part-4_133725.57) | 2026-10-17 15:19:23
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 570 minutes (lustre-reviews_review-dne-zfs-part-4_133725.57) | 2026-10-17 15:24:25
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 575 minutes (lustre-reviews_review-dne-zfs-part-4_133725.57) | 2026-10-17 15:29:25
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 1216346:0:(tgt_grant.c:614:tgt_grant_incoming()) lustre-OST0002: cli 6d002035-4761-427d-b076-8351c03b3667/ffff8892f335e400 dirty 1703936 pend 0 grant -1703936

Link to test (2026-10-17 11:45)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1214578:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
CPU: 0 UID: 0 PID: 1214578 Comm: ll_ost_io00_007 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c2/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1d80 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 490 minutes (lustre-reviews_review-dne-zfs-part-4_133370.57) | 2026-10-11 11:03:56
LustreError: 1214576:0:(tgt_grant.c:744:tgt_grant_check()) lustre-OST0000: cli fc55e168-9013-417f-80ac-7e990352aa76 claims 1703936 GRANT, real grant 0
LustreError: 1214578:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0000: cli fc55e168-9013-417f-80ac-7e990352aa76/ffff8c145ad10c00 dirty 6815744 pend 0 grant -3407872

Link to test (2026-10-11 07:17)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1149908:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG
CPU: 0 UID: 0 PID: 1149908 Comm: ll_ost_io00_002 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1e90 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 1178527:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0007: cli fc08883d-c194-4fcf-907d-ce77a89b6bf8 claims 1703936 GRANT, real grant 0
LustreError: 1178527:0:(tgt_grant.c:793:tgt_grant_check()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 540 minutes (lustre-reviews_review-dne-zfs-part-4_133293.57) | 2026-10-10 14:27:24
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 1203528:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0001: cli fc08883d-c194-4fcf-907d-ce77a89b6bf8 claims 1703936 GRANT, real grant 0
LustreError: 1203528:0:(tgt_grant.c:793:tgt_grant_check()) Skipped 2 previous similar messages
LustreError: 1149908:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0001: cli fc08883d-c194-4fcf-907d-ce77a89b6bf8/ffff8b4f6fb54c00 dirty 3407872 pend 0 grant -1703936

Link to test (2026-10-10 10:38)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 867196:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG
CPU: 0 PID: 867196 Comm: ll_ost_io00_004 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
? _raw_spin_unlock+0xa/0x30
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f70 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 675 minutes (lustre-reviews_review-dne-zfs-part-4_133191.41) | 2026-10-09 18:20:04
LustreError: 867199:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0001: cli c476eca6-d926-4e3c-b08d-0a85e4918b2a claims 1703936 GRANT, real grant 0
LustreError: 867196:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0001: cli c476eca6-d926-4e3c-b08d-0a85e4918b2a/ffff8d5bdb542400 dirty 0 pend 0 grant -1703936

Link to test (2026-10-09 14:33)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1213662:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 1213662 Comm: ll_ost_io00_007 Kdump: loaded Tainted: P OE ------ --- 6.12.0-211.47.1.el10_2_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x14c0 [ofd]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1e90 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Link to test (2026-10-02 11:14)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 848096:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG
CPU: 0 PID: 848096 Comm: ll_ost_io00_010 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
? _raw_spin_unlock+0xa/0x30
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f70 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 480 minutes (lustre-reviews_review-dne-zfs-part-4_132518.41) | 2026-09-30 07:24:56
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 485 minutes (lustre-reviews_review-dne-zfs-part-4_132518.41) | 2026-09-30 07:29:57
LustreError: 848091:0:(tgt_grant.c:793:tgt_grant_check()) lustre-OST0007: cli 01edaecc-55f1-41a5-94cb-3e88d6fb319f claims 1703936 GRANT, real grant 0
LustreError: 848096:0:(tgt_grant.c:607:tgt_grant_incoming()) lustre-OST0007: cli 01edaecc-55f1-41a5-94cb-3e88d6fb319f/ffff8ed32686c000 dirty 0 pend 0 grant -3407872

Link to test (2026-10-01 23:25)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

Link to test (2026-10-01 23:25)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 845785:0:(tgt_grant.c:611:tgt_grant_incoming()) LBUG
CPU: 0 PID: 845785 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
? _raw_spin_unlock+0xa/0x30
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb7/0x1f70 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Link to test (2026-10-01 23:25)

replay-ost-single test 8a: Verify redo io: redo io when get -EINPROGRESS error

LustreError: 161957:0:(tgt_grant.c:570:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 161957 Comm: ll_ost_io00_003 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x27a/0x570 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
? null_alloc_rs+0xe3/0x340 [ptlrpc]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14b/0x280 [ptlrpc]
tgt_brw_write+0xb07/0x1fb0 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? libcfs_debug_msg+0x815/0xbc0 [libcfs]
? srso_alias_return_thunk+0x5/0xfbef5
? vsnprintf+0x257/0x700
tgt_handle_request0.isra.0+0x144/0x880 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x293/0xda0 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
? ptlrpc_server_handle_req_in.isra.0+0x721/0xa30 [ptlrpc]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfa/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x31/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: lctl set_param fail_loc=0x230
Lustre: *** cfs_fail_loc=230, val=0***
LustreError: 137676:0:(tgt_grant.c:752:tgt_grant_check()) lustre-OST0000: cli 9d3a116d-0f9a-411d-a518-2bb2b3d88f2e claims 1703936 GRANT, real grant 393216
Lustre: *** cfs_fail_loc=230, val=0***
Lustre: Skipped 769 previous similar messages
LustreError: 161957:0:(tgt_grant.c:566:tgt_grant_incoming()) lustre-OST0000: cli 9d3a116d-0f9a-411d-a518-2bb2b3d88f2e/ff33d350ea8c7400 dirty 1703936 pend 0 grant -3407872

Link to test (2026-10-01 23:25)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 863227:0:(tgt_grant.c:570:tgt_grant_incoming()) LBUG
CPU: 1 PID: 863227 Comm: ll_ost_io00_009 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x432/0x5a0 [ptlrpc]
tgt_grant_prepare_write+0x266/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb1/0x1e20 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Link to test (2026-09-27 22:57)

sanity-benchmark test iozone: iozone

LustreError: 51364:0:(tgt_grant.c:570:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 51364 Comm: ll_ost_io00_018 Kdump: loaded Tainted: G W OE ------ --- 6.12.0-211.47.1.el10_2_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c5/0x580 [ptlrpc]
tgt_grant_prepare_write+0x27a/0x570 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x14c0 [ofd]
? null_alloc_rs+0xe3/0x340 [ptlrpc]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14b/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1d80 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
? srso_alias_return_thunk+0x5/0xfbef5
? libcfs_debug_msg+0x815/0xbc0 [libcfs]
? srso_alias_return_thunk+0x5/0xfbef5
? vsnprintf+0x257/0x700
tgt_handle_request0.isra.0+0x124/0x810 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x293/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfa/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x31/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: /usr/sbin/lctl mark min OST has 7243628kB available, using 7485392kB file size
Lustre: DEBUG MARKER: min OST has 7243628kB available, using 7485392kB file size
Autotest: Test running for 220 minutes (lustre-master_full-part-3_4758.195) | 2026-09-19 09:12:46
Lustre: lustre-OST0001: Client aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp) reconnecting
LustreError: 51434:0:(ldlm_lib.c:3661:target_bulk_io()) @@@ Reconnect on bulk WRITE req@00000000ec30026e x1876739834721088/t0(0) o4->aa741fc5-047d-4a4b-8002-5dd0855b6e2f@10.240.23.36@tcp:722/0 lens 488/448 e 1 to 0 dl 1789809762 ref 1 fl Interpret:/0/0 rc 0/0 job:'iozone.500' uid:4294967295 gid:4294967295 projid:4294967295
Lustre: lustre-OST0001: Bulk IO write error with aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp), client will retry: rc = -110
LustreError: 51368:0:(tgt_grant.c:752:tgt_grant_check()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f claims 4218880 GRANT, real grant 2613248
LustreError: 118219:0:(tgt_grant.c:752:tgt_grant_check()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f claims 2646016 GRANT, real grant 0
Lustre: lustre-OST0002: Client aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp) reconnecting
Lustre: Skipped 7 previous similar messages
LustreError: 118352:0:(ldlm_lib.c:3661:target_bulk_io()) @@@ Reconnect on bulk WRITE req@000000007d179b21 x1876739834872576/t0(0) o4->aa741fc5-047d-4a4b-8002-5dd0855b6e2f@10.240.23.36@tcp:0/0 lens 488/448 e 0 to 0 dl 1789809795 ref 1 fl Interpret:/0/0 rc 0/0 job:'iozone.500' uid:4294967295 gid:4294967295 projid:4294967295
LustreError: 118352:0:(ldlm_lib.c:3661:target_bulk_io()) Skipped 4 previous similar messages
Lustre: lustre-OST0003: Bulk IO write error with aa741fc5-047d-4a4b-8002-5dd0855b6e2f (at 10.240.23.36@tcp), client will retry: rc = -110
Lustre: Skipped 4 previous similar messages
LustreError: 118346:0:(tgt_grant.c:453:tgt_grant_space_left()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f/00000000b7a28c13 left=6328066048 < tot_grant=18446744073707030720 unstable=1597440 pending=5734400 dirty=0
LustreError: 51364:0:(tgt_grant.c:566:tgt_grant_incoming()) lustre-OST0001: cli aa741fc5-047d-4a4b-8002-5dd0855b6e2f/00000000b7a28c13 dirty 13778944 pend 11051008 grant -13721600

Link to test (2026-09-19 13:04)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1208551:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
CPU: 0 UID: 0 PID: 1208551 Comm: ll_ost_io00_004 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c2/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb06/0x1d80 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 595 minutes (lustre-reviews_review-dne-zfs-part-4_131375.69) | 2026-09-10 10:58:15
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 600 minutes (lustre-reviews_review-dne-zfs-part-4_131375.69) | 2026-09-10 11:03:15
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 1208551:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0004: cli c8655ffb-738e-4ae9-807f-6a4b2b3ff77f/ffff8e879d56b800 dirty 0 pend 0 grant -3407872

Link to test (2026-09-10 07:17)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 862676:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
CPU: 1 PID: 862676 Comm: ll_ost_io00_005 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.39.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
lbug_with_loc+0x3a/0x80 [libcfs]
tgt_grant_incoming.constprop.0+0x42f/0x590 [ptlrpc]
tgt_grant_prepare_write+0x262/0x540 [ptlrpc]
ofd_preprw_write.constprop.0+0xd4/0x14a0 [ofd]
ofd_preprw+0x3cf/0x9f0 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xeb1/0x1e20 [ptlrpc]
tgt_handle_request0+0x159/0x820 [ptlrpc]
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x113/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Autotest: Test running for 405 minutes (lustre-reviews_review-dne-zfs-part-4_131370.53) | 2026-09-10 06:25:03
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 410 minutes (lustre-reviews_review-dne-zfs-part-4_131370.53) | 2026-09-10 06:30:03
LustreError: 862676:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0001: cli 61cc3f3b-1b47-4e58-9ea3-5d92a9ff66dd/ffff8ac40241f000 dirty 0 pend 0 grant -1703936

Link to test (2026-09-10 02:40)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 812826:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
CPU: 1 PID: 812826 Comm: ll_ost_io00_001 Kdump: loaded Tainted: P OE ------ --- 5.14.0-611.55.1.el9_7_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x34/0x48
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.constprop.0+0x561/0x570 [ptlrpc]
tgt_grant_prepare_write+0x2fc/0x550 [ptlrpc]
ofd_preprw_write.constprop.0+0xd1/0x1340 [ofd]
ofd_preprw+0x545/0x930 [ofd]
obd_preprw+0x150/0x2a0 [ptlrpc]
tgt_brw_write+0xdb8/0x1ad0 [ptlrpc]
tgt_handle_request0+0x149/0x770 [ptlrpc]
tgt_request_handle+0x3fd/0xd00 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9be/0xea0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x104/0x110
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 785 minutes (lustre-reviews_review-dne-zfs-part-4_131086.37) | 2026-09-06 13:43:05
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 790 minutes (lustre-reviews_review-dne-zfs-part-4_131086.37) | 2026-09-06 13:48:05
LustreError: 812826:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0006: cli 37809623-126e-4656-82e8-1a293bae938a/ffff982076b05400 dirty 1703936 pend 0 grant -3407872

Link to test (2026-09-06 10:33)

sanity-quota test 58: project ID should be kept for new mirrors created by FID

LustreError: 1161720:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
LustreError: 1134636:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
CPU: 1 UID: 0 PID: 1161720 Comm: ll_ost_io00_017 Kdump: loaded Tainted: P OE ------ --- 6.12.0-124.56.1.el10_1_lustre.x86_64 #1 PREEMPT(voluntary)
Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4e/0x70
lbug_with_loc.cold+0x5/0x50 [libcfs]
tgt_grant_incoming.isra.0+0x3c2/0x580 [ptlrpc]
tgt_grant_prepare_write+0x28a/0x580 [ptlrpc]
ofd_preprw_write.isra.0+0xd5/0x1490 [ofd]
ofd_preprw+0x3f5/0x9d0 [ofd]
obd_preprw+0x14e/0x280 [ptlrpc]
tgt_brw_write+0xb13/0x1db0 [ptlrpc]
? tgt_request_preprocess+0xcb/0x4e0 [ptlrpc]
tgt_handle_request0.isra.0+0x127/0x810 [ptlrpc]
tgt_request_handle+0x232/0xd50 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x296/0xda0 [ptlrpc]
? lprocfs_counter_add+0x12a/0x190 [obdclass]
ptlrpc_main+0xaeb/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xfd/0x240
? __pfx_kthread+0x10/0x10
ret_from_fork+0x34/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>

Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=+quota+trace
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0000.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0001.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0002.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0003.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0004.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0005.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0006.quota_slave.enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl dl
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-zfs.lustre-OST0007.quota_slave.enabled
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 485 minutes (lustre-reviews_review-dne-zfs-part-4_129227.69)
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 1161715:0:(tgt_grant.c:744:tgt_grant_check()) lustre-OST0005: cli ff50c0d7-5e17-41d3-b894-754fb711da98 claims 1703936 GRANT, real grant 0
Lustre: DEBUG MARKER: zpool trim lustre-ost1
Lustre: DEBUG MARKER: zpool trim lustre-ost2
Lustre: DEBUG MARKER: zpool trim lustre-ost4
Lustre: DEBUG MARKER: zpool trim lustre-ost6
Lustre: DEBUG MARKER: zpool trim lustre-ost3
Lustre: DEBUG MARKER: zpool trim lustre-ost8
Lustre: DEBUG MARKER: zpool trim lustre-ost5
Lustre: DEBUG MARKER: zpool trim lustre-ost7
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Autotest: Test running for 490 minutes (lustre-reviews_review-dne-zfs-part-4_129227.69)
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
Lustre: DEBUG MARKER: lctl set_param -n osd*.*OS*.force_sync=1
LustreError: 1161715:0:(tgt_grant.c:744:tgt_grant_check()) lustre-OST0007: cli ff50c0d7-5e17-41d3-b894-754fb711da98 claims 1703936 GRANT, real grant 0
LustreError: 1161715:0:(tgt_grant.c:744:tgt_grant_check()) Skipped 1 previous similar message
LustreError: 1161720:0:(tgt_grant.c:558:tgt_grant_incoming()) lustre-OST0007: cli ff50c0d7-5e17-41d3-b894-754fb711da98/ffff893c82984000 dirty 3407872 pend 5111808 grant -10223616

Link to test (2026-08-13 21:38)

Return to new crashes list

Editing crashreport #75276

Added fields:

Failures list (last 100):