| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| sanityn test 80b: Accessing directory during migration | LustreError: 356094:0:(osd_handler.c:4174:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 356094:0:(osd_handler.c:4174:osd_destroy()) LBUG CPU: 0 PID: 356094 Comm: mdt_io00_004 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x4b8/0x630 [osd_ldiskfs] lod_sub_destroy+0x1de/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? srso_alias_return_thunk+0x5/0xfbef5 ? lod_sub_ref_del+0x1de/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xef5/0x1170 [mdd] ? srso_alias_return_thunk+0x5/0xfbef5 ? lprocfs_counter_add+0x117/0x180 [obdclass] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1381/0x1e00 [mdt] ? __entry_text_end+0x102749/0x10274d mdt_reint_rec+0x119/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x147/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ptlrpc_main+0x9bf/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK> | LustreError: 356087:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 356087:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 326841:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 326841:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 326830:0:(mdd_object.c:3928:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200016b61:0x525:0x0]: rc = -2 LustreError: 326840:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 326840:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 7 previous similar messages LustreError: 326840:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 326840:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 8 previous similar messages LustreError: 331554:0:(mdd_object.c:3928:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200016b61:0x57c:0x0]: rc = -2 Lustre: 326840:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0002: [0x200016b61:0x494:0x0]/migrate_dir is open, migrate only dentry LustreError: 356087:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 356087:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 5 previous similar messages LustreError: 356087:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 356087:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 5 previous similar messages LustreError: 326840:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 326840:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 7 previous similar messages LustreError: 326840:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 326840:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 8 previous similar messages LustreError: 326840:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 326840:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 19 previous similar messages LustreError: 326840:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 326840:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 20 previous similar messages Lustre: 326840:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0002: [0x200016b61:0x494:0x0]/migrate_dir is open, migrate only dentry Lustre: 326840:0:(mdd_dir.c:4873:mdd_migrate_object()) Skipped 1 previous similar message LustreError: 326830:0:(mdd_object.c:3928:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200016b61:0x77a:0x0]: rc = -2 Lustre: 326840:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0002: [0x2800013a1:0x6a:0x0]/file4 is open, migrate only dentry Lustre: 326840:0:(mdd_dir.c:4873:mdd_migrate_object()) Skipped 1 previous similar message Lustre: 356094:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0002: [0x2800013a1:0x87:0x0]/file3 is open, migrate only dentry Lustre: 356094:0:(mdd_dir.c:4873:mdd_migrate_object()) Skipped 1 previous similar message LustreError: 356094:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 356094:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 32 previous similar messages LustreError: 356094:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x494:0x0]/migrate_dir failed: rc = -1 LustreError: 356094:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 40 previous similar messages LustreError: 356094:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200016b66:0x7b9:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 341144:0:(osd_handler.c:4174:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 341144:0:(osd_handler.c:4174:osd_destroy()) LBUG CPU: 1 PID: 341144 Comm: mdt_io00_003 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x4b8/0x630 [osd_ldiskfs] lod_sub_destroy+0x1de/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? srso_alias_return_thunk+0x5/0xfbef5 ? lod_sub_ref_del+0x1de/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xef5/0x1170 [mdd] ? srso_alias_return_thunk+0x5/0xfbef5 ? lprocfs_counter_add+0x117/0x180 [obdclass] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1381/0x1e00 [mdt] ? __entry_text_end+0x102749/0x10274d mdt_reint_rec+0x119/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x147/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ptlrpc_main+0x9bf/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK> | LustreError: 332926:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332926:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x42b:0x0]/migrate_dir failed: rc = -1 LustreError: 341144:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x450:0x0]/target_file failed: rc = -2 LustreError: 341144:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 341144:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 341144:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 1 previous similar message LustreError: 332926:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x2800013a0:0x49:0x0]/target_file failed: rc = -2 LustreError: 332926:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 10 previous similar messages LustreError: 332926:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332926:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 8 previous similar messages LustreError: 332926:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332926:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 16 previous similar messages LustreError: 332926:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x200016b61:0x42b:0x0]/migrate_dir failed: rc = -1 LustreError: 332926:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 17 previous similar messages LustreError: 332927:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332927:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 28 previous similar messages LustreError: 332927:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x200016b61:0x42b:0x0]/migrate_dir failed: rc = -1 LustreError: 332927:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 29 previous similar messages LustreError: 332927:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0001: nlink == 0 on [0x2400032e5:0xe7:0x0], maybe an upgraded file? (LU-3915) Lustre: 332917:0:(out_handler.c:877:out_tx_end()) lustre-MDT0003-osd: error during execution of #4 from /tmp/rpmbuild-lustre-jenkins-jvnTOacG/BUILD/lustre-2.17.50_72_g378a63d/lustre/ptlrpc/../../lustre/target/out_handler.c:559: rc = -2 LustreError: 332917:0:(out_handler.c:886:out_tx_end()) lustre-MDT0003-osd: undo for /tmp/rpmbuild-lustre-jenkins-jvnTOacG/BUILD/lustre-2.17.50_72_g378a63d/lustre/ptlrpc/../../lustre/target/out_handler.c:407: rc = -524 Lustre: 332927:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0003: [0x2c0000bd1:0x27:0x0]/file4 is open, migrate only dentry LustreError: 341146:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0001: nlink == 0 on [0x2400032e5:0x118:0x0], maybe an upgraded file? (LU-3915) Lustre: 332918:0:(out_handler.c:877:out_tx_end()) lustre-MDT0003-osd: error during execution of #4 from /tmp/rpmbuild-lustre-jenkins-jvnTOacG/BUILD/lustre-2.17.50_72_g378a63d/lustre/ptlrpc/../../lustre/target/out_handler.c:559: rc = -2 LustreError: 332918:0:(out_handler.c:886:out_tx_end()) lustre-MDT0003-osd: undo for /tmp/rpmbuild-lustre-jenkins-jvnTOacG/BUILD/lustre-2.17.50_72_g378a63d/lustre/ptlrpc/../../lustre/target/out_handler.c:407: rc = -524 LustreError: 332918:0:(out_handler.c:886:out_tx_end()) Skipped 2 previous similar messages LustreError: 341146:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 341146:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 45 previous similar messages LustreError: 341146:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x42b:0x0]/migrate_dir failed: rc = -1 LustreError: 341146:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 50 previous similar messages Lustre: 332926:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0003: [0x200016b61:0x42b:0x0]/migrate_dir is open, migrate only dentry LustreError: 332927:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x2400032e0:0x3a6:0x0]/target_file failed: rc = -2 LustreError: 332927:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 75 previous similar messages LustreError: 332927:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332927:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 70 previous similar messages LustreError: 332911:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0001: nlink == 0 on [0x2400032e5:0x2e8:0x0], maybe an upgraded file? (LU-3915) LustreError: 332911:0:(osd_handler.c:4792:osd_ref_del()) Skipped 1 previous similar message LustreError: 341144:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0001: nlink == 0 on [0x2400032e5:0x33a:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 385999:0:(osd_handler.c:4174:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 385999:0:(osd_handler.c:4174:osd_destroy()) LBUG CPU: 1 PID: 385999 Comm: mdt_io00_004 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-570.62.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x4b8/0x630 [osd_ldiskfs] lod_sub_destroy+0x1de/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? srso_alias_return_thunk+0x5/0xfbef5 ? lod_sub_ref_del+0x1de/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xef5/0x1170 [mdd] ? srso_alias_return_thunk+0x5/0xfbef5 ? lprocfs_counter_add+0x117/0x180 [obdclass] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1381/0x1e00 [mdt] ? __entry_text_end+0x102739/0x10273d mdt_reint_rec+0x119/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x147/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ptlrpc_main+0x9bf/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK> | Lustre: 354363:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0000: [0x200016b60:0x3:0x0]/file3 is open, migrate only dentry LustreError: 385994:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 385994:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x4f3:0x0]/migrate_dir failed: rc = -1 Lustre: 354364:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0000: [0x200016b60:0x8:0x0]/file4 is open, migrate only dentry LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 11 previous similar messages LustreError: 354364:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x4f3:0x0]/migrate_dir failed: rc = -1 LustreError: 354364:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 11 previous similar messages Lustre: 385994:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0000: [0x200016b61:0x4f3:0x0]/migrate_dir is open, migrate only dentry Lustre: 385994:0:(mdd_dir.c:4873:mdd_migrate_object()) Skipped 2 previous similar messages LustreError: 354363:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200016b66:0x534:0x0], maybe an upgraded file? (LU-3915) Lustre: 354355:0:(out_handler.c:877:out_tx_end()) lustre-MDT0002-osd: error during execution of #4 from /tmp/rpmbuild-lustre-jenkins-oghYb4ZV/BUILD/lustre-2.17.50_62_g56ee267/lustre/ptlrpc/../../lustre/target/out_handler.c:559: rc = -2 LustreError: 354355:0:(out_handler.c:886:out_tx_end()) lustre-MDT0002-osd: undo for /tmp/rpmbuild-lustre-jenkins-oghYb4ZV/BUILD/lustre-2.17.50_62_g56ee267/lustre/ptlrpc/../../lustre/target/out_handler.c:407: rc = -524 LustreError: 354363:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2800013a0:0xcb:0x0]/source_file failed: rc = -2 LustreError: 354363:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 12 previous similar messages LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 11 previous similar messages LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 4 previous similar messages LustreError: 354364:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200016b61:0x4f3:0x0]/migrate_dir failed: rc = -1 LustreError: 354364:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 5 previous similar messages Lustre: 354363:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0002: [0x2800013a1:0x2b:0x0]/file4 is open, migrate only dentry LustreError: 354363:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200016b66:0x5b6:0x0], maybe an upgraded file? (LU-3915) LustreError: 354363:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 354363:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 354363:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x4f3:0x0]/migrate_dir failed: rc = -1 LustreError: 354363:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 5 previous similar messages Lustre: 354364:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0000: [0x200016b60:0x4a:0x0]/file4 is open, migrate only dentry Lustre: 354364:0:(mdd_dir.c:4873:mdd_migrate_object()) Skipped 1 previous similar message LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 354364:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 31 previous similar messages LustreError: 354364:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x4f3:0x0]/migrate_dir failed: rc = -1 LustreError: 354364:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 32 previous similar messages LustreError: 385998:0:(mdd_object.c:3928:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200016b61:0x8ab:0x0]: rc = -2 Lustre: 385999:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0000: [0x200016b60:0x8b:0x0]/file4 is open, migrate only dentry Lustre: 385999:0:(mdd_dir.c:4873:mdd_migrate_object()) Skipped 5 previous similar messages LustreError: 354363:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 354363:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 89 previous similar messages LustreError: 354363:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x4f3:0x0]/migrate_dir failed: rc = -1 LustreError: 354363:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 90 previous similar messages LustreError: 354362:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200016b66:0x80b:0x0], maybe an upgraded file? (LU-3915) LustreError: 354362:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200016b66:0x8c9:0x0], maybe an upgraded file? (LU-3915) LustreError: 354353:0:(mdd_object.c:3928:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x200016b61:0xbb1:0x0]: rc = -2 LustreError: 354363:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200016b66:0x922:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 327912:0:(osd_handler.c:4174:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 327912:0:(osd_handler.c:4174:osd_destroy()) LBUG CPU: 1 PID: 327912 Comm: mdt_io00_001 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x4b8/0x630 [osd_ldiskfs] lod_sub_destroy+0x1de/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? srso_alias_return_thunk+0x5/0xfbef5 ? lod_sub_ref_del+0x1de/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xef5/0x1170 [mdd] ? srso_alias_return_thunk+0x5/0xfbef5 ? lprocfs_counter_add+0x117/0x180 [obdclass] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1381/0x1e00 [mdt] ? __entry_text_end+0x102749/0x10274d mdt_reint_rec+0x119/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x147/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ptlrpc_main+0x9bf/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK> | LustreError: 327913:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2400032e0:0x1f:0x0]/link_file failed: rc = -2 LustreError: 327913:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 357055:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200016b61:0x4b2:0x0]/link_file failed: rc = -2 LustreError: 357055:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 6 previous similar messages LustreError: 327913:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 327913:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 327912:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2800013a0:0x4d:0x0]/link_file failed: rc = -2 LustreError: 327912:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 3 previous similar messages LustreError: 327912:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 357055:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 357055:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 16 previous similar messages LustreError: 357055:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x48f:0x0]/migrate_dir failed: rc = -1 LustreError: 357055:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 18 previous similar messages Lustre: 327913:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0002: [0x2800013a1:0x21:0x0]/file4 is open, migrate only dentry LustreError: 357055:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 357055:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 357055:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200016b61:0x48f:0x0]/migrate_dir failed: rc = -1 LustreError: 357055:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 3 previous similar messages Lustre: 327913:0:(mdd_dir.c:4873:mdd_migrate_object()) lustre-MDD0000: [0x200016b60:0x33:0x0]/file3 is open, migrate only dentry LustreError: 327913:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 327913:0:(mdd_dir.c:4789:mdd_migrate_cmd_check()) Skipped 37 previous similar messages LustreError: 327913:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200016b61:0x48f:0x0]/migrate_dir failed: rc = -1 LustreError: 327913:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 44 previous similar messages LustreError: 327912:0:(osd_handler.c:4792:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x2800013a2:0x26f:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 122682:0:(osd_handler.c:4044:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 122682:0:(osd_handler.c:4044:osd_destroy()) LBUG CPU: 0 PID: 122682 Comm: mdt_io00_003 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-503.40.1_lustre.ddn1.el9.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1de/0x4b0 [lod] lod_destroy+0x266/0xa10 [lod] ? srso_alias_return_thunk+0x5/0xfbef5 ? lod_sub_ref_del+0x1de/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xf26/0x1500 [mdd] ? srso_alias_return_thunk+0x5/0xfbef5 ? lprocfs_counter_add+0x109/0x170 [obdclass] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x132b/0x1d10 [mdt] ? __entry_text_end+0x102749/0x10274d mdt_reint_rec+0x119/0x270 [mdt] mdt_reint_internal+0x4e4/0x850 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x147/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2ca/0xda0 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ptlrpc_main+0xa7b/0xfa0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK> | Lustre: 5446:0:(mdd_dir.c:4755:mdd_migrate_object()) lustre-MDD0000: [0x2000153f5:0xbb9:0x0]/file3 is open, migrate only dentry LustreError: 122682:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 122682:0:(mdt_reint.c:2581:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000412:0x2d4:0x0]/migrate_dir failed: rc = -1 LustreError: 5446:0:(mdt_reint.c:2581:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2000153f5:0xbec:0x0]/target_file failed: rc = -2 LustreError: 5446:0:(mdt_reint.c:2581:mdt_reint_migrate()) Skipped 17 previous similar messages LustreError: 5446:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 5446:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 381178:0:(mdt_reint.c:2581:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000412:0x326:0x0]/source_file failed: rc = -2 LustreError: 381178:0:(mdt_reint.c:2581:mdt_reint_migrate()) Skipped 30 previous similar messages LustreError: 381178:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 381178:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) Skipped 15 previous similar messages LustreError: 381178:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 381178:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) Skipped 6 previous similar messages LustreError: 381178:0:(mdt_reint.c:2581:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000412:0x2d4:0x0]/migrate_dir failed: rc = -1 LustreError: 381178:0:(mdt_reint.c:2581:mdt_reint_migrate()) Skipped 7 previous similar messages LustreError: 381178:0:(mdt_reint.c:2581:mdt_reint_migrate()) lustre-MDT0002: migrate [0x240002b13:0xf326:0x0]/source_file failed: rc = -2 LustreError: 381178:0:(mdt_reint.c:2581:mdt_reint_migrate()) Skipped 20 previous similar messages LustreError: 5446:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 5446:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) Skipped 15 previous similar messages Lustre: 381178:0:(mdd_dir.c:4755:mdd_migrate_object()) lustre-MDD0002: [0x280000bd3:0x44:0x0]/file4 is open, migrate only dentry Lustre: 381178:0:(mdd_dir.c:4755:mdd_migrate_object()) Skipped 1 previous similar message LustreError: 122682:0:(mdt_reint.c:2581:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000412:0x2d4:0x0]/migrate_dir failed: rc = -1 LustreError: 122682:0:(mdt_reint.c:2581:mdt_reint_migrate()) Skipped 77 previous similar messages LustreError: 381178:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 381178:0:(mdd_dir.c:4672:mdd_migrate_cmd_check()) Skipped 30 previous similar messages Lustre: 5446:0:(mdd_dir.c:4755:mdd_migrate_object()) lustre-MDD0002: [0x2c0000412:0x2d4:0x0]/migrate_dir is open, migrate only dentry LustreError: 122682:0:(osd_handler.c:4657:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x280000bd7:0x165c:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 340572:0:(osd_handler.c:4158:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 340572:0:(osd_handler.c:4158:osd_destroy()) LBUG CPU: 1 PID: 340572 Comm: mdt_io00_004 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x4b8/0x630 [osd_ldiskfs] lod_sub_destroy+0x1de/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? srso_alias_return_thunk+0x5/0xfbef5 ? lod_sub_ref_del+0x1de/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xef5/0x1170 [mdd] ? srso_alias_return_thunk+0x5/0xfbef5 ? lprocfs_counter_add+0x117/0x180 [obdclass] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1381/0x1e00 [mdt] ? __entry_text_end+0x102749/0x10274d mdt_reint_rec+0x119/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x147/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc] ? srso_alias_return_thunk+0x5/0xfbef5 ptlrpc_main+0x9bf/0xea0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xdd/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x29/0x50 </TASK> | Lustre: 332378:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0001: [0x2400032e0:0x13:0x0]/file3 is open, migrate only dentry LustreError: 332378:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000bd0:0x9:0x0]/source_file failed: rc = -2 LustreError: 332378:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332378:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x200016b61:0x51e:0x0]/migrate_dir failed: rc = -1 LustreError: 332378:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 6 previous similar messages LustreError: 332378:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332378:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 6 previous similar messages LustreError: 332377:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332377:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 332377:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x51e:0x0]/migrate_dir failed: rc = -1 LustreError: 332377:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 5 previous similar messages LustreError: 340569:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 340569:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 13 previous similar messages LustreError: 340569:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x200016b61:0x51e:0x0]/migrate_dir failed: rc = -1 LustreError: 340569:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 13 previous similar messages Lustre: 332377:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0001: [0x2400032e1:0x25:0x0]/file4 is open, migrate only dentry Lustre: 332377:0:(mdd_dir.c:4823:mdd_migrate_object()) Skipped 2 previous similar messages LustreError: 332377:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332377:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 29 previous similar messages LustreError: 332377:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x51e:0x0]/migrate_dir failed: rc = -1 LustreError: 332377:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 29 previous similar messages LustreError: 332378:0:(osd_handler.c:4776:osd_ref_del()) lustre-MDT0003: nlink == 0 on [0x2c0000bd5:0xe9:0x0], maybe an upgraded file? (LU-3915) LustreError: 340569:0:(osd_handler.c:4776:osd_ref_del()) lustre-MDT0003: nlink == 0 on [0x2c0000bd5:0x125:0x0], maybe an upgraded file? (LU-3915) Lustre: 340569:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0003: [0x2c0000bd1:0x44:0x0]/file4 is open, migrate only dentry LustreError: 340569:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 340569:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 58 previous similar messages LustreError: 340569:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x51e:0x0]/migrate_dir failed: rc = -1 LustreError: 340569:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 61 previous similar messages Lustre: 340572:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0001: [0x200016b61:0x51e:0x0]/migrate_dir is open, migrate only dentry Lustre: 340569:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0001: [0x200016b61:0x51e:0x0]/migrate_dir is open, migrate only dentry LustreError: 332377:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 332377:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 75 previous similar messages LustreError: 332377:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x200016b61:0x51e:0x0]/migrate_dir failed: rc = -1 LustreError: 332377:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 85 previous similar messages LustreError: 340572:0:(osd_handler.c:4776:osd_ref_del()) lustre-MDT0003: nlink == 0 on [0x2c0000bd5:0x504:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 337467:0:(osd_handler.c:4079:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 337467:0:(osd_handler.c:4079:osd_destroy()) LBUG CPU: 0 PID: 337467 Comm: mdt_io00_004 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xf4d/0x11c0 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1384/0x1e00 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x3fd/0xd00 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc] ptlrpc_main+0x9e3/0xed0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 117335:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 117335:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000410:0x3bb:0x0]/migrate_dir failed: rc = -1 LustreError: 117335:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 117335:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 117335:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000410:0x3bb:0x0]/migrate_dir failed: rc = -1 LustreError: 117335:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 3 previous similar messages LustreError: 50938:0:(mdd_object.c:3889:mdd_close()) lustre-MDD0003: failed to get lu_attr of [0x2c0000410:0x426:0x0]: rc = -2 Lustre: 9058:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0003: [0x2c0000410:0x3bb:0x0]/migrate_dir is open, migrate only dentry LustreError: 9058:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 9058:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 6 previous similar messages LustreError: 9058:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000410:0x3bb:0x0]/migrate_dir failed: rc = -1 LustreError: 9058:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 6 previous similar messages Lustre: 117335:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0003: [0x2c0000410:0x3bb:0x0]/migrate_dir is open, migrate only dentry Lustre: 117335:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0003: [0x2c0000400:0xe9:0x0]/file3 is open, migrate only dentry LustreError: 117335:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 117335:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 21 previous similar messages LustreError: 117335:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000410:0x3bb:0x0]/migrate_dir failed: rc = -1 LustreError: 117335:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 21 previous similar messages Lustre: 117335:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0001: [0x240002b11:0xf4:0x0]/source_file is open, migrate only dentry LustreError: 9058:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 9058:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 25 previous similar messages LustreError: 9058:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000410:0x3bb:0x0]/migrate_dir failed: rc = -1 LustreError: 9058:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 26 previous similar messages Lustre: 9058:0:(mdd_dir.c:4823:mdd_migrate_object()) lustre-MDD0003: [0x2c0000410:0x3bb:0x0]/migrate_dir is open, migrate only dentry Lustre: 9058:0:(mdd_dir.c:4823:mdd_migrate_object()) Skipped 4 previous similar messages LustreError: 337467:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 337467:0:(mdd_dir.c:4740:mdd_migrate_cmd_check()) Skipped 52 previous similar messages LustreError: 337467:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0001: migrate [0x2c0000410:0x3bb:0x0]/migrate_dir failed: rc = -1 LustreError: 337467:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 54 previous similar messages LustreError: 337467:0:(osd_handler.c:4692:osd_ref_del()) lustre-MDT0001: nlink == 0 on [0x240002b15:0x1721:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 6109:0:(osd_handler.c:3964:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 6109:0:(osd_handler.c:3964:osd_destroy()) LBUG CPU: 0 PID: 6109 Comm: mdt_io00_001 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xf45/0x11c0 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1384/0x1e00 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4ea/0x9b0 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x1eb/0xb80 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc] ptlrpc_main+0xa7b/0xfa0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 93263:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 93263:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000410:0x3a7:0x0]/migrate_dir failed: rc = -1 LustreError: 6111:0:(osd_handler.c:4577:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x280000bd7:0x1481:0x0], maybe an upgraded file? (LU-3915) Lustre: 8043:0:(out_handler.c:877:out_tx_end()) lustre-MDT0000-osd: error during execution of #4 from /tmp/rpmbuild-lustre-jenkins-gfYSYCH5/BUILD/lustre-2.16.56_57_g2283689/lustre/ptlrpc/../../lustre/target/out_handler.c:559: rc = -2 LustreError: 8043:0:(out_handler.c:886:out_tx_end()) lustre-MDT0000-osd: undo for /tmp/rpmbuild-lustre-jenkins-gfYSYCH5/BUILD/lustre-2.16.56_57_g2283689/lustre/ptlrpc/../../lustre/target/out_handler.c:407: rc = -524 LustreError: 6111:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2000153f4:0xcd9:0x0]/source_file failed: rc = -2 LustreError: 6111:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 2 previous similar messages LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) Skipped 2 previous similar messages LustreError: 6111:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000410:0x3a7:0x0]/migrate_dir failed: rc = -1 LustreError: 6111:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 6109:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000410:0x3a7:0x0]/migrate_dir failed: rc = -1 LustreError: 6109:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 12 previous similar messages LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) Skipped 10 previous similar messages LustreError: 93263:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000410:0x3a7:0x0]/migrate_dir failed: rc = -1 LustreError: 93263:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 10 previous similar messages Lustre: 6109:0:(mdd_dir.c:4824:mdd_migrate_object()) lustre-MDD0002: [0x280000bd4:0x1d:0x0]/file1 is open, migrate only dentry LustreError: 336389:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 336389:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) Skipped 9 previous similar messages LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 6111:0:(mdd_dir.c:4741:mdd_migrate_cmd_check()) Skipped 23 previous similar messages LustreError: 6111:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000410:0x3a7:0x0]/migrate_dir failed: rc = -1 LustreError: 6111:0:(mdt_reint.c:2562:mdt_reint_migrate()) Skipped 25 previous similar messages Lustre: 336389:0:(mdd_dir.c:4824:mdd_migrate_object()) lustre-MDD0002: [0x2c0000410:0x3a7:0x0]/migrate_dir is open, migrate only dentry LustreError: 6111:0:(osd_handler.c:4577:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x280000bd7:0x15c6:0x0], maybe an upgraded file? (LU-3915) Lustre: 53104:0:(out_handler.c:877:out_tx_end()) lustre-MDT0000-osd: error during execution of #4 from /tmp/rpmbuild-lustre-jenkins-gfYSYCH5/BUILD/lustre-2.16.56_57_g2283689/lustre/ptlrpc/../../lustre/target/out_handler.c:559: rc = -2 LustreError: 53104:0:(out_handler.c:886:out_tx_end()) lustre-MDT0000-osd: undo for /tmp/rpmbuild-lustre-jenkins-gfYSYCH5/BUILD/lustre-2.16.56_57_g2283689/lustre/ptlrpc/../../lustre/target/out_handler.c:407: rc = -524 LustreError: 53104:0:(out_handler.c:886:out_tx_end()) Skipped 2 previous similar messages LustreError: 6109:0:(osd_handler.c:4577:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x280000bd7:0x15ec:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 343165:0:(osd_handler.c:3964:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 343165:0:(osd_handler.c:3964:osd_destroy()) LBUG CPU: 1 PID: 343165 Comm: mdt_io00_003 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-503.26.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x26c/0xb60 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xe89/0x1320 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1265/0x1bf0 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4e4/0x850 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x1eb/0xb80 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2cd/0xd20 [ptlrpc] ptlrpc_main+0xa7b/0xfa0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 343165:0:(mdt_reint.c:2511:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200000404:0xbbf:0x0]/file1 failed: rc = -16 LustreError: 343165:0:(mdt_reint.c:2511:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 300841:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300841:0:(mdt_reint.c:2511:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x2c7:0x0]/migrate_dir failed: rc = -114 LustreError: 300841:0:(mdt_reint.c:2511:mdt_reint_migrate()) Skipped 13 previous similar messages LustreError: 300841:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300841:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 5 previous similar messages LustreError: 300840:0:(mdt_reint.c:2511:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000404:0xc55:0x0]/source_file failed: rc = -2 LustreError: 300840:0:(mdt_reint.c:2511:mdt_reint_migrate()) Skipped 7 previous similar messages LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 8 previous similar messages LustreError: 300840:0:(mdt_reint.c:2511:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000404:0x2c7:0x0]/migrate_dir failed: rc = -1 LustreError: 300840:0:(mdt_reint.c:2511:mdt_reint_migrate()) Skipped 26 previous similar messages LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 343165:0:(mdt_reint.c:2511:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x423:0x0]/target_file failed: rc = -2 LustreError: 343165:0:(mdt_reint.c:2511:mdt_reint_migrate()) Skipped 51 previous similar messages LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 43 previous similar messages LustreError: 343165:0:(mdt_reint.c:2511:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000404:0x577:0x0]/file3 failed: rc = -16 LustreError: 343165:0:(mdt_reint.c:2511:mdt_reint_migrate()) Skipped 110 previous similar messages LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300840:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 40 previous similar messages LustreError: 300840:0:(osd_handler.c:4577:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200000405:0x10c7:0x0], maybe an upgraded file? (LU-3915) Lustre: 300832:0:(out_handler.c:909:out_tx_end()) lustre-MDT0002-osd: error during execution of #4 from /tmp/rpmbuild-lustre-jenkins-mCnDu4de/BUILD/lustre-2.16.52_122_g02cb6be/lustre/ptlrpc/../../lustre/target/out_handler.c:582: rc = -2 LustreError: 300832:0:(out_handler.c:918:out_tx_end()) lustre-MDT0002-osd: undo for /tmp/rpmbuild-lustre-jenkins-mCnDu4de/BUILD/lustre-2.16.52_122_g02cb6be/lustre/ptlrpc/../../lustre/target/out_handler.c:422: rc = -524 LustreError: 343165:0:(osd_handler.c:4577:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200000405:0x1152:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 2345798:0:(osd_handler.c:4026:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 2345798:0:(osd_handler.c:4026:osd_destroy()) LBUG CPU: 0 PID: 2345798 Comm: mdt_io00_004 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-427.42.1_lustre.ddn1.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x266/0xa10 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xe84/0x1130 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1224/0x1bb0 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4e4/0x850 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x1eb/0xb80 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2cd/0xd20 [ptlrpc] ptlrpc_main+0xa7e/0xfa0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 2077479:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 2077479:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000404:0x5052:0x0]/migrate_dir failed: rc = -1 LustreError: 2077479:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 2097275:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x280000404:0x5052:0x0]/migrate_dir failed: rc = -114 LustreError: 2097275:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 7 previous similar messages LustreError: 2097275:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 2097275:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 2 previous similar messages LustreError: 2097275:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000404:0x50d1:0x0]/source_file failed: rc = -114 LustreError: 2097275:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 7 previous similar messages LustreError: 2097275:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 2077479:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 2077479:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 6 previous similar messages LustreError: 2097275:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x280000404:0x5052:0x0]/migrate_dir failed: rc = -1 LustreError: 2097275:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 15 previous similar messages LustreError: 2097275:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 2097275:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 5 previous similar messages LustreError: 2077479:0:(osd_handler.c:4639:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x280000405:0x153d:0x0], maybe an upgraded file? (LU-3915) LustreError: 2077479:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0002: migrate [0x280000404:0x5052:0x0]/migrate_dir failed: rc = -1 LustreError: 2077479:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 49 previous similar messages LustreError: 2077479:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 2077479:0:(mdd_dir.c:4614:mdd_migrate_cmd_check()) Skipped 17 previous similar messages LustreError: 2345798:0:(osd_handler.c:4639:osd_ref_del()) lustre-MDT0002: nlink == 0 on [0x280000405:0x15b1:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 346108:0:(osd_handler.c:4024:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 346108:0:(osd_handler.c:4024:osd_destroy()) LBUG CPU: 0 PID: 346108 Comm: mdt_io00_003 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-427.42.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x58 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x266/0xa00 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] mdd_migrate_object+0xe84/0x1130 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1224/0x1bb0 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4e4/0x850 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x1eb/0xb80 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2cd/0xd20 [ptlrpc] ptlrpc_main+0xa7e/0xfa0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 346108:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x2c0:0x0]/migrate_dir failed: rc = -114 LustreError: 346108:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 346108:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 301504:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 301504:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 346108:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000404:0x2c0:0x0]/migrate_dir failed: rc = -114 LustreError: 346108:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 21 previous similar messages LustreError: 301504:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 301504:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) Skipped 13 previous similar messages LustreError: 346108:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x2c0:0x0]/migrate_dir failed: rc = -1 LustreError: 346108:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 49 previous similar messages LustreError: 301504:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 301504:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) Skipped 15 previous similar messages LustreError: 571656:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x2c0:0x0]/migrate_dir failed: rc = -1 LustreError: 571656:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 106 previous similar messages LustreError: 571656:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 571656:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) Skipped 62 previous similar messages LustreError: 346108:0:(osd_handler.c:4637:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200000405:0xeeb:0x0], maybe an upgraded file? (LU-3915) LustreError: 301504:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x2c0:0x0]/migrate_dir failed: rc = -1 LustreError: 301504:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 207 previous similar messages LustreError: 346108:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 346108:0:(mdd_dir.c:4452:mdd_migrate_cmd_check()) Skipped 97 previous similar messages LustreError: 346108:0:(osd_handler.c:4637:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200000405:0xfd4:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 300567:0:(osd_handler.c:4018:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 300567:0:(osd_handler.c:4018:osd_destroy()) LBUG CPU: 1 PID: 300567 Comm: mdt_io00_002 Kdump: loaded Tainted: G W OE ------- --- 5.14.0-362.24.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x58 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x266/0xa00 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] ? asm_sysvec_call_function_single+0x16/0x20 mdd_migrate_object+0xe84/0x1130 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1224/0x1bb0 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4e4/0x850 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x1eb/0xb80 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2cd/0xd20 [ptlrpc] ptlrpc_main+0xaa0/0xfe0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 461458:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0000: migrate [0x2c0000404:0x2b5:0x0]/migrate_dir failed: rc = -114 LustreError: 461458:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 1 previous similar message LustreError: 300567:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0002: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 3 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 300567:0:(mdd_dir.c:4472:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 461458:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0002: migrate [0x2c0000404:0x2b5:0x0]/migrate_dir failed: rc = -1 LustreError: 461458:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 10 previous similar messages LustreError: 300567:0:(osd_handler.c:4631:osd_ref_del()) lustre-MDT0000: nlink == 0 on [0x200000405:0xf80:0x0], maybe an upgraded file? (LU-3915) | Link to test |
| sanityn test 80b: Accessing directory during migration | LustreError: 453714:0:(osd_handler.c:4017:osd_destroy()) ASSERTION( !lu_object_is_dying(dt->do_lu.lo_header) ) failed: LustreError: 453714:0:(osd_handler.c:4017:osd_destroy()) LBUG CPU: 1 PID: 453714 Comm: mdt_io00_003 Kdump: loaded Tainted: G OE ------- --- 5.14.0-362.24.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x58 [libcfs] osd_destroy+0x477/0x5f0 [osd_ldiskfs] lod_sub_destroy+0x1e1/0x4b0 [lod] lod_destroy+0x266/0xa00 [lod] ? lod_sub_ref_del+0x1e1/0x4b0 [lod] mdd_migrate_create+0x3f9/0x8f0 [mdd] ? __ldiskfs_journal_start_sb+0x111/0x130 [ldiskfs] mdd_migrate_object+0xe84/0x1130 [mdd] ? mdd_migrate+0x14/0x20 [mdd] mdd_migrate+0x14/0x20 [mdd] mdt_reint_migrate+0x1224/0x1bb0 [mdt] mdt_reint_rec+0x11c/0x270 [mdt] mdt_reint_internal+0x4e4/0x850 [mdt] mdt_reint+0x59/0x110 [mdt] tgt_handle_request0+0x14a/0x770 [ptlrpc] tgt_request_handle+0x1eb/0xb80 [ptlrpc] ptlrpc_server_handle_request.isra.0+0x2cd/0xd20 [ptlrpc] ptlrpc_main+0xaa0/0xfe0 [ptlrpc] ? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc] kthread+0xe0/0x100 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x50 </TASK> | LustreError: 289544:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 289544:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0001: migrate [0x2c0000404:0x2c1:0x0]/migrate_dir failed: rc = -1 LustreError: 289543:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 2 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 289543:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) Skipped 3 previous similar messages LustreError: 289543:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000404:0x2c1:0x0]/migrate_dir failed: rc = -1 LustreError: 289543:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 10 previous similar messages LustreError: 453714:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 453714:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) Skipped 15 previous similar messages LustreError: 453714:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000404:0x2c1:0x0]/migrate_dir failed: rc = -1 LustreError: 453714:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 33 previous similar messages LustreError: 289543:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 289543:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) Skipped 19 previous similar messages LustreError: 308258:0:(mdd_object.c:3881:mdd_close()) lustre-MDD0003: failed to get lu_attr of [0x2c0000404:0x329:0x0]: rc = -2 LustreError: 536286:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000404:0x34f:0x0]/source_file failed: rc = -114 LustreError: 536286:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 21 previous similar messages LustreError: 536286:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 536286:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) Skipped 13 previous similar messages LustreError: 289543:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0001: migrate [0x2c0000404:0x2c1:0x0]/migrate_dir failed: rc = -1 LustreError: 289543:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 69 previous similar messages LustreError: 289543:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0001: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 0 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 289543:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) Skipped 38 previous similar messages LustreError: 289543:0:(mdt_reint.c:2531:mdt_reint_migrate()) lustre-MDT0003: migrate [0x2c0000404:0x2c1:0x0]/migrate_dir failed: rc = -1 LustreError: 289543:0:(mdt_reint.c:2531:mdt_reint_migrate()) Skipped 129 previous similar messages LustreError: 453714:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) lustre-MDD0003: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration: rc = -1 LustreError: 453714:0:(mdd_dir.c:4467:mdd_migrate_cmd_check()) Skipped 92 previous similar messages LustreError: 453714:0:(osd_handler.c:4627:osd_ref_del()) lustre-MDT0003: nlink == 0 on [0x2c0000405:0xc06:0x0], maybe an upgraded file? (LU-3915) | Link to test |