| Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
| Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
| Limit to a test: (Copy from below "Failing text"): | |
| Delete these reports as invalid (real bug in review or some such) | |
| Bug or comment: | |
| Extra info: |
| Failing Test | Full Crash | Messages before crash | Comment |
|---|---|---|---|
| conf-sanity testing /usr/lib64/lustre/tests/disk2_10-ldiskfs.tar.bz2 upgrade with writeconf | LustreError: 294484:0:(lod_dev.c:2171:lod_device_free()) ASSERTION( atomic_read(&lu->ld_ref) == 0 ) failed: lu is ffff8b7c7755c000 LustreError: 294484:0:(lod_dev.c:2171:lod_device_free()) LBUG CPU: 1 PID: 294484 Comm: umount Kdump: loaded Tainted: G OE ------- --- 5.14.0-362.18.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] lod_device_free+0x25b/0x260 [lod] obd_cleanup+0xed/0x1c0 [obdclass] class_free_dev+0x223/0x470 [obdclass] class_detach+0x28f/0x2e0 [obdclass] class_process_config+0x94b/0x1bb0 [obdclass] class_manual_cleanup+0x453/0x7a0 [obdclass] lod_obd_disconnect+0xff/0x1d0 [lod] obd_disconnect.isra.0+0x15b/0x300 [mdd] mdd_process_config+0x183/0x450 [mdd] mdt_stack_fini+0x498/0x700 [mdt] mdt_fini+0x2ee/0x570 [mdt] mdt_device_fini+0x2b/0xc0 [mdt] obd_precleanup+0x1e4/0x220 [obdclass] class_cleanup+0x2d5/0x600 [obdclass] class_process_config+0x10a9/0x1bb0 [obdclass] class_manual_cleanup+0x439/0x7a0 [obdclass] server_put_super+0x7ee/0xa40 [ptlrpc] generic_shutdown_super+0x74/0x120 kill_anon_super+0x14/0x30 deactivate_locked_super+0x31/0xa0 cleanup_mnt+0x100/0x160 task_work_run+0x5c/0x90 exit_to_user_mode_loop+0x122/0x130 exit_to_user_mode_prepare+0xb6/0x100 syscall_exit_to_user_mode+0x12/0x40 do_syscall_64+0x69/0x90 ? exc_page_fault+0x62/0x150 entry_SYSCALL_64_after_hwframe+0x72/0xdc RIP: 0033:0x7f184e34e60b | Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids Lustre: DEBUG MARKER: mkdir -p /tmp/t32/mnt/mdt /tmp/t32/mnt/mdt1 /tmp/t32/mnt/ost /tmp/t32/mnt/ost1 Lustre: DEBUG MARKER: tar xjvf /usr/lib64/lustre/tests/disk2_10-ldiskfs.tar.bz2 -S -C /tmp/t32 Lustre: DEBUG MARKER: cat /tmp/t32/commit Lustre: DEBUG MARKER: cat /tmp/t32/kernel Lustre: DEBUG MARKER: cat /tmp/t32/arch Lustre: DEBUG MARKER: cat /tmp/t32/bspace Lustre: DEBUG MARKER: cat /tmp/t32/ispace Lustre: DEBUG MARKER: test -f /tmp/t32/blimit Lustre: DEBUG MARKER: cat /tmp/t32/blimit Lustre: DEBUG MARKER: test -f /tmp/t32/ilimit Lustre: DEBUG MARKER: cat /tmp/t32/ilimit Lustre: DEBUG MARKER: test -f /tmp/t32/mdt2 Lustre: DEBUG MARKER: test -f /tmp/t32/ost2 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 Lustre: DEBUG MARKER: tunefs.lustre --dryrun /tmp/t32/mdt Lustre: DEBUG MARKER: tunefs.lustre --dryrun /tmp/t32/mdt2 Lustre: DEBUG MARKER: tunefs.lustre --quota /tmp/t32/mdt loop0: detected capacity change from 0 to 400000 Lustre: DEBUG MARKER: tunefs.lustre --quota /tmp/t32/mdt2 loop0: detected capacity change from 0 to 400000 Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-71vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lctl device_list Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-71vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lctl device_list Lustre: DEBUG MARKER: trevis-71vm6.trevis.whamcloud.com: executing /usr/sbin/lctl device_list Lustre: DEBUG MARKER: trevis-71vm6.trevis.whamcloud.com: executing /usr/sbin/lctl device_list Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-71vm6.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-71vm6.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: trevis-71vm6.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: trevis-71vm6.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: mount -t lustre -o loop,writeconf /tmp/t32/mdt /tmp/t32/mnt/mdt loop0: detected capacity change from 0 to 400000 LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: t32fs-MDT0000-osd: enabled 'large_dir' feature on device /dev/loop0 Lustre: Skipped 1 previous similar message Lustre: 290817:0:(scrub.c:190:scrub_file_load()) t32fs-MDT0000: reset scrub OI count for format change (LU-16655) Lustre: 290817:0:(scrub.c:190:scrub_file_load()) Skipped 1 previous similar message Lustre: MGS: Logs for fs t32fs were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 Lustre: Setting parameter t32fs-MDT0000-mdtlov.lov.stripesize in log t32fs-MDT0000 Lustre: t32fs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: mount -t lustre -o mgsnode=10.240.42.1@tcp,loop,writeconf /tmp/t32/mdt2 /tmp/t32/mnt/mdt1 loop1: detected capacity change from 0 to 400000 LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: MGS: Regenerating t32fs-MDT0001 log by user request: rc = 0 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n mdt.t32fs*.enable_remote_dir=1 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n mdt.t32fs-MDT0000.uuid Lustre: DEBUG MARKER: tunefs.lustre --dryrun /tmp/t32/ost Lustre: DEBUG MARKER: tunefs.lustre --dryrun /tmp/t32/ost2 Lustre: DEBUG MARKER: tunefs.lustre --quota /tmp/t32/ost loop2: detected capacity change from 0 to 400000 Lustre: DEBUG MARKER: tunefs.lustre --quota /tmp/t32/ost2 loop2: detected capacity change from 0 to 400000 Lustre: DEBUG MARKER: mount -t lustre -onomgs -oloop,mgsnode=10.240.42.1@tcp,writeconf /tmp/t32/ost /tmp/t32/mnt/ost loop2: detected capacity change from 0 to 400000 LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: mount -t lustre -onomgs -oloop,mgsnode=10.240.42.1@tcp,writeconf /tmp/t32/ost2 /tmp/t32/mnt/ost1 loop3: detected capacity change from 0 to 400000 LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n obdfilter.t32fs-OST0000.uuid Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n obdfilter.t32fs-OST0001.uuid Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-OST0000.failover.node=10.240.42.1@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.failover.node=10.240.42.1@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-OST0000.osc.max_dirty_mb=15 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.mdc.max_rpcs_in_flight=9 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.lov.stripesize=4M Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.mdd.atime_diff=70 Lustre: DEBUG MARKER: /usr/sbin/lctl pool_new t32fs.interop Lustre: t32fs-OST0000: new connection from t32fs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:68 to 0x280000400:129) Lustre: t32fs-OST0001: new connection from t32fs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:67 to 0x2c0000400:129) Lustre: t32fs-OST0000: new connection from t32fs-MDT0000-mdtlov (cleaning up unused objects from 0x0:11 to 0x0:97) Lustre: t32fs-OST0001: new connection from t32fs-MDT0000-mdtlov (cleaning up unused objects from 0x0:10 to 0x0:97) Lustre: t32fs-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 Lustre: t32fs-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0001.failover.node=10.240.42.1@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0001.mdc.max_rpcs_in_flight=9 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0001.lov.stripesize=4M Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs.quota.mdt='ug' Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs.quota.ost='ug' Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled LustreError: 293859:0:(qsd_reint.c:633:qqi_reint_delayed()) t32fs-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed. LustreError: 293859:0:(qsd_reint.c:633:qqi_reint_delayed()) Skipped 1 previous similar message Lustre: DEBUG MARKER: test -f /tmp/t32/list Lustre: DEBUG MARKER: test -f /tmp/t32/list2 Lustre: DEBUG MARKER: cat /tmp/t32/list2 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n osd*.*.force_sync=1 Lustre: DEBUG MARKER: test -f /tmp/t32/sha1sums Lustre: DEBUG MARKER: cat /tmp/t32/sha1sums Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.mdc.max_rpcs_in_flight='14' Lustre: DEBUG MARKER: umount -d /tmp/t32/mnt/mdt Lustre: Failing over t32fs-MDT0000 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@00000000457ad1b8[0x4, 1, [0x200000006:0x20000:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....mdt@00000000536dde58mdt-object@00000000457ad1b8( , writecount=0) LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....mdd@00000000240cc35bmdd-object@00000000240cc35b(open_count=0, valid=0, cltime=0ns, flags=0) LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....lod@000000008001f965lod-object@000000008001f965 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@0000000095f86524osd-ldiskfs-object@0000000095f86524(i:000000009287f258:95/3644679363)[lfix] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@00000000457ad1b8 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@00000000f4ffbb55[0x4, 1, [0x200000003:0x2:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....local_storage@000000005ac13a3a LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000d756047bosd-ldiskfs-object@00000000d756047b(i:000000005bcea782:81/3644679279)[lfix] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@00000000f4ffbb55 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@00000000030cff12[0x4, 1, [0x200000003:0x0:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....local_storage@000000005b731364 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000c13be813osd-ldiskfs-object@00000000c13be813(i:000000002804053f:80/3644679278)[plain] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@00000000030cff12 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@00000000384bf451[0x4, 1, [0x200000003:0x3:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000aa614a37 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000037d2cd9osd-ldiskfs-object@00000000037d2cd9(i:00000000507eaaad:26696/3644679280)[plain] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@00000000384bf451 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@00000000fcc9737f[0x5, 1, [0x200000001:0x1017:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000c5bea707 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000123d4b96osd-ldiskfs-object@00000000123d4b96(i:0000000064de9295:12/203470252)[plain] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@00000000fcc9737f LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@00000000e5f0ee8d[0x4, 1, [0x1:0x0:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....local_storage@000000001906373c LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000008c8be405osd-ldiskfs-object@000000008c8be405(i:000000008144a638:82/3644679314)[plain] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@00000000e5f0ee8d LustreError: 294484:0:(lu_object.c:998:lu_site_print()) header@000000001bab998d[0x4, 1, [0xa:0x0:0x0] hash exist]{ LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000698e50c3 LustreError: 294484:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000004b07f48aosd-ldiskfs-object@000000004b07f48a(i:0000000053e46656:83/3644679348)[plain] LustreError: 294484:0:(lu_object.c:998:lu_site_print()) } header@000000001bab998d | Link to test |
| sanity-quota test 7c: Quota reintegration (restart mds during reintegration) | LustreError: 606486:0:(lod_dev.c:2171:lod_device_free()) ASSERTION( atomic_read(&lu->ld_ref) == 0 ) failed: lu is ffff90c5f7b92000 LustreError: 606486:0:(lod_dev.c:2171:lod_device_free()) LBUG CPU: 1 PID: 606486 Comm: umount Kdump: loaded Tainted: G OE ------- --- 5.14.0-362.18.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] lod_device_free+0x25b/0x260 [lod] obd_cleanup+0xed/0x1c0 [obdclass] class_free_dev+0x223/0x470 [obdclass] class_detach+0x28f/0x2e0 [obdclass] class_process_config+0x94b/0x1bb0 [obdclass] ? __kmalloc+0x19b/0x370 class_manual_cleanup+0x453/0x7a0 [obdclass] lod_obd_disconnect+0xff/0x1d0 [lod] obd_disconnect.isra.0+0x15b/0x300 [mdd] mdd_process_config+0x183/0x450 [mdd] mdt_stack_fini+0x498/0x700 [mdt] mdt_fini+0x2ee/0x570 [mdt] mdt_device_fini+0x2b/0xc0 [mdt] obd_precleanup+0x1e4/0x220 [obdclass] class_cleanup+0x2d5/0x600 [obdclass] class_process_config+0x10a9/0x1bb0 [obdclass] ? __kmalloc+0x19b/0x370 class_manual_cleanup+0x439/0x7a0 [obdclass] server_put_super+0x7ee/0xa40 [ptlrpc] generic_shutdown_super+0x74/0x120 kill_anon_super+0x14/0x30 deactivate_locked_super+0x31/0xa0 cleanup_mnt+0x100/0x160 task_work_run+0x5c/0x90 exit_to_user_mode_loop+0x122/0x130 exit_to_user_mode_prepare+0xb6/0x100 syscall_exit_to_user_mode+0x12/0x40 do_syscall_64+0x69/0x90 ? syscall_exit_work+0x103/0x130 ? syscall_exit_to_user_mode+0x22/0x40 ? do_syscall_64+0x69/0x90 ? syscall_exit_to_user_mode+0x22/0x40 ? do_syscall_64+0x69/0x90 ? do_syscall_64+0x69/0x90 ? syscall_exit_to_user_mode+0x22/0x40 ? do_syscall_64+0x69/0x90 ? exc_page_fault+0x62/0x150 entry_SYSCALL_64_after_hwframe+0x72/0xdc RIP: 0033:0x7f76f1f4e60b | Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.*MDT*.sync_* Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osp.*.destroys_in_flight Lustre: DEBUG MARKER: lctl set_param fail_val=0 fail_loc=0 Lustre: DEBUG MARKER: lctl set_param -n os[cd]*.*MDT*.force_sync=1 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param lustre.quota.ost=none Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param lustre.quota.ost=ugp LustreError: 606335:0:(qsd_reint.c:633:qqi_reint_delayed()) lustre-MDT0002: Delaying reintegration for qtype:0 until pending updates are flushed. LustreError: 606335:0:(qsd_reint.c:633:qqi_reint_delayed()) Skipped 11 previous similar messages Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds1' ' /proc/mounts || true Lustre: DEBUG MARKER: umount -d /mnt/lustre-mds1 Lustre: Failing over lustre-MDT0000 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@00000000ea0285e0[0x4, 1, [0x200000006:0x10000:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....mdt@00000000ddf2dbeemdt-object@00000000ea0285e0( , writecount=0) LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....mdd@00000000b4b26a85mdd-object@00000000b4b26a85(open_count=0, valid=0, cltime=0ns, flags=0) LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....lod@00000000bfe6e4bblod-object@00000000bfe6e4bb LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000fe42e802osd-ldiskfs-object@00000000fe42e802(i:000000005de75228:190/640227297)[lfix] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@00000000ea0285e0 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@00000000a3ed426e[0x4, 1, [0x200000003:0x0:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....local_storage@000000002debca29 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000e80b6fb1osd-ldiskfs-object@00000000e80b6fb1(i:000000002d0af79a:112/3301758490)[plain] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@00000000a3ed426e LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@0000000093272eb2[0x4, 1, [0x1:0x0:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000b9ff0389 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000006a23858cosd-ldiskfs-object@000000006a23858c(i:00000000dfaf1e64:147/882313078)[plain] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@0000000093272eb2 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@00000000bccf0d94[0x4, 1, [0x200000003:0x2:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000415bad07 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000008131e643osd-ldiskfs-object@000000008131e643(i:00000000d647915c:113/3034145679)[lfix] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@00000000bccf0d94 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@00000000925d1977[0x4, 1, [0x200000003:0x3:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000235d0a67 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000002abecb3aosd-ldiskfs-object@000000002abecb3a(i:00000000e0ef4676:31715/509602748)[plain] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@00000000925d1977 LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@0000000039714b3b[0x5, 1, [0x200000001:0x1017:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....local_storage@0000000047e6e11a LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000904604eeosd-ldiskfs-object@00000000904604ee(i:00000000f7514a49:31713/272312328)[plain] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@0000000039714b3b LustreError: 606486:0:(lu_object.c:998:lu_site_print()) header@0000000059a2371a[0x4, 1, [0xa:0x0:0x0] hash exist]{ LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000afed122d LustreError: 606486:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000005169020aosd-ldiskfs-object@000000005169020a(i:00000000cdb2e044:181/1886485665)[plain] LustreError: 606486:0:(lu_object.c:998:lu_site_print()) } header@0000000059a2371a | Link to test |
| conf-sanity testing /usr/lib64/lustre/tests/disk2_7-ldiskfs.tar.bz2 upgrade with DNE | LustreError: 332528:0:(lod_dev.c:2171:lod_device_free()) ASSERTION( atomic_read(&lu->ld_ref) == 0 ) failed: lu is ffff90d7baac1000 LustreError: 332528:0:(lod_dev.c:2171:lod_device_free()) LBUG CPU: 1 PID: 332528 Comm: umount Kdump: loaded Tainted: G OE ------- --- 5.14.0-362.18.1_lustre.el9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl+0x34/0x48 lbug_with_loc.cold+0x5/0x43 [libcfs] lod_device_free+0x256/0x260 [lod] obd_cleanup+0xed/0x1c0 [obdclass] class_free_dev+0x142/0x2a0 [obdclass] class_detach+0x28f/0x2e0 [obdclass] class_process_config+0x94b/0x1bb0 [obdclass] class_manual_cleanup+0x450/0x790 [obdclass] lod_obd_disconnect+0xff/0x1d0 [lod] obd_disconnect.isra.0+0x15b/0x300 [mdd] mdd_process_config+0x183/0x450 [mdd] mdt_stack_fini+0x495/0x6f0 [mdt] mdt_fini+0x2ee/0x570 [mdt] mdt_device_fini+0x2b/0xc0 [mdt] obd_precleanup+0x1e4/0x220 [obdclass] class_cleanup+0x2d5/0x600 [obdclass] class_process_config+0x10a9/0x1bb0 [obdclass] class_manual_cleanup+0x436/0x790 [obdclass] server_put_super+0x7dd/0xa30 [ptlrpc] generic_shutdown_super+0x74/0x120 kill_anon_super+0x14/0x30 deactivate_locked_super+0x31/0xa0 cleanup_mnt+0x100/0x160 task_work_run+0x5c/0x90 exit_to_user_mode_loop+0x122/0x130 exit_to_user_mode_prepare+0xb6/0x100 syscall_exit_to_user_mode+0x12/0x40 do_syscall_64+0x69/0x90 ? do_syscall_64+0x69/0x90 ? syscall_exit_to_user_mode+0x22/0x40 ? do_syscall_64+0x69/0x90 ? syscall_exit_to_user_mode+0x22/0x40 ? do_syscall_64+0x69/0x90 ? do_syscall_64+0x69/0x90 ? exc_page_fault+0x62/0x150 entry_SYSCALL_64_after_hwframe+0x72/0xdc RIP: 0033:0x7f986db4e60b | Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids Lustre: DEBUG MARKER: mkdir -p /tmp/t32/mnt/mdt /tmp/t32/mnt/mdt1 /tmp/t32/mnt/ost /tmp/t32/mnt/ost1 Lustre: DEBUG MARKER: tar xjvf /usr/lib64/lustre/tests/disk2_7-ldiskfs.tar.bz2 -S -C /tmp/t32 Lustre: DEBUG MARKER: cat /tmp/t32/commit Lustre: DEBUG MARKER: cat /tmp/t32/kernel Lustre: DEBUG MARKER: cat /tmp/t32/arch Lustre: DEBUG MARKER: cat /tmp/t32/bspace Lustre: DEBUG MARKER: cat /tmp/t32/ispace Lustre: DEBUG MARKER: test -f /tmp/t32/blimit Lustre: DEBUG MARKER: test -f /tmp/t32/ilimit Lustre: DEBUG MARKER: test -f /tmp/t32/mdt2 Lustre: DEBUG MARKER: test -f /tmp/t32/ost2 Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 Lustre: DEBUG MARKER: tunefs.lustre --dryrun /tmp/t32/mdt Lustre: DEBUG MARKER: tunefs.lustre --quota /tmp/t32/mdt loop0: detected capacity change from 0 to 400000 Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-74vm10.trevis.whamcloud.com: executing \/usr\/sbin\/lctl device_list Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-74vm10.trevis.whamcloud.com: executing \/usr\/sbin\/lctl device_list Lustre: DEBUG MARKER: trevis-74vm10.trevis.whamcloud.com: executing /usr/sbin/lctl device_list Lustre: DEBUG MARKER: trevis-74vm10.trevis.whamcloud.com: executing /usr/sbin/lctl device_list Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-74vm10.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-74vm10.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: trevis-74vm10.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: trevis-74vm10.trevis.whamcloud.com: executing losetup -a Lustre: DEBUG MARKER: mount -t lustre -o loop,writeconf /tmp/t32/mdt /tmp/t32/mnt/mdt loop0: detected capacity change from 0 to 400000 LDISKFS-fs (loop0): recovery complete LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: MGS: Logs for fs t32fs were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 Lustre: t32fs-MDT0000: in recovery but waiting for the first client to connect Lustre: DEBUG MARKER: grep -c /mnt/lustre-mds1' ' /proc/mounts || true Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && Lustre: DEBUG MARKER: modprobe dm-flakey; Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=trevis-74vm10@tcp --fsname=t32fs --mdt --index=1 --param=sys.timeout=20 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --mkfsoptions="-b 4096 -E lazy_itable_init" --reformat --mkfsoption LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Quota mode: journalled. LDISKFS-fs (dm-2): unmounting filesystem. Lustre: DEBUG MARKER: tunefs.lustre --dryrun /dev/vg_Role_MDS/scratch1 Lustre: DEBUG MARKER: mount -t lustre -o loop,writeconf /dev/vg_Role_MDS/scratch1 /tmp/t32/mnt/mdt1 loop1: detected capacity change from 0 to 212992 LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Quota mode: journalled. LDISKFS-fs (loop1): unmounting filesystem. LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: MGS: Regenerating t32fs-MDT0001 log by user request: rc = 0 Lustre: Skipped 2 previous similar messages Lustre: srv-t32fs-MDT0001: No data found on store. Initialize space. Lustre: t32fs-MDT0001: new disk, initializing Lustre: t32fs-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects Lustre: ctl-t32fs-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt Lustre: Skipped 1 previous similar message Lustre: cli-ctl-t32fs-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n mdt.t32fs*.enable_remote_dir=1 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n mdt.t32fs-MDT0000.uuid Lustre: DEBUG MARKER: tunefs.lustre --dryrun /tmp/t32/ost Lustre: DEBUG MARKER: tunefs.lustre --quota /tmp/t32/ost loop2: detected capacity change from 0 to 400000 Lustre: DEBUG MARKER: mount -t lustre -onomgs -oloop,mgsnode=10.240.42.65@tcp,writeconf /tmp/t32/ost /tmp/t32/mnt/ost loop2: detected capacity change from 0 to 400000 LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Quota mode: journalled. Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n obdfilter.t32fs-OST0000.uuid Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-OST0000.failover.node=10.240.42.65@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.failover.node=10.240.42.65@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-OST0000.osc.max_dirty_mb=15 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.mdc.max_rpcs_in_flight=9 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.lov.stripesize=4M Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.mdd.atime_diff=70 Lustre: DEBUG MARKER: /usr/sbin/lctl pool_new t32fs.interop Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0001.failover.node=10.240.42.65@tcp Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0001.mdc.max_rpcs_in_flight=9 Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0001.lov.stripesize=4M Lustre: t32fs-MDT0000: Denying connection for new client b77b56c8-9c6e-4db2-b51c-b96212f98299 (at 10.240.43.203@tcp), waiting for 2 known clients (1 recovered, 0 in progress, and 0 evicted) to recover in 0:46 Lustre: Skipped 1 previous similar message Lustre: t32fs-MDT0000: recovery is timed out, evict stale exports Lustre: 328019:0:(genops.c:1514:class_disconnect_stale_exports()) t32fs-MDT0000: disconnect stale client b404de28-5b38-b546-39bc-2ef0be4d230f@<unknown> Lustre: t32fs-MDT0000: disconnecting 1 stale clients Lustre: t32fs-MDT0000: Recovery over after 1:00, of 2 clients 1 recovered and 1 was evicted. Lustre: t32fs-MDT0000-osp-MDT0001: Connection restored to 10.240.42.65@tcp (at 0@lo) Lustre: t32fs-OST0000: new connection from t32fs-MDT0000-mdtlov (cleaning up unused objects from 0x0:67 to 0x0:129) Lustre: t32fs-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 LustreError: 327987:0:(mdt_identity.c:110:mdt_identity_do_upcall()) t32fs-MDT0000: error invoking upcall /home/zam/work/git/lustre-wc-rel/lustre/tests/../utils/l_getidentity t32fs-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/t32fs-MDT0000/identity_upcall, time 43424us: rc = -2 LustreError: 327987:0:(mdt_identity.c:110:mdt_identity_do_upcall()) Skipped 58 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs.quota.mdt='ug' Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-MDT0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs.quota.ost='ug' Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osd-ldiskfs.t32fs-OST0000.quota_slave.enabled LustreError: 331385:0:(qsd_reint.c:633:qqi_reint_delayed()) t32fs-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed. LustreError: 331385:0:(qsd_reint.c:633:qqi_reint_delayed()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: test -f /tmp/t32/list Lustre: DEBUG MARKER: cat /tmp/t32/list Lustre: 327983:0:(osd_internal.h:1331:osd_trans_exec_op()) t32fs-MDT0001: opcode 8: before 255 < left 302, rollback = 9 Lustre: 327983:0:(osd_internal.h:1331:osd_trans_exec_op()) Skipped 35 previous similar messages Lustre: 327983:0:(osd_handler.c:1969:osd_trans_dump_creds()) create: 6/24/5, destroy: 0/0/0 Lustre: 327983:0:(osd_handler.c:1969:osd_trans_dump_creds()) Skipped 35 previous similar messages Lustre: 327983:0:(osd_handler.c:1976:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 5/389/0 Lustre: 327983:0:(osd_handler.c:1976:osd_trans_dump_creds()) Skipped 35 previous similar messages Lustre: 327983:0:(osd_handler.c:1983:osd_trans_dump_creds()) write: 11/192/0, punch: 0/0/0, quota 1/3/0 Lustre: 327983:0:(osd_handler.c:1983:osd_trans_dump_creds()) Skipped 35 previous similar messages Lustre: 327983:0:(osd_handler.c:1993:osd_trans_dump_creds()) insert: 16/302/0, delete: 0/0/0 Lustre: 327983:0:(osd_handler.c:1993:osd_trans_dump_creds()) Skipped 35 previous similar messages Lustre: 327983:0:(osd_handler.c:2000:osd_trans_dump_creds()) ref_add: 4/4/0, ref_del: 0/0/0 Lustre: 327983:0:(osd_handler.c:2000:osd_trans_dump_creds()) Skipped 35 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n osd*.*.force_sync=1 Lustre: DEBUG MARKER: test -f /tmp/t32/sha1sums Lustre: DEBUG MARKER: cat /tmp/t32/sha1sums Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n mdt.t32fs*.enable_remote_dir=1 LustreError: 330713:0:(mdt_identity.c:110:mdt_identity_do_upcall()) t32fs-MDT0000: error invoking upcall /home/zam/work/git/lustre-wc-rel/lustre/tests/../utils/l_getidentity t32fs-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/t32fs-MDT0000/identity_upcall, time 92us: rc = -2 LustreError: 330713:0:(mdt_identity.c:110:mdt_identity_do_upcall()) Skipped 457 previous similar messages Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param t32fs-MDT0000.mdc.max_rpcs_in_flight='14' Lustre: DEBUG MARKER: umount -d /tmp/t32/mnt/mdt Lustre: Failing over t32fs-MDT0000 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@00000000fe3b9053[0x4, 1, [0x200000003:0x0:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....local_storage@0000000090cce275 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000d22c7894osd-ldiskfs-object@00000000d22c7894(i:00000000203e3c8b:80/1606589865)[plain] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@00000000fe3b9053 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@0000000080075579[0x4, 1, [0x200000003:0x2:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000870bfbf0 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000006f1987c4osd-ldiskfs-object@000000006f1987c4(i:00000000f375988c:25026/1606589866)[plain] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@0000000080075579 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@0000000045da19f9[0x4, 1, [0x200000006:0x1020000:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....mdt@000000000c5dd5c1mdt-object@0000000045da19f9( , writecount=0) LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....mdd@000000006e323345mdd-object@000000006e323345(open_count=0, valid=0, cltime=0ns, flags=0) LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....lod@00000000a518cdf8lod-object@00000000a518cdf8 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000789273d5osd-ldiskfs-object@00000000789273d5(i:0000000005495b54:94/1606589949)[lfix] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@0000000045da19f9 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@00000000f57fa039[0x5, 1, [0x200000001:0x1017:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....local_storage@000000002fb2ad76 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000d522a1ebosd-ldiskfs-object@00000000d522a1eb(i:00000000dea70551:12/3659111522)[plain] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@00000000f57fa039 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@0000000074210ed8[0x4, 1, [0xa:0x0:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....local_storage@00000000f95c2dd3 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000575eac38osd-ldiskfs-object@00000000575eac38(i:0000000070150365:82/1606589934)[plain] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@0000000074210ed8 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@0000000046634c37[0x4, 1, [0x200000003:0x1f:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....local_storage@000000007456829f LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@00000000d000ef43osd-ldiskfs-object@00000000d000ef43(i:00000000356915bf:150/1935397403)[lfix] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@0000000046634c37 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) header@00000000732a79b8[0x4, 1, [0x1:0x0:0x0] hash exist]{ LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....local_storage@0000000057548fe8 LustreError: 332528:0:(lu_object.c:998:lu_site_print()) ....osd-ldiskfs@000000004675151eosd-ldiskfs-object@000000004675151e(i:00000000f951b39a:81/1606589900)[plain] LustreError: 332528:0:(lu_object.c:998:lu_site_print()) } header@00000000732a79b8 | Link to test |