Editing crashreport #73737

ReasonCrashing FunctionWhere to cut BacktraceReports Count
general protection fault, lod_qos_parse_configlod_prepare_create
lod_declare_striped_create
lod_declare_xattr_set
dt_declare_xattr_set
mdd_create_data
mdt_create_data
mdt_mfd_open
mdt_finish_open
mdt_open_by_fid_lock
mdt_reint_open
mdt_reint_rec
mdt_reint_internal
mdt_intent_open
mdt_intent_opc
mdt_intent_policy
ldlm_lock_enqueue
ldlm_handle_enqueue
tgt_enqueue
tgt_handle_request0
tgt_request_handle
ptlrpc_server_handle_request
ptlrpc_main
kthread
ret_from_fork
10

Added fields:

Match messages in logs
(every line would be required to be present in log output
Copy from "Messages before crash" column below):
Match messages in full crash
(every line would be required to be present in crash log output
Copy from "Full Crash" column below):
Limit to a test:
(Copy from below "Failing text"):
Delete these reports as invalid (real bug in review or some such)
Bug or comment:
Extra info:

Failures list (last 100):

Failing TestFull CrashMessages before crashComment
sanity-sec test 27d: test multiple filesets mounting rules (rw)
general protection fault, probably for non-canonical address 0x61a75877a6569328: 0000 [#1] PREEMPT SMP PTI
CPU: 0 PID: 288866 Comm: mdt00_002 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffbbe340927768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9304f44c78c0 RCX: ffff9304e6d13eb4
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0e3fd40
RBP: ffffbbe3409277e8 R08: ffffbbe3409276c8 R09: 0000000000000000
R10: ffff9304f2078a80 R11: 0000000000000000 R12: 0000000000000000
R13: ffff9304e6d13e90 R14: ffff9304c2996000 R15: 61a75877a6569300
FS: 0000000000000000(0000) GS:ffff93057fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fee477cf4e0 CR3: 00000000036ac003 CR4: 00000000001706f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? _raw_spin_unlock+0xa/0x30
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2d0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) tls ec(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common i2c_piix4 virtio_balloon pcspkr joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix libata crct10dif_pclmul crc32_pclmul crc32c_intel virtio_net ghash_clmulni_intel virtio_blk net_failover failover serio_raw [last unloaded: libcfs(OE)]
Lustre: DEBUG MARKER: /usr/sbin/lctl set_param mdt.*.identity_upcall=NONE
Lustre: 317200:0:(mdt_lproc.c:310:identity_upcall_store()) lustre-MDT0000: disable "identity_upcall" with ACL enabled maybe cause unexpected "EACCESS"
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add c0
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_range --name c0 --range 10.240.40.171@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype uid --idmap 60003:60000
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype gid --idmap 60003:60000
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype uid --idmap 60004:60002
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype gid --idmap 60004:60002
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add c1
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_range --name c1 --range 10.240.40.172@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype uid --idmap 60005:60001
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype gid --idmap 60005:60001
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype uid --idmap 60006:60002
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype gid --idmap 60006:60002
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_activate 1
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_modify --name default --property admin --value 1
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param nodemap.default.admin_nodemap
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_modify --name default --property trusted --value 1
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param nodemap.default.trusted_nodemap
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_modify --name c0 --property admin --value 1
Lustre: DEBUG MARKER: /usr/sbin/lctl nodemap_modify --name c0 --property trusted --value 1
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n nodemap.active
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param nodemap.c0.trusted_nodemap
Autotest: Test running for 60 minutes (lustre-reviews_review-dne-part-2_115677.43)
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
Link to test
sanity-lfsck test 45: LFSCK should fix UID/GID/PROJID of OST object
general protection fault, probably for non-canonical address 0xf16f5cd8b4172428: 0000 [#1] PREEMPT SMP PTI
CPU: 0 PID: 207226 Comm: mdt00_000 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffff9cb38184f768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff91126e130b60 RCX: ffff911250e6df0c
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0fd9d40
RBP: ffff9cb38184f7e8 R08: ffff9cb38184f6c8 R09: 0000000000000000
R10: ffff911245fe7cc0 R11: 0000000000000000 R12: 0000000000000000
R13: ffff911250e6dee8 R14: ffff9112766f7000 R15: f16f5cd8b4172400
FS: 0000000000000000(0000) GS:ffff9112ffc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fda6d9c6584 CR3: 0000000003160003 CR4: 00000000001706f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: dm_flakey tls osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common virtio_balloon pcspkr i2c_piix4 joydev fuse drm ext4 mbcache jbd2 ata_generic ata_piix libata crct10dif_pclmul crc32_pclmul crc32c_intel virtio_net virtio_blk ghash_clmulni_intel net_failover failover serio_raw [last unloaded: dm_flakey]
Lustre: lustre-MDT0000: Recovery over after 0:05, of 5 clients 5 recovered and 0 were evicted.
Lustre: Skipped 1 previous similar message
Link to test
racer test 1: racer on clients: onyx-102vm4.onyx.whamcloud.com,onyx-102vm5 DURATION=300
general protection fault, probably for non-canonical address 0x380664007886f28: 0000 [#1] PREEMPT SMP PTI
CPU: 1 PID: 35712 Comm: mdt00_018 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffa87d4ad1b768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff8e2902992620 RCX: ffff8e28c6c4b804
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0f98d40
RBP: ffffa87d4ad1b7e8 R08: ffffa87d4ad1b6c8 R09: 0000000000000000
R10: ffff8e290e84f600 R11: 0000000000000000 R12: 0000000000000000
R13: ffff8e28c6c4b7e0 R14: ffff8e28ed3bf000 R15: 0380664007886f00
FS: 0000000000000000(0000) GS:ffff8e297fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa752e75000 CR3: 00000000077b6004 CR4: 00000000001706f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: tls rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd nfs_acl osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common i2c_piix4 virtio_balloon pcspkr joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix crct10dif_pclmul crc32_pclmul libata crc32c_intel ghash_clmulni_intel virtio_net virtio_blk net_failover failover serio_raw
Lustre: 21656:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8e2901904d00 x1840145208082048/t4295033075(0) o101->fa4e13a5-5de2-4e7f-a0d0-540c0d9800bd@10.240.28.108@tcp:75/0 lens 376/912 e 0 to 0 dl 1754900180 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
Lustre: 22159:0:(mdd_dir.c:4836:mdd_migrate_object()) lustre-MDD0002: [0x280000406:0x1:0x0]/18 is open, migrate only dentry
LustreError: 35721:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000408:0x2:0x0]/19 failed: rc = -2
LustreError: 21616:0:(mdd_object.c:3899:mdd_close()) lustre-MDD0000: failed to get lu_attr of [0x20000040a:0x18:0x0]: rc = -2
LustreError: 35724:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0000: migrate [0x200000408:0x2:0x0]/13 failed: rc = -2
Lustre: 22159:0:(mdd_dir.c:4836:mdd_migrate_object()) lustre-MDD0000: [0x200000408:0x2:0x0]/4 is open, migrate only dentry
Lustre: 22159:0:(mdd_dir.c:4836:mdd_migrate_object()) Skipped 1 previous similar message
Lustre: 35721:0:(mdd_dir.c:4836:mdd_migrate_object()) lustre-MDD0000: [0x200000408:0x2:0x0]/11 is open, migrate only dentry
Lustre: 35721:0:(mdd_dir.c:4836:mdd_migrate_object()) Skipped 2 previous similar messages
Lustre: 22159:0:(mdd_dir.c:4836:mdd_migrate_object()) lustre-MDD0000: [0x200000408:0x2:0x0]/3 is open, migrate only dentry
Lustre: 22159:0:(mdd_dir.c:4836:mdd_migrate_object()) Skipped 1 previous similar message
LustreError: 10278:0:(mdd_object.c:3899:mdd_close()) lustre-MDD0002: failed to get lu_attr of [0x280000406:0x1f:0x0]: rc = -2
Lustre: 13102:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8e2902c31a00 x1840145211568000/t4295033885(0) o101->fa4e13a5-5de2-4e7f-a0d0-540c0d9800bd@10.240.28.108@tcp:83/0 lens 376/840 e 0 to 0 dl 1754900188 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0
LustreError: 35736:0:(mdt_reint.c:2562:mdt_reint_migrate()) lustre-MDT0002: migrate [0x200000408:0xed:0x0]/20 failed: rc = -2
Link to test
sanity-benchmark test fsx: fsx
general protection fault, probably for non-canonical address 0xfc3e3baa4a9c1e28: 0000 [#1] PREEMPT SMP PTI
CPU: 0 PID: 34691 Comm: mdt00_004 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffb22604937768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9c5ac859bee0 RCX: ffff9c5af111a5bc
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0f64d40
RBP: ffffb226049377e8 R08: ffffb226049376c8 R09: 0000000000000000
R10: ffff9c5ac8def0c0 R11: 0000000000000000 R12: 0000000000000000
R13: ffff9c5af111a598 R14: ffff9c5ac7685000 R15: fc3e3baa4a9c1e00
FS: 0000000000000000(0000) GS:ffff9c5b7fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4f244084e0 CR3: 0000000003862001 CR4: 00000000001706f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: tls dm_flakey osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common i2c_piix4 pcspkr virtio_balloon joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix crct10dif_pclmul crc32_pclmul crc32c_intel virtio_net libata ghash_clmulni_intel virtio_blk net_failover failover serio_raw [last unloaded: dm_flakey]
Link to test
parallel-scale test cascading_rw: cascading_rw
general protection fault, probably for non-canonical address 0xf13e4d8c0d8fd628: 0000 [#1] PREEMPT SMP PTI
CPU: 1 PID: 10274 Comm: mdt00_002 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffff9f0500903768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff8d09c6f739a0 RCX: ffff8d097462c204
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0fa1d40
RBP: ffff9f05009037e8 R08: ffff9f05009036c8 R09: 0000000000000000
R10: ffff8d09c7128600 R11: 0000000000000000 R12: 0000000000000000
R13: ffff8d097462c1e0 R14: ffff8d0967c4f000 R15: f13e4d8c0d8fd600
FS: 0000000000000000(0000) GS:ffff8d09ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2b949244e0 CR3: 000000000347a004 CR4: 00000000001706f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: tls osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common pcspkr virtio_balloon i2c_piix4 joydev fuse drm ext4 mbcache jbd2 crct10dif_pclmul crc32_pclmul ata_generic crc32c_intel virtio_net ata_piix net_failover libata ghash_clmulni_intel virtio_blk failover serio_raw
Link to test
sanity-hsm test 1a: mmap
general protection fault, probably for non-canonical address 0xcbb45735a568028: 0000 [#1] PREEMPT SMP PTI
CPU: 1 PID: 13470 Comm: mdt00_003 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffbbac47f1b768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9da645b9d620 RCX: ffff9da662306364
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0d70d40
RBP: ffffbbac47f1b7e8 R08: ffffbbac47f1b6c8 R09: 0000000000000000
R10: ffff9da6722cf0c0 R11: 0000000000000000 R12: 0000000000000000
R13: ffff9da662306340 R14: ffff9da6706e6000 R15: 0cbb45735a568000
FS: 0000000000000000(0000) GS:ffff9da6ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9c73bf44e0 CR3: 0000000002e0a001 CR4: 00000000000606f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common virtio_balloon pcspkr i2c_piix4 joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix crct10dif_pclmul crc32_pclmul crc32c_intel virtio_net libata ghash_clmulni_intel net_failover virtio_blk failover serio_raw
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n mdt.lustre-MDT0000.hsm.actions | awk '/'0x200000405:0x2:0x0'.*action='ARCHIVE'/ {print $13}' | cut -f2 -d=
Autotest: Test running for 5 minutes (lustre-reviews_review-dne-part-4_115677.45)
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n mdt.lustre-MDT0000.hsm.actions | awk '/'0x200000405:0x2:0x0'.*action='ARCHIVE'/ {print $13}' | cut -f2 -d=
Link to test
sanity test 39r: lazy atime update on OST
general protection fault, probably for non-canonical address 0xe88375e581eb6d28: 0000 [#1] PREEMPT SMP PTI
CPU: 0 PID: 10240 Comm: mdt00_000 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffada8408eb768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff8d0a77fbf2a0 RCX: ffff8d0a747e2bec
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0d29d40
RBP: ffffada8408eb7e8 R08: ffffada8408eb6c8 R09: 0000000000000000
R10: ffff8d0a76f02180 R11: ffffffffc1a96eab R12: 0000000000000000
R13: ffff8d0a747e2bc8 R14: ffff8d0a46ddd000 R15: e88375e581eb6d00
FS: 0000000000000000(0000) GS:ffff8d0affc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f01e61d94e0 CR3: 0000000009c92002 CR4: 00000000000606f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_alloc_comp_entries+0x2cb/0x4b0 [lod]
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs sunrpc rfkill intel_rapl_msr intel_rapl_common virtio_balloon pcspkr i2c_piix4 joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix libata crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel virtio_net net_failover virtio_blk failover serio_raw
Link to test
replay-ost-single test 9: Verify that no req deadline happened during recovery
general protection fault, probably for non-canonical address 0x4061ecc9f80a9228: 0000 [#1] PREEMPT SMP PTI
CPU: 0 PID: 86234 Comm: mdt00_002 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffbc21c1293768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9be4c7639380 RCX: ffff9be4edd67444
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0dcbd40
RBP: ffffbc21c12937e8 R08: ffffbc21c12936c8 R09: 0000000000000000
R10: ffff9be4dd99f300 R11: 0000000000000000 R12: 0000000000000000
R13: ffff9be4edd67420 R14: ffff9be4c53ab000 R15: 4061ecc9f80a9200
FS: 0000000000000000(0000) GS:ffff9be57fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f22f6ffba50 CR3: 000000006be10005 CR4: 00000000000606f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: dm_flakey tls osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common pcspkr virtio_balloon i2c_piix4 joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix crct10dif_pclmul crc32_pclmul crc32c_intel libata ghash_clmulni_intel virtio_net virtio_blk net_failover failover serio_raw [last unloaded: dm_flakey]
Link to test
sanityn test 9b: append to striped sparse file
general protection fault, probably for non-canonical address 0x735ee57b52e0aa28: 0000 [#1] PREEMPT SMP PTI
CPU: 1 PID: 10249 Comm: mdt00_000 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffafdf0065b768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff8c036e233540 RCX: ffff8c035c1cfc04
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0d3bd40
RBP: ffffafdf0065b7e8 R08: ffffafdf0065b6c8 R09: 0000000000000000
R10: ffff8c0371f8e480 R11: 0000000000000000 R12: 0000000000000000
R13: ffff8c035c1cfbe0 R14: ffff8c036b50f000 R15: 735ee57b52e0aa00
FS: 0000000000000000(0000) GS:ffff8c037bd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055c870ac4040 CR3: 000000011bcc8006 CR4: 00000000003706f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common virtio_balloon pcspkr i2c_piix4 joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix libata crct10dif_pclmul crc32_pclmul virtio_net crc32c_intel ghash_clmulni_intel net_failover failover virtio_blk serio_raw
Link to test
replay-single test 2c: setstripe replay
general protection fault, probably for non-canonical address 0xc12a7e96762a9d28: 0000 [#1] PREEMPT SMP PTI
CPU: 1 PID: 10242 Comm: mdt00_001 Kdump: loaded Tainted: G OE ------- --- 5.14.0-503.40.1_lustre.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:lod_qos_parse_config+0x620/0x1110 [lod]
Code: 00 00 00 48 89 44 24 20 0f b6 83 a9 00 00 00 48 89 0c 24 48 89 4c 24 38 e9 e9 00 00 00 41 c7 45 04 01 00 00 00 b8 01 00 00 00 <41> 89 47 28 41 8b 45 18 4c 89 ff 41 89 47 2c 41 8b b6 48 01 00 00
RSP: 0018:ffffa1d600857768 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff8ac0f7797380 RCX: ffff8ac0c45e8ffc
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc10edd40
RBP: ffffa1d6008577e8 R08: ffffa1d6008576c8 R09: 0000000000000000
R10: ffff8ac0f7550240 R11: 0000000000000000 R12: 0000000000000000
R13: ffff8ac0c45e8fd8 R14: ffff8ac0ecb60000 R15: c12a7e96762a9d00
FS: 0000000000000000(0000) GS:ffff8ac17fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f231709e4e0 CR3: 000000004da10003 CR4: 00000000000606f0
Call Trace:
<TASK>
? show_trace_log_lvl+0x1c4/0x2df
? show_trace_log_lvl+0x1c4/0x2df
? lod_prepare_create+0x15f/0x340 [lod]
? __die_body.cold+0x8/0xd
? die_addr+0x39/0x60
? exc_general_protection+0x1c8/0x420
? asm_exc_general_protection+0x22/0x30
? lod_qos_parse_config+0x620/0x1110 [lod]
? lod_qos_parse_config+0x528/0x1110 [lod]
lod_prepare_create+0x15f/0x340 [lod]
lod_declare_striped_create+0xda/0x440 [lod]
lod_declare_xattr_set+0x4e2/0x970 [lod]
dt_declare_xattr_set.constprop.0+0x69/0x1a0 [mdd]
mdd_create_data+0x43e/0x6b0 [mdd]
mdt_create_data+0x224/0x2f0 [mdt]
mdt_mfd_open+0x89f/0xdd0 [mdt]
mdt_finish_open+0x6e6/0xa20 [mdt]
mdt_open_by_fid_lock+0x675/0xcb0 [mdt]
mdt_reint_open+0x92a/0x25e0 [mdt]
? swabber_dumper_helper.isra.0+0x2f7/0x340 [ptlrpc]
? mdt_root_squash.isra.0+0x1e/0x440 [mdt]
? old_init_ucred_common+0x2b0/0x670 [mdt]
mdt_reint_rec+0x11c/0x270 [mdt]
mdt_reint_internal+0x4ea/0x9b0 [mdt]
mdt_intent_open+0x13c/0x430 [mdt]
? __pfx_mdt_intent_open+0x10/0x10 [mdt]
mdt_intent_opc.constprop.0+0xe8/0x5e0 [mdt]
mdt_intent_policy+0xff/0x480 [mdt]
ldlm_lock_enqueue+0x40b/0x8d0 [ptlrpc]
? cfs_hash_bd_add_locked+0x1f/0x90 [obdclass]
? __pfx_cfs_hash_rw_unlock+0x10/0x10 [obdclass]
ldlm_handle_enqueue+0x324/0x16d0 [ptlrpc]
tgt_enqueue+0x60/0x240 [ptlrpc]
tgt_handle_request0+0x14a/0x770 [ptlrpc]
tgt_request_handle+0x1eb/0xb80 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e8/0xd80 [ptlrpc]
ptlrpc_main+0x9e3/0xed0 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0xe0/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x50
</TASK>
Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common virtio_balloon i2c_piix4 pcspkr joydev drm fuse ext4 mbcache jbd2 ata_generic ata_piix crct10dif_pclmul libata crc32_pclmul crc32c_intel virtio_net net_failover ghash_clmulni_intel virtio_blk failover serio_raw
Lustre: DEBUG MARKER: sync; sync; sync
Lustre: DEBUG MARKER: /usr/sbin/lctl --device lustre-MDT0000 notransno
Lustre: DEBUG MARKER: modprobe dm-flakey;
Lustre: DEBUG MARKER: dmsetup table /dev/mapper/mds1_flakey
Lustre: DEBUG MARKER: dmsetup suspend --nolockfs --noflush /dev/mapper/mds1_flakey
Lustre: DEBUG MARKER: dmsetup load /dev/mapper/mds1_flakey --table "0 3964928 flakey 252:0 0 0 1800 1 drop_writes"
Lustre: DEBUG MARKER: dmsetup resume /dev/mapper/mds1_flakey
Lustre: DEBUG MARKER: /usr/sbin/lctl mark mds1 REPLAY BARRIER on lustre-MDT0000
Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
Link to test
Return to new crashes list