Match messages in logs (every line would be required to be present in log output Copy from "Messages before crash" column below): | |
Match messages in full crash (every line would be required to be present in crash log output Copy from "Full Crash" column below): | |
Limit to a test: (Copy from below "Failing text"): | |
Delete these reports as invalid (real bug in review or some such) | |
Bug or comment: | |
Extra info: |
Failing Test | Full Crash | Messages before crash | Comment |
---|---|---|---|
racer test 2: racer rename: onyx-80vm1.onyx.whamcloud.com,onyx-80vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 620613 Comm: ll_sa_620472 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.53.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 23 69 5b c3 cc cc cc cc 48 89 df e8 25 09 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffab60825cfe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000b RDX: 000000000010000c RSI: ffff97e0dc9e1b70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff97e078d72a00 R11: 0000000000000000 R12: ffff97e0dc9e1890 R13: ffff97e078d72a98 R14: ffff97e078d72a00 R15: ffff97e078d72aa8 FS: 0000000000000000(0000) GS:ffff97e0ffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000092610006 CR4: 00000000001706e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs intel_rapl_msr lockd intel_rapl_common grace fscache crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 142340:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 19 [0x200000bec:0x1a04:0x0] inode@0000000000000000: rc = -5 LustreError: 142340:0:(statahead.c:825:ll_statahead_interpret_work()) Skipped 1 previous similar message Lustre: dir [0x280000be6:0x221c:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 12 previous similar messages LustreError: 462106:0:(lov_object.c:1350:lov_layout_change()) lustre-clilov-ffff97e071926800: cannot apply new layout on [0x200000bec:0x1a04:0x0] : rc = -5 LustreError: 462106:0:(lov_object.c:1350:lov_layout_change()) Skipped 28 previous similar messages Lustre: dir [0x200000bea:0x1a14:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x240000beb:0x3504:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Autotest: Test running for 215 minutes (lustre-reviews_review-dne-part-9_114440.34) Lustre: dir [0x240000bea:0x2cc3:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages | Link to test |
racer test 1: racer on clients: onyx-96vm1.onyx.whamcloud.com,onyx-96vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 235276 Comm: ll_sa_234986 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.51.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 83 4f 5b c3 cc cc cc cc 48 89 df e8 25 09 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb7230116be08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000b RDX: 000000000010000c RSI: ffffa030c84a4570 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffffa03089915000 R11: 0000000000000000 R12: ffffa030c84a4290 R13: ffffa03089915098 R14: ffffa03089915000 R15: ffffa030899150a8 FS: 0000000000000000(0000) GS:ffffa030ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000044e10004 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 ata_generic mbcache jbd2 ata_piix libata crc32c_intel serio_raw virtio_net virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA LustreError: 157292:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bec:0x73:0x0]: rc = -5 LustreError: 157292:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 156300:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bec:0x73:0x0]: rc = -5 LustreError: 156300:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 5[159228]: segfault at 0 ip 000055729d009b48 sp 00007ffd626363e0 error 6 in 5[55729d004000+7000] Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <20> 33 20 6f 72 20 6c 61 74 65 72 20 3c 68 74 74 70 73 3a 2f 2f 67 LustreError: 158989:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03087a33000: inode [0x200000bec:0x7:0x0] mdc close failed: rc = -116 Lustre: dir [0x2c0000beb:0xc:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 156957:0:(mdc_request.c:1484:mdc_read_page()) lustre-MDT0002-mdc-ffffa03087a33000: dir page locate: [0x280000bd1:0x6f:0x0] at 0: rc -5 Lustre: Skipped 3 previous similar messages LustreError: 156917:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03087a33000: inode [0x200000bec:0x39:0x0] mdc close failed: rc = -2 Lustre: dir [0x280000be7:0x1c8:0x0] stripe 4 readdir failed: -2, directory is partially accessed! Lustre: Skipped 6 previous similar messages LustreError: 157425:0:(mdc_request.c:1484:mdc_read_page()) lustre-MDT0001-mdc-ffffa03087a33000: dir page locate: [0x240000beb:0x68:0x0] at 0: rc -5 LustreError: 157425:0:(mdc_request.c:1484:mdc_read_page()) Skipped 6 previous similar messages LustreError: 157074:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03087a33000: inode [0x2c0000be9:0x13:0x0] mdc close failed: rc = -2 LustreError: 157074:0:(file.c:248:ll_close_inode_openhandle()) Skipped 2 previous similar messages LustreError: 164359:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03044a39000: inode [0x280000be7:0x1c8:0x0] mdc close failed: rc = -2 LustreError: 164359:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message Lustre: dir [0x200000be9:0x9d:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages LustreError: 164735:0:(mdc_request.c:1484:mdc_read_page()) lustre-MDT0001-mdc-ffffa03044a39000: dir page locate: [0x240000bd1:0x4f:0x0] at 0: rc -5 LustreError: 164735:0:(mdc_request.c:1484:mdc_read_page()) Skipped 2 previous similar messages LustreError: 155220:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be6:0x1b2:0x0]: rc = -5 LustreError: 155220:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 173795:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be9:0x15f:0x0]: rc = -5 LustreError: 173795:0:(lcommon_cl.c:188:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 173795:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 173795:0:(llite_lib.c:3769:ll_prep_inode()) Skipped 2 previous similar messages LustreError: 174369:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03044a39000: inode [0x2c0000bec:0x145:0x0] mdc close failed: rc = -2 LustreError: 174369:0:(file.c:248:ll_close_inode_openhandle()) Skipped 2 previous similar messages LustreError: 176658:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be9:0x32f:0x0]: rc = -5 LustreError: 176658:0:(lcommon_cl.c:188:cl_file_inode_init()) Skipped 9 previous similar messages LustreError: 176658:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 176658:0:(llite_lib.c:3769:ll_prep_inode()) Skipped 9 previous similar messages 9[181257]: segfault at 8 ip 00007f77b38e8735 sp 00007ffe0cc36210 error 4 in ld-2.28.so[7f77b38c7000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 181114:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03087a33000: inode [0x280000be8:0x2fc:0x0] mdc close failed: rc = -116 LustreError: 181114:0:(file.c:248:ll_close_inode_openhandle()) Skipped 5 previous similar messages 7[182423]: segfault at 8 ip 00007f95814a6735 sp 00007ffe9b123c30 error 4 in ld-2.28.so[7f9581485000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 182060:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000beb:0x178:0x0]: rc = -5 LustreError: 182060:0:(lcommon_cl.c:188:cl_file_inode_init()) Skipped 7 previous similar messages LustreError: 182060:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 182060:0:(llite_lib.c:3769:ll_prep_inode()) Skipped 7 previous similar messages 3[182450]: segfault at 8 ip 00007fe43c5d5735 sp 00007ffee0b8deb0 error 4 in ld-2.28.so[7fe43c5b4000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 178554:0:(llite_nfs.c:430:ll_dir_get_parent_fid()) lustre: failure inode [0x200000bec:0x28f:0x0] get parent: rc = -116 LustreError: 187262:0:(lov_object.c:1350:lov_layout_change()) lustre-clilov-ffffa03044a39000: cannot apply new layout on [0x280000be7:0x59f:0x0] : rc = -5 LustreError: 187262:0:(vvp_io.c:1905:vvp_io_init()) lustre: refresh file layout [0x280000be7:0x59f:0x0] error -5. LustreError: 191300:0:(lov_object.c:1350:lov_layout_change()) lustre-clilov-ffffa03044a39000: cannot apply new layout on [0x280000be7:0x59f:0x0] : rc = -5 LustreError: 191300:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be7:0x5ea:0x0]: rc = -5 LustreError: 191300:0:(lcommon_cl.c:188:cl_file_inode_init()) Skipped 59 previous similar messages LustreError: 191300:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 191300:0:(llite_lib.c:3769:ll_prep_inode()) Skipped 59 previous similar messages Lustre: dir [0x200000be9:0x5ff:0x0] stripe 4 readdir failed: -2, directory is partially accessed! LustreError: 186969:0:(mdc_request.c:1484:mdc_read_page()) lustre-MDT0000-mdc-ffffa03044a39000: dir page locate: [0x200000beb:0x3b1:0x0] at 0: rc -5 Lustre: Skipped 53 previous similar messages LustreError: 186969:0:(mdc_request.c:1484:mdc_read_page()) Skipped 43 previous similar messages LustreError: 187749:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03044a39000: inode [0x200000be9:0x2d9:0x0] mdc close failed: rc = -2 LustreError: 187749:0:(file.c:248:ll_close_inode_openhandle()) Skipped 1 previous similar message LustreError: 143278:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 16 [0x0:0x0:0x0] inode@0000000000000000: rc = -5 LustreError: 191150:0:(lov_object.c:1350:lov_layout_change()) lustre-clilov-ffffa03044a39000: cannot apply new layout on [0x240000beb:0x3e9:0x0] : rc = -5 LustreError: 191150:0:(vvp_io.c:1905:vvp_io_init()) lustre: refresh file layout [0x240000beb:0x3e9:0x0] error -5. 1[201006]: segfault at 8 ip 00007f92245c7735 sp 00007fffc0e3c160 error 4 in ld-2.28.so[7f92245a6000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 0[201010]: segfault at 8 ip 00007f6aa9b78735 sp 00007ffdde974f40 error 4 in ld-2.28.so[7f6aa9b57000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 13[200681]: segfault at 8 ip 00007f38b6c0a735 sp 00007fffb941ed50 error 4 in ld-2.28.so[7f38b6be9000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 1[202611]: segfault at 8 ip 00007f2644da9735 sp 00007ffcb6190b50 error 4 in ld-2.28.so[7f2644d88000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 202950:0:(llite_nfs.c:430:ll_dir_get_parent_fid()) lustre: failure inode [0x200000be9:0x5ff:0x0] get parent: rc = -2 LustreError: 202950:0:(llite_nfs.c:430:ll_dir_get_parent_fid()) Skipped 19 previous similar messages LustreError: 204514:0:(lov_object.c:1350:lov_layout_change()) lustre-clilov-ffffa03044a39000: cannot apply new layout on [0x240000beb:0x3e9:0x0] : rc = -5 14[205933]: segfault at 8 ip 00007f81b62d2735 sp 00007ffef91e6d40 error 4 in ld-2.28.so[7f81b62b1000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 Lustre: dir [0x2c0000beb:0x4a4:0x0] stripe 1 readdir failed: -2, directory is partially accessed! LustreError: 195129:0:(mdc_request.c:1484:mdc_read_page()) lustre-MDT0000-mdc-ffffa03044a39000: dir page locate: [0x200000bd0:0x7b:0x0] at 0: rc -5 LustreError: 207297:0:(lov_object.c:1350:lov_layout_change()) lustre-clilov-ffffa03044a39000: cannot apply new layout on [0x200000beb:0x91b:0x0] : rc = -5 LustreError: 207297:0:(vvp_io.c:1905:vvp_io_init()) lustre: refresh file layout [0x200000beb:0x91b:0x0] error -5. 6[213900]: segfault at 8 ip 00007fcf3eefd735 sp 00007ffee0c91590 error 4 in ld-2.28.so[7fcf3eedc000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 211900:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be8:0x902:0x0]: rc = -5 LustreError: 211900:0:(lcommon_cl.c:188:cl_file_inode_init()) Skipped 30 previous similar messages LustreError: 211900:0:(llite_lib.c:3769:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 211900:0:(llite_lib.c:3769:ll_prep_inode()) Skipped 30 previous similar messages LustreError: 215967:0:(file.c:248:ll_close_inode_openhandle()) lustre-clilmv-ffffa03044a39000: inode [0x280000be9:0xa06:0x0] mdc close failed: rc = -2 LustreError: 215967:0:(file.c:248:ll_close_inode_openhandle()) Skipped 16 previous similar messages 0[224680]: segfault at 8 ip 00007f71dd9e6735 sp 00007fffea2c8370 error 4 in ld-2.28.so[7f71dd9c5000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 Lustre: dir [0x200000be9:0x9f3:0x0] stripe 3 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages | Link to test |
racer test 2: racer rename: onyx-83vm8.onyx.whamcloud.com,onyx-83vm9 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 869932 Comm: ll_sa_869824 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.44.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a a3 59 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffae6088fbfe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006 RDX: ffff9c3f3fd38160 RSI: ffff9c3ef8868f70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff9c3e81003180 R09: fffffc21c1c58980 R10: 0000000000000000 R11: 000000000000000f R12: ffff9c3ef8868c90 R13: ffff9c3ee7d27498 R14: ffff9c3ee7d27400 R15: ffff9c3ee7d274a8 FS: 0000000000000000(0000) GS:ffff9c3f3fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000067e10006 CR4: 00000000001706e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c9/0x2220 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon joydev pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel serio_raw net_failover failover virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x200000bea:0x1bc2:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages Autotest: Test running for 270 minutes (lustre-reviews_review-dne-part-9_113836.34) Lustre: dir [0x280000be8:0x3af7:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 9 previous similar messages LustreError: 861073:0:(llite_nfs.c:430:ll_dir_get_parent_fid()) lustre: failure inode [0x280000be8:0x3d3e:0x0] get parent: rc = -116 LustreError: 861073:0:(llite_nfs.c:430:ll_dir_get_parent_fid()) Skipped 5 previous similar messages | Link to test |
racer test 2: racer rename: onyx-108vm4.onyx.whamcloud.com,onyx-108vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 914585 Comm: ll_sa_914399 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.50.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 03 77 5b c3 cc cc cc cc 48 89 df e8 25 09 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb7a64570fe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000b RDX: 000000000010000c RSI: ffff9129b47de970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff912970899800 R11: 0000000000000000 R12: ffff9129b47de690 R13: ffff912970899898 R14: ffff912970899800 R15: ffff9129708998a8 FS: 0000000000000000(0000) GS:ffff9129ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000001e410001 CR4: 00000000001706f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_net virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 726924:0:(lov_object.c:1348:lov_layout_change()) lustre-clilov-ffff91295804c800: cannot apply new layout on [0x280000be7:0x783:0x0] : rc = -5 LustreError: 734041:0:(lov_object.c:1348:lov_layout_change()) lustre-clilov-ffff91295804c800: cannot apply new layout on [0x2c0000be9:0x3c0:0x0] : rc = -5 LustreError: 734041:0:(lov_object.c:1348:lov_layout_change()) Skipped 1 previous similar message Lustre: dir [0x2c0000be7:0x8ac:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: dir [0x200000be9:0x153b:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages Lustre: dir [0x2c0000be8:0x18b9:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages Autotest: Test running for 230 minutes (lustre-reviews_review-dne-part-9_113737.32) Lustre: dir [0x280000beb:0x21a2:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message | Link to test |
racer test 2: racer rename: trevis-85vm4.trevis.whamcloud.com,trevis-85vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 795925 Comm: ll_sa_795812 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.50.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 03 46 5b c3 cc cc cc cc 48 89 df e8 25 09 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa446874a7e08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000b RDX: 000000000010000c RSI: ffff9402caf13970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9402df545c00 R11: 0000000000000000 R12: ffff9402caf13690 R13: ffff9402df545c98 R14: ffff9402df545c00 R15: ffff9402df545ca8 FS: 0000000000000000(0000) GS:ffff9402ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000009aa10002 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic virtio_net ata_piix crc32c_intel serio_raw virtio_blk libata net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 535760:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 15 [0x2c0000bea:0x19a:0x0] inode@0000000000000000: rc = -5 Lustre: dir [0x200000bea:0x1c66:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 51 previous similar messages Autotest: Test running for 210 minutes (lustre-reviews_review-dne-part-9_113647.32) Lustre: dir [0x2c0000bea:0x27bf:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 7 previous similar messages | Link to test |
racer test 2: racer rename: trevis-29vm4.trevis.whamcloud.com,trevis-29vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 648454 Comm: ll_sa_648334 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.50.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 63 5d 5b c3 cc cc cc cc 48 89 df e8 25 09 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffad8a0892fe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100007 RDX: 0000000000100008 RSI: ffffa096201b6f70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffffa096081fec00 R11: 0000000000000000 R12: ffffa096201b6c90 R13: ffffa096081fec98 R14: ffffa096081fec00 R15: ffffa096081feca8 FS: 0000000000000000(0000) GS:ffffa0967fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000004fc10004 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon pcspkr joydev i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_blk virtio_net net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x200000bea:0xc43:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 152 previous similar messages Lustre: dir [0x200000bee:0x116b:0x0] stripe 2 readdir failed: -2, directory is partially accessed! | Link to test |
racer test 2: racer rename: onyx-142vm4.onyx.whamcloud.com,onyx-142vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 910989 Comm: ll_sa_910834 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.50.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 43 6a 5b c3 cc cc cc cc 48 89 df e8 25 09 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffae43825bbe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000d RDX: 000000000010000e RSI: ffff8d27f0116970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff8d27c24d1e00 R11: 0000000000000000 R12: ffff8d27f0116690 R13: ffff8d27c24d1e98 R14: ffff8d27c24d1e00 R15: ffff8d27c24d1ea8 FS: 0000000000000000(0000) GS:ffff8d27fbc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000013410006 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 intel_rapl_msr intel_rapl_common dns_resolver crct10dif_pclmul nfs lockd grace fscache crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_blk virtio_net net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 864067:0:(lov_object.c:1348:lov_layout_change()) lustre-clilov-ffff8d27c374d800: cannot apply new layout on [0x2c0000be9:0xb2f:0x0] : rc = -5 LustreError: 866330:0:(lcommon_cl.c:188:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000bea:0x3fe:0x0]: rc = -5 LustreError: 866330:0:(lcommon_cl.c:188:cl_file_inode_init()) Skipped 63 previous similar messages LustreError: 866330:0:(llite_lib.c:3700:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 866330:0:(llite_lib.c:3700:ll_prep_inode()) Skipped 63 previous similar messages Lustre: dir [0x240000beb:0x781:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 20 previous similar messages Autotest: Test running for 210 minutes (lustre-reviews_review-dne-part-9_113402.11) | Link to test |
racer test 2: racer rename: onyx-31vm1.onyx.whamcloud.com,onyx-31vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 846984 Comm: ll_sa_846850 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.44.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 03 60 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffc156019dbe08 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff9ddf7c21b370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff9ddf37313400 R11: 0000000000000000 R12: ffff9ddf7c21b090 R13: ffff9ddf37313498 R14: ffff9ddf37313400 R15: ffff9ddf373134a8 FS: 0000000000000000(0000) GS:ffff9ddfbfc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000008b210003 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c9/0x2220 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net serio_raw crc32c_intel net_failover failover virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x2c0000be9:0x14dd:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 13 previous similar messages Autotest: Test running for 270 minutes (lustre-reviews_review-dne-part-9_112800.11) Lustre: dir [0x280000be7:0x2a5d:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x200000be9:0x3608:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 10 previous similar messages | Link to test |
racer test 2: racer rename: onyx-143vm1.onyx.whamcloud.com,onyx-143vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1169952 Comm: ll_sa_1169815 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.46.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 03 5a 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffbaea841abe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000b RDX: 000000008010000c RSI: ffff8fa0634a3370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff8f9fa3b7f400 R11: 0000000000000000 R12: ffff8fa0634a3090 R13: ffff8f9fa3b7f498 R14: ffff8f9fa3b7f400 R15: ffff8f9fa3b7f4a8 FS: 0000000000000000(0000) GS:ffff8fa07bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000017010006 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev i2c_piix4 pcspkr virtio_balloon sunrpc ata_generic ext4 mbcache jbd2 ata_piix libata virtio_net crc32c_intel serio_raw net_failover virtio_blk failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Autotest: Test running for 300 minutes (lustre-reviews_review-dne-part-9_112783.11) LustreError: 926186:0:(llite_lib.c:3696:ll_prep_inode()) lustre: new_inode - fatal error: rc = -2 LustreError: 926186:0:(llite_lib.c:3696:ll_prep_inode()) Skipped 2040 previous similar messages Lustre: dir [0x280000be6:0x14e9:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 76 previous similar messages Lustre: dir [0x240000be7:0x21b7:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x200000beb:0x2e3b:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x2c0000bea:0x40ca:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages | Link to test |
racer test 2: racer rename: onyx-147vm1.onyx.whamcloud.com,onyx-147vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 738654 Comm: ll_sa_738521 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.46.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a a3 5b 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffff9ca98215fe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000d RDX: 000000000010000e RSI: ffff91668818b370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9166aae0c600 R11: 0000000000000000 R12: ffff91668818b090 R13: ffff9166aae0c698 R14: ffff9166aae0c600 R15: ffff9166aae0c6a8 FS: 0000000000000000(0000) GS:ffff9166bbd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 00000000ae010003 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr i2c_piix4 virtio_balloon sunrpc ext4 mbcache jbd2 ata_generic crc32c_intel ata_piix serio_raw libata virtio_net net_failover failover virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c | Link to test |
racer test 1: racer on clients: onyx-32vm1.onyx.whamcloud.com,onyx-32vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 683020 Comm: ll_sa_464692 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.46.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a e3 52 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa61d0233fe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100007 RDX: 0000000000100008 RSI: ffff9755a8a47570 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9755cdb95000 R11: 0000000000000000 R12: ffff9755a8a47290 R13: ffff9755cdb95098 R14: ffff9755cdb95000 R15: ffff9755cdb950a8 FS: 0000000000000000(0000) GS:ffff97563fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000062e10006 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x56c/0x1f60 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_interpret+0x440/0x440 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel i2c_piix4 pcspkr joydev virtio_balloon sunrpc ext4 mbcache jbd2 ata_generic crc32c_intel ata_piix libata virtio_net serio_raw virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA LustreError: 153422:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755cb6e6800: inode [0x200000beb:0xf:0x0] mdc close failed: rc = -116 LustreError: 154494:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755cb6e6800: inode [0x200000be9:0x37:0x0] mdc close failed: rc = -116 LustreError: 156503:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755cb6e6800: inode [0x2c0000be6:0x45:0x0] mdc close failed: rc = -2 LustreError: 156852:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755baefb800: inode [0x200000be9:0x2e:0x0] mdc close failed: rc = -2 Lustre: dir [0x200000bec:0xdd:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 159407:0:(lov_object.c:1341:lov_layout_change()) lustre-clilov-ffff9755baefb800: cannot apply new layout on [0x240000be6:0x121:0x0] : rc = -5 LustreError: 159407:0:(vvp_io.c:1903:vvp_io_init()) lustre: refresh file layout [0x240000be6:0x121:0x0] error -5. Lustre: dir [0x280000be8:0x79:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message 2[162370]: segfault at 8 ip 00007f28cc3fd735 sp 00007ffe6fa1b0a0 error 4 in ld-2.28.so[7f28cc3dc000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 158816:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755cb6e6800: inode [0x280000be9:0x9b:0x0] mdc close failed: rc = -116 LustreError: 158816:0:(file.c:247:ll_close_inode_openhandle()) Skipped 3 previous similar messages LustreError: 163874:0:(lov_object.c:1341:lov_layout_change()) lustre-clilov-ffff9755baefb800: cannot apply new layout on [0x240000be6:0x121:0x0] : rc = -5 LustreError: 163874:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000be6:0x121:0x0]: rc = -5 LustreError: 163874:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 Autotest: Test running for 210 minutes (lustre-reviews_review-dne-part-9_112503.11) LustreError: 163874:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000be9:0xba:0x0]: rc = -5 LustreError: 163874:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 1 previous similar message LustreError: 163874:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 163874:0:(llite_lib.c:3698:ll_prep_inode()) Skipped 1 previous similar message LustreError: 163706:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755cb6e6800: inode [0x200000be9:0x7b:0x0] mdc close failed: rc = -116 LustreError: 163706:0:(file.c:247:ll_close_inode_openhandle()) Skipped 3 previous similar messages 10[441045]: segfault at 0 ip 000055ba9ba49b47 sp 00007ffcc8262c20 error 6 in 10[55ba9ba45000+7000] Code: Unable to access opcode bytes at RIP 0x55ba9ba49b1d. LustreError: 442890:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x2b:0x0]: rc = -5 LustreError: 442890:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 6 previous similar messages LustreError: 442890:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 442890:0:(llite_lib.c:3698:ll_prep_inode()) Skipped 6 previous similar messages LustreError: 445409:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000be6:0x1f6:0x0]: rc = -5 LustreError: 445409:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 Lustre: dir [0x200000bed:0xb2:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 446566:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be8:0x2b:0x0]: rc = -5 LustreError: 446566:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 446566:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 446566:0:(llite_lib.c:3698:ll_prep_inode()) Skipped 2 previous similar messages LustreError: 449096:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9755baefb800: inode [0x280000be8:0x1ca:0x0] mdc close failed: rc = -116 LustreError: 449096:0:(file.c:247:ll_close_inode_openhandle()) Skipped 8 previous similar messages Lustre: dir [0x2c0000be8:0x2b9:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 452633:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000be9:0x31e:0x0]: rc = -5 LustreError: 452633:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 2 previous similar messages LustreError: 452633:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 452633:0:(llite_lib.c:3698:ll_prep_inode()) Skipped 2 previous similar messages Lustre: dir [0x200000bea:0x40e:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 452189:0:(mdc_request.c:1479:mdc_read_page()) lustre-MDT0003-mdc-ffff9755baefb800: dir page locate: [0x2c0000bd1:0x71:0x0] at 0: rc -5 LustreError: 457866:0:(mdc_request.c:1479:mdc_read_page()) lustre-MDT0001-mdc-ffff9755cb6e6800: dir page locate: [0x240000bd1:0x73:0x0] at 0: rc -5 LustreError: 457866:0:(mdc_request.c:1479:mdc_read_page()) Skipped 1 previous similar message LustreError: 466458:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be6:0x387:0x0]: rc = -5 LustreError: 466458:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 4 previous similar messages LustreError: 466458:0:(llite_lib.c:3698:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 466458:0:(llite_lib.c:3698:ll_prep_inode()) Skipped 4 previous similar messages LustreError: lustre-OST0001-osc-ffff9755cb6e6800: operation ost_sync to node 10.240.22.242@tcp failed: rc = -107 Lustre: lustre-OST0001-osc-ffff9755cb6e6800: Connection to lustre-OST0001 (at 10.240.22.242@tcp) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-OST0001-osc-ffff9755cb6e6800: This client was evicted by lustre-OST0001; in progress operations using this service will fail. LustreError: 681484:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-OST0001-osc-ffff9755cb6e6800: namespace resource [0x340000407:0x1c7d:0x0].0x0 (ffff9755ee3e3540) refcount nonzero (2) after lock cleanup; forcing cleanup. LustreError: lustre-MDT0003-mdc-ffff9755cb6e6800: operation mds_close to node 10.240.22.246@tcp failed: rc = -107 Lustre: lustre-MDT0003-mdc-ffff9755cb6e6800: Connection to lustre-MDT0003 (at 10.240.22.246@tcp) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0003-mdc-ffff9755cb6e6800: This client was evicted by lustre-MDT0003; in progress operations using this service will fail. | Link to test |
racer test 2: racer rename: onyx-146vm1.onyx.whamcloud.com,onyx-146vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1007119 Comm: ll_sa_1006972 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.44.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 63 60 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffbc04c60dbe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff9a4c6001d770 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff9a4cc611a200 R11: 0000000000000000 R12: ffff9a4c6001d490 R13: ffff9a4cc611a298 R14: ffff9a4cc611a200 R15: ffff9a4cc611a2a8 FS: 0000000000000000(0000) GS:ffff9a4cfbc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000005b210002 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c9/0x2220 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr nfs intel_rapl_common lockd grace fscache crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix crc32c_intel libata virtio_net serio_raw virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 743827:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000beb:0x57e:0x0]: rc = -5 LustreError: 743827:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 28 previous similar messages LustreError: 743827:0:(llite_lib.c:3695:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 743827:0:(llite_lib.c:3695:ll_prep_inode()) Skipped 28 previous similar messages Lustre: dir [0x200000beb:0x1628:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages Autotest: Test running for 255 minutes (lustre-reviews_review-dne-part-9_112356.11) Lustre: dir [0x280000bec:0x3062:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 7 previous similar messages | Link to test |
racer test 2: racer rename: onyx-150vm1.onyx.whamcloud.com,onyx-150vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 613268 Comm: ll_sa_613125 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.44.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 03 76 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa965c802be08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff9504307bf570 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff950363312a00 R11: 0000000000000000 R12: ffff9504307bf290 R13: ffff950363312a98 R14: ffff950363312a00 R15: ffff950363312aa8 FS: 0000000000000000(0000) GS:ffff95043bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 00000000bc010003 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c9/0x2220 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs intel_rapl_msr lockd intel_rapl_common grace fscache crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net net_failover serio_raw failover virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c | Link to test |
racer test 2: racer rename: trevis-23vm4.trevis.whamcloud.com,trevis-23vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 816608 Comm: ll_sa_816492 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.44.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 83 4b 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb7228471fe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006 RDX: ffff8cf9ffc38160 RSI: ffff8cf988aaef70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff8cf941003180 R09: fffffaf8c0e92c80 R10: 0000000000000000 R11: 000000000000000f R12: ffff8cf988aaec90 R13: ffff8cf952e67098 R14: ffff8cf952e67000 R15: ffff8cf952e670a8 FS: 0000000000000000(0000) GS:ffff8cf9ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000001a410002 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c9/0x2220 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon i2c_piix4 joydev pcspkr sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 92179:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be4:0x963:0x0]: rc = -5 LustreError: 92179:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 17 previous similar messages LustreError: 92179:0:(llite_lib.c:3695:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 92179:0:(llite_lib.c:3695:ll_prep_inode()) Skipped 17 previous similar messages LustreError: 92179:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 16 [0x2c0000be4:0x963:0x0] inode@0000000000000000: rc = -5 LustreError: 576924:0:(lov_object.c:1341:lov_layout_change()) lustre-clilov-ffff8cf98fabf000: cannot apply new layout on [0x240000be8:0x304:0x0] : rc = -5 Autotest: Test running for 395 minutes (lustre-reviews_review-dne-part-9_112015.11) LustreError: 584326:0:(lov_object.c:1341:lov_layout_change()) lustre-clilov-ffff8cf98fabf000: cannot apply new layout on [0x240000be8:0x304:0x0] : rc = -5 LustreError: 585127:0:(llite_nfs.c:430:ll_dir_get_parent_fid()) lustre: failure inode [0x2c0000be4:0xb66:0x0] get parent: rc = -116 Lustre: dir [0x200000be9:0x23a2:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 74 previous similar messages Lustre: dir [0x280000be8:0x16e8:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Autotest: Test running for 400 minutes (lustre-reviews_review-dne-part-9_112015.11) | Link to test |
racer test 2: racer rename: onyx-55vm1.onyx.whamcloud.com,onyx-55vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 534493 Comm: ll_sa_534362 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.44.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 d3 6a 43 47 5b c3 cc cc cc cc 48 89 df e8 25 19 af ff 39 05 23 8f 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb62806e0fe08 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff921e1e398970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff921dd8815800 R11: 0000000000000000 R12: ffff921e1e398690 R13: ffff921dd8815898 R14: ffff921dd8815800 R15: ffff921dd88158a8 FS: 0000000000000000(0000) GS:ffff921e3fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000062410005 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c9/0x2220 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_net virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 419940:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 14 [0x280000bea:0x881:0x0] inode@0000000000000000: rc = -5 LustreError: 419940:0:(statahead.c:825:ll_statahead_interpret_work()) Skipped 6 previous similar messages LustreError: 435283:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000be9:0xac6:0x0]: rc = -5 LustreError: 435283:0:(lcommon_cl.c:179:cl_file_inode_init()) Skipped 38 previous similar messages LustreError: 435283:0:(llite_lib.c:3692:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 435283:0:(llite_lib.c:3692:ll_prep_inode()) Skipped 38 previous similar messages Lustre: dir [0x280000bea:0x15cd:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages LustreError: 380208:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 19 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 380208:0:(statahead.c:825:ll_statahead_interpret_work()) Skipped 11 previous similar messages | Link to test |
racer test 2: racer rename: onyx-45vm1.onyx.whamcloud.com,onyx-45vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 612513 Comm: ll_sa_612372 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.40.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a 23 7b 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffae9384ed7e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000b RDX: 000000008010000c RSI: ffff98851ddd3970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff988523b65000 R11: 0000000000000000 R12: ffff98851ddd3690 R13: ffff988523b65098 R14: ffff988523b65000 R15: ffff988523b650a8 FS: 0000000000000000(0000) GS:ffff98853fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000057610002 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic crc32c_intel ata_piix libata serio_raw virtio_net virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: lustre-OST0001-osc-ffff9884bce20000: operation ldlm_enqueue to node 10.240.23.246@tcp failed: rc = -107 Lustre: lustre-OST0001-osc-ffff9884bce20000: Connection to lustre-OST0001 (at 10.240.23.246@tcp) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-OST0001-osc-ffff9884bce20000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. Lustre: lustre-OST0001-osc-ffff9884bce20000: Connection restored to (at 10.240.23.246@tcp) LustreError: 397301:0:(lcommon_cl.c:179:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be6:0x13e7:0x0]: rc = -5 LustreError: 397301:0:(llite_lib.c:3715:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 Autotest: Test running for 300 minutes (lustre-reviews_review-dne-part-9_111811.11) Lustre: dir [0x2c0000be6:0x21b0:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 23 previous similar messages LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 2 [0x0:0x0:0x0] inode@0000000000000000: rc = -1 LustreError: 66532:0:(statahead.c:825:ll_statahead_interpret_work()) Skipped 1 previous similar message | Link to test |
racer test 2: racer rename: trevis-24vm4.trevis.whamcloud.com,trevis-24vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 780152 Comm: ll_sa_780001 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.40.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a 83 6e 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb5b644d33e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000b RDX: ffff954a3fc38160 RSI: ffff954a27915170 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff954981003180 R09: fffffb19029c7e80 R10: 0000000000000000 R11: 000000000000000f R12: ffff954a27914e90 R13: ffff9549bc9d1c98 R14: ffff9549bc9d1c00 R15: ffff9549bc9d1ca8 FS: 0000000000000000(0000) GS:ffff954a3fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000085410004 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon i2c_piix4 joydev pcspkr sunrpc ext4 ata_generic mbcache jbd2 ata_piix libata crc32c_intel virtio_net serio_raw virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x240000be6:0x6d4:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 6 previous similar messages LustreError: 549248:0:(statahead.c:825:ll_statahead_interpret_work()) lustre: failed to prep 5 [0x2c0000be9:0x59a:0x0] inode@0000000000000000: rc = -5 Lustre: dir [0x2c0000be9:0x1b9c:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x200000be8:0x2289:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages | Link to test |
racer test 2: racer rename: onyx-52vm2.onyx.whamcloud.com,onyx-52vm3 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 630962 Comm: ll_sa_630793 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.40.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a 83 63 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffbb6002d37e10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000005 RDX: ffff99cfffc38160 RSI: ffff99cfc2663f70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff99cf41003180 R09: fffff86cc11dc900 R10: 0000000000000000 R11: 000000000000000f R12: ffff99cfc2663c90 R13: ffff99cf54939898 R14: ffff99cf54939800 R15: ffff99cf549398a8 FS: 0000000000000000(0000) GS:ffff99cfffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000002ba10001 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw net_failover failover virtio_blk CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x20000041c:0x52b:0x0] stripe 1 readdir failed: -2, directory is partially accessed! | Link to test |
racer test 2: racer rename: onyx-37vm1.onyx.whamcloud.com,onyx-37vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 154506 Comm: ll_sa_154436 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.40.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a a3 55 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffab19429bbe10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000008 RDX: ffff92e3bfd38160 RSI: ffff92e32e940970 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff92e301003180 R09: ffffcd234042a500 R10: 0000000000000000 R11: 000000000000000f R12: ffff92e32e940690 R13: ffff92e3252c1a98 R14: ffff92e3252c1a00 R15: ffff92e3252c1aa8 FS: 0000000000000000(0000) GS:ffff92e3bfd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000004ea10006 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_blk virtio_net net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x240000403:0x17d8:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Autotest: Test running for 5 minutes (lustre-reviews_custom_111373.1005) | Link to test |
racer test 2: racer rename: onyx-108vm1.onyx.whamcloud.com,onyx-108vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 36044 Comm: ll_sa_35977 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.40.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a 23 7a 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa29b452bbe10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006 RDX: ffff89137fd38160 RSI: ffff8912e0f3ef70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff8912c1003180 R09: fffffa0000af7800 R10: 0000000000000000 R11: 000000000000000f R12: ffff8912e0f3ec90 R13: ffff8912e3ebe298 R14: ffff8912e3ebe200 R15: ffff8912e3ebe2a8 FS: 0000000000000000(0000) GS:ffff89137fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000039610003 CR4: 00000000001706e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr virtio_balloon joydev i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic virtio_net crc32c_intel ata_piix serio_raw virtio_blk libata net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=2 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c | Link to test |
racer test 1: racer on clients: onyx-138vm4.onyx.whamcloud.com,onyx-138vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 541813 Comm: ll_sa_541572 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.37.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a e3 6a 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa997c6e97e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000001 RDX: ffff92273bc38160 RSI: ffff922692e25170 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff922700003180 R09: 0000000000000000 R10: 0000000000000000 R11: 000000000000000f R12: ffff922692e24e90 R13: ffff92260d9f4298 R14: ffff92260d9f4200 R15: ffff92260d9f42a8 FS: 0000000000000000(0000) GS:ffff92273bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000027810002 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 intel_rapl_msr dns_resolver intel_rapl_common crct10dif_pclmul nfs lockd grace fscache crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_net virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_ENABLE_REMOTE_DIRS=true RACER_ENABLE_STRIPED_DIRS=true RACER_ENABLE_MIGRATION=true RACER_ENABLE_FILE_MIGRATE=true RACER_ENABLE_PFL=true RACER_ENABLE_DOM=true RACER_ENABLE_FLR=true RACER_MA LustreError: 81474:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9226a23e2800: inode [0x200000bec:0x3:0x0] mdc close failed: rc = -116 LustreError: 81994:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9226a23e2800: inode [0x200000bec:0x3:0x0] mdc close failed: rc = -116 LustreError: 83476:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9226a23e2800: inode [0x200000bec:0x6e:0x0] mdc close failed: rc = -116 LustreError: 83476:0:(file.c:247:ll_close_inode_openhandle()) Skipped 1 previous similar message LustreError: 86211:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff922719431800: inode [0x280000be9:0x4:0x0] mdc close failed: rc = -116 LustreError: 86211:0:(file.c:247:ll_close_inode_openhandle()) Skipped 4 previous similar messages 2[93752]: segfault at 8 ip 00007f68d0240735 sp 00007fffb26bcb40 error 4 in ld-2.28.so[7f68d021f000+2f000] Code: 81 39 52 e5 74 64 0f 84 99 09 00 00 48 85 c0 75 e4 48 83 3d 64 f7 20 00 00 0f 85 aa 09 00 00 49 8b 47 68 49 8b 97 68 02 00 00 <48> 8b 40 08 48 85 d2 74 23 48 8b 72 08 48 01 c6 80 3e 00 74 17 48 LustreError: 97079:0:(lcommon_cl.c:177:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be8:0x65:0x0]: rc = -5 LustreError: 97079:0:(llite_lib.c:3712:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: lustre-MDT0002-mdc-ffff9226a23e2800: operation ldlm_enqueue to node 10.240.25.225@tcp failed: rc = -107 Lustre: lustre-MDT0002-mdc-ffff9226a23e2800: Connection to lustre-MDT0002 (at 10.240.25.225@tcp) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0002-mdc-ffff9226a23e2800: This client was evicted by lustre-MDT0002; in progress operations using this service will fail. LustreError: 91157:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9226a23e2800: inode [0x280000be7:0xed:0x0] mdc close failed: rc = -108 LustreError: 91157:0:(file.c:247:ll_close_inode_openhandle()) Skipped 2 previous similar messages LustreError: 94103:0:(file.c:5986:ll_inode_revalidate_fini()) lustre: revalidate FID [0x280000be8:0x7a:0x0] error: rc = -5 Lustre: lustre-MDT0002-mdc-ffff9226a23e2800: Connection restored to 10.240.25.225@tcp (at 10.240.25.225@tcp) LustreError: lustre-MDT0000-mdc-ffff9226a23e2800: operation ldlm_enqueue to node 10.240.25.225@tcp failed: rc = -107 LustreError: Skipped 2 previous similar messages Lustre: lustre-MDT0000-mdc-ffff9226a23e2800: Connection to lustre-MDT0000 (at 10.240.25.225@tcp) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-mdc-ffff9226a23e2800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. LustreError: 97814:0:(file.c:5986:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000bed:0x130:0x0] error: rc = -5 LustreError: 93120:0:(llite_lib.c:1997:ll_md_setattr()) md_setattr fails: rc = -108 LustreError: 97814:0:(file.c:5986:ll_inode_revalidate_fini()) Skipped 5 previous similar messages LustreError: 102566:0:(mdc_request.c:1457:mdc_read_page()) lustre-MDT0000-mdc-ffff9226a23e2800: [0x200000bea:0x2:0x0] lock enqueue fails: rc = -108 LustreError: 102566:0:(statahead.c:1801:is_first_dirent()) lustre: reading dir [0x200000bea:0x2:0x0] at 0 stat_pid = 96113 : rc = -108 LustreError: 99854:0:(vvp_io.c:1903:vvp_io_init()) lustre: refresh file layout [0x200000bea:0xac:0x0] error -108. Lustre: dir [0x2c0000be8:0xb0:0x0] stripe 1 readdir failed: -108, directory is partially accessed! LustreError: 96719:0:(llite_nfs.c:426:ll_dir_get_parent_fid()) lustre: failure inode [0x200000bec:0x138:0x0] get parent: rc = -108 LustreError: 96719:0:(llite_nfs.c:426:ll_dir_get_parent_fid()) Skipped 2 previous similar messages Lustre: lustre-MDT0000-mdc-ffff9226a23e2800: Connection restored to 10.240.25.225@tcp (at 10.240.25.225@tcp) LustreError: 464861:0:(lcommon_cl.c:177:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bea:0x118:0x0]: rc = -5 LustreError: 464861:0:(llite_lib.c:3712:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 94896:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff922719431800: inode [0x200000bea:0xb0:0x0] mdc close failed: rc = -2 LustreError: 94896:0:(file.c:247:ll_close_inode_openhandle()) Skipped 29 previous similar messages LustreError: 488316:0:(lcommon_cl.c:177:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bea:0x118:0x0]: rc = -5 LustreError: 488316:0:(llite_lib.c:3712:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 498461:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9226a23e2800: inode [0x200000beb:0x2a0:0x0] mdc close failed: rc = -116 LustreError: 498461:0:(file.c:247:ll_close_inode_openhandle()) Skipped 6 previous similar messages Autotest: Test running for 310 minutes (lustre-reviews_review-dne-part-9_111171.11) LustreError: 527877:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9226a23e2800: inode [0x280000be6:0x3b0:0x0] mdc close failed: rc = -116 LustreError: 527877:0:(file.c:247:ll_close_inode_openhandle()) Skipped 12 previous similar messages | Link to test |
racer test 2: racer rename: trevis-36vm1.trevis.whamcloud.com,trevis-36vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 296056 Comm: ll_sa_295925 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.37.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 e3 6a 23 68 5b c3 cc cc cc cc 48 89 df e8 e5 18 af ff 39 05 f3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb32b0792fe10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000004 RDX: ffff9469bfd38160 RSI: ffff9469456d0f70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff946901003180 R09: ffffdf5b00110d00 R10: 0000000000000000 R11: 000000000000000f R12: ffff9469456d0c90 R13: ffff94698cc89698 R14: ffff94698cc89600 R15: ffff94698cc896a8 FS: 0000000000000000(0000) GS:ffff9469bfd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000089610001 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon joydev pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw net_failover virtio_blk failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x200000be9:0x13c0:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message | Link to test |
racer test 2: racer rename: onyx-80vm1.onyx.whamcloud.com,onyx-80vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 1016854 Comm: ll_sa_1016763 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a 83 56 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb98d452d3e10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000008 RDX: ffff99b93fd38160 RSI: ffff99b91d62fb70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff99b881003180 R09: fffff0eec1209d00 R10: 0000000000000000 R11: 000000000000000f R12: ffff99b91d62f890 R13: ffff99b916a9d698 R14: ffff99b916a9d600 R15: ffff99b916a9d6a8 FS: 0000000000000000(0000) GS:ffff99b93fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000004c210003 CR4: 00000000001706e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_blk serio_raw virtio_net net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x200000bee:0x168c:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages | Link to test |
racer test 2: racer rename: onyx-58vm1.onyx.whamcloud.com,onyx-58vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 567944 Comm: ll_sa_567793 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a e3 48 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb9d882bdbe10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000c RDX: 000000000010000d RSI: ffff9cef7c85b370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9cef88c58a00 R11: 0000000000000000 R12: ffff9cef7c85b090 R13: ffff9cef88c58a98 R14: ffff9cef88c58a00 R15: ffff9cef88c58aa8 FS: 0000000000000000(0000) GS:ffff9cefffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000042a10002 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel serio_raw net_failover virtio_blk failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 349513:0:(lov_object.c:1341:lov_layout_change()) lustre-clilov-ffff9cef92319800: cannot apply new layout on [0x2c0000be5:0x10e6:0x0] : rc = -5 Autotest: Test running for 270 minutes (lustre-reviews_review-dne-part-9_110121.11) Lustre: dir [0x280000be6:0x34dd:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 14 previous similar messages | Link to test |
racer test 2: racer rename: onyx-144vm1.onyx.whamcloud.com,onyx-144vm2 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 3235121 Comm: ll_sa_3234929 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a c3 49 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb8b185517e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000008 RDX: ffff9350fbd38160 RSI: ffff934fc938f570 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff9350c0003180 R09: fffff74ec0d6e480 R10: 0000000000000000 R11: 000000000000000f R12: ffff934fc938f290 R13: ffff934fd8444498 R14: ffff934fd8444400 R15: ffff934fd84444a8 FS: 0000000000000000(0000) GS:ffff9350fbd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000007ca10005 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: loop ib_core nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs intel_rapl_msr lockd grace fscache intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix crc32c_intel libata serio_raw virtio_blk virtio_net net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c | Link to test |
racer test 2: racer rename: onyx-135vm1.onyx.whamcloud.com,onyx-135vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 795465 Comm: ll_sa_795361 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a 83 59 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffab4903223e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000a RDX: 000000008010000b RSI: ffff9a32292fe370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff9a3262482000 R11: 0000000000000000 R12: ffff9a32292fe090 R13: ffff9a3262482098 R14: ffff9a3262482000 R15: ffff9a32624820a8 FS: 0000000000000000(0000) GS:ffff9a32fbc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000133410004 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss intel_rapl_msr nfsv4 intel_rapl_common crct10dif_pclmul dns_resolver nfs crc32_pclmul lockd grace fscache ghash_clmulni_intel joydev pcspkr i2c_piix4 virtio_balloon sunrpc ata_generic ext4 mbcache jbd2 ata_piix libata virtio_net crc32c_intel net_failover failover virtio_blk serio_raw CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Autotest: Test running for 240 minutes (lustre-reviews_review-dne-part-9_109854.11) Lustre: dir [0x200000bee:0x362d:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 64 previous similar messages | Link to test |
racer test 2: racer rename: onyx-124vm1.onyx.whamcloud.com,onyx-124vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1018477 Comm: ll_sa_1018349 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a e3 59 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffbd700495be10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100009 RDX: 000000000010000a RSI: ffff9d88f8f72d70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9d88ab8cb200 R11: 0000000000000000 R12: ffff9d88f8f72a90 R13: ffff9d88ab8cb298 R14: ffff9d88ab8cb200 R15: ffff9d88ab8cb2a8 FS: 0000000000000000(0000) GS:ffff9d893fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000005e610004 CR4: 00000000001706f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x271/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr nfs lockd intel_rapl_common grace fscache crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel serio_raw net_failover virtio_blk failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 906512:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be4:0x8fb:0x0]: rc = -5 LustreError: 906512:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 51 previous similar messages LustreError: 906512:0:(llite_lib.c:3736:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 906512:0:(llite_lib.c:3736:ll_prep_inode()) Skipped 51 previous similar messages LustreError: 169354:0:(statahead.c:836:ll_statahead_interpret_work()) lustre: failed to prep 15 [0x280000be7:0x6dd:0x0] inode@0000000000000000: rc = -5 Autotest: Test running for 190 minutes (lustre-reviews_review-dne-part-9_109832.11) Lustre: dir [0x240000bea:0x1838:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 6 previous similar messages | Link to test |
racer test 2: racer rename: onyx-150vm5.onyx.whamcloud.com,onyx-150vm6 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 718877 Comm: ll_sa_718725 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a 43 7b 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffae4545f8fe10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000008 RDX: ffff9344bbc38160 RSI: ffff9344b568a770 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff934480003180 R09: ffffde9a44b25280 R10: 0000000000000000 R11: 000000000000000f R12: ffff9344b568a490 R13: ffff93439a8d7e98 R14: ffff93439a8d7e00 R15: ffff93439a8d7ea8 FS: 0000000000000000(0000) GS:ffff9344bbc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000044810002 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd intel_rapl_msr grace fscache intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr i2c_piix4 virtio_balloon sunrpc ata_generic ext4 mbcache jbd2 ata_piix libata crc32c_intel virtio_net net_failover serio_raw failover virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 675891:0:(mdc_request.c:1469:mdc_read_page()) lustre-MDT0002-mdc-ffff93449fdb8800: dir page locate: [0x280000bd5:0x8b:0x0] at 0: rc -5 LustreError: 675891:0:(mdc_request.c:1469:mdc_read_page()) Skipped 5 previous similar messages LustreError: 144257:0:(statahead.c:836:ll_statahead_interpret_work()) lustre: failed to prep 19 [0x240000be9:0x102b:0x0] inode@0000000000000000: rc = -5 Lustre: dir [0x280000be6:0x12fd:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 24 previous similar messages | Link to test |
racer test 2: racer rename: onyx-147vm3.onyx.whamcloud.com,onyx-147vm4 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 929220 Comm: ll_sa_929091 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a a3 5c 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb59b4716be10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000003 RDX: ffffa092fbc38160 RSI: ffffa0927f550f70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffffa092c0003180 R09: ffffeafdc4e74000 R10: 0000000000000000 R11: 000000000000000f R12: ffffa0927f550c90 R13: ffffa092c454e298 R14: ffffa092c454e200 R15: ffffa092c454e2a8 FS: 0000000000000000(0000) GS:ffffa092fbc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000038c10004 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 intel_rapl_msr intel_rapl_common dns_resolver crct10dif_pclmul nfs crc32_pclmul lockd grace fscache ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic virtio_net crc32c_intel ata_piix libata net_failover serio_raw virtio_blk failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c | Link to test |
racer test 2: racer rename: onyx-76vm1.onyx.whamcloud.com,onyx-76vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 814297 Comm: ll_sa_814150 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a 63 74 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb41e41ae7e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000003 RDX: ffff997f3fc38160 RSI: ffff997f26c10370 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff997e81003180 R09: ffffd7ea400fda00 R10: 0000000000000000 R11: 000000000000000f R12: ffff997f26c10090 R13: ffff997eb1b5a698 R14: ffff997eb1b5a600 R15: ffff997eb1b5a6a8 FS: 0000000000000000(0000) GS:ffff997f3fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000088810005 CR4: 00000000001706f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? apic_timer_interrupt+0xa/0x20 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul dns_resolver nfs lockd grace fscache ghash_clmulni_intel joydev pcspkr i2c_piix4 virtio_balloon sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c | Link to test |
racer test 2: racer rename: onyx-59vm4.onyx.whamcloud.com,onyx-59vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 477969 Comm: ll_sa_477873 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.27.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 6a 03 48 5b c3 cc cc cc cc 48 89 df e8 95 15 af ff 39 05 c3 8e 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa71685487e10 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000003 RDX: ffff91c8bfc38160 RSI: ffff91c86e5d0370 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff91c801003180 R09: fffff5ee40f4c280 R10: 0000000000000000 R11: 000000000000000f R12: ffff91c86e5d0090 R13: ffff91c8512dc098 R14: ffff91c8512dc000 R15: ffff91c8512dc0a8 FS: 0000000000000000(0000) GS:ffff91c8bfc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000009ac10005 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x6c4/0x2210 [lustre] ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw net_failover virtio_blk failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x280000417:0x8d1:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x280000416:0xf21:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages Lustre: dir [0x2c0000416:0xdb9:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: dir [0x200000419:0x11b3:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x280000418:0x137a:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 2 previous similar messages Lustre: dir [0x2c0000414:0x107f:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages | Link to test |
racer test 2: racer rename: onyx-77vm1.onyx.whamcloud.com,onyx-77vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 728580 Comm: ll_sa_728442 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 83 6e 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb1d986dcfe08 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000009 RDX: ffff9f997cc38160 RSI: ffff9f98fbfc2170 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff9f98c1003180 R09: ffffee23c25a0780 R10: 0000000000000000 R11: 000000000000000f R12: ffff9f9959358200 R13: ffff9f9959358298 R14: ffff9f98fbfc1e90 R15: ffff9f99593582a8 FS: 0000000000000000(0000) GS:ffff9f997cc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 00000000b9c10001 CR4: 00000000001706f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul nfsv4 dns_resolver nfs lockd grace fscache ghash_clmulni_intel i2c_piix4 joydev pcspkr virtio_balloon sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw net_failover virtio_blk failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 583389:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000bee:0x15dd:0x0]: rc = -5 LustreError: 583389:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 68 previous similar messages LustreError: 583389:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 583389:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 68 previous similar messages Lustre: dir [0x240000beb:0x167d:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Autotest: Test running for 220 minutes (lustre-reviews_review-dne-part-9_109407.11) | Link to test |
racer test 2: racer rename: onyx-122vm7,onyx-80vm10.onyx.whamcloud.com DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 931931 Comm: ll_sa_931799 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 c3 59 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffff98164190fe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000004 RDX: ffff8c0939738160 RSI: ffff8c08d100ad70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff8c0881003180 R09: ffffc7e940d8c380 R10: 0000000000000000 R11: 000000000000000f R12: ffff8c08e7e16e00 R13: ffff8c08e7e16e98 R14: ffff8c08d100aa90 R15: ffff8c08e7e16ea8 FS: 0000000000000000(0000) GS:ffff8c0939700000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 00000000bb210003 CR4: 00000000001706e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel net_failover virtio_blk serio_raw failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 861764:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000bea:0xa79:0x0]: rc = -5 LustreError: 861764:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 24 previous similar messages LustreError: 861764:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 861764:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 24 previous similar messages Lustre: dir [0x2c0000be7:0x8c6:0x0] stripe 0 readdir failed: -2, directory is partially accessed! LustreError: 146165:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for 9 [0x240000be7:0x896:0x0]: rc = -5 Lustre: dir [0x2c0000bea:0xf31:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages | Link to test |
racer test 2: racer rename: trevis-106vm7.trevis.whamcloud.com,trevis-106vm8 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1296891 Comm: ll_sa_1296729 Kdump: loaded Tainted: G W OE --------- - - 4.18.0-477.27.1.el8_8.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 b4 e5 75 5b e9 1d a2 42 00 48 89 df e8 b5 98 b0 ff 39 05 83 dc 5d 01 77 e3 5b e9 07 a2 42 00 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d e9 e5 RSP: 0018:ffffa3f488a77e10 EFLAGS: 00010216 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100005 RDX: 0000000000100006 RSI: ffff94252d790370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000000 R09: ffffffffc1045c00 R10: ffff9425020f3000 R11: 0000000000000001 R12: ffff94252d790090 R13: ffff9425020f3098 R14: ffff9425020f3000 R15: ffff9425020f30a8 FS: 0000000000000000(0000) GS:ffff94257fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000055810003 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ll_statahead_thread+0x6c4/0x2210 [lustre] ? __raw_spin_unlock_irq+0x5/0x20 ? finish_task_switch+0x86/0x2e0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) ib_core tcp_diag inet_diag loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common nfs crct10dif_pclmul lockd grace fscache crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net net_failover virtio_blk serio_raw failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x200006997:0x1296:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 119 previous similar messages | Link to test |
racer test 2: racer rename: onyx-141vm4.onyx.whamcloud.com,onyx-141vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 1051378 Comm: ll_sa_1051249 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 63 70 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffbccf0895fe08 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000007 RDX: ffff97fd7bd38160 RSI: ffff97fd76321b70 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff97fd40003180 R09: ffffe91f84b4aa00 R10: 0000000000000000 R11: 000000000000000f R12: ffff97fd7a5fd800 R13: ffff97fd7a5fd898 R14: ffff97fd76321890 R15: ffff97fd7a5fd8a8 FS: 0000000000000000(0000) GS:ffff97fd7bd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 00000000b8e10006 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net net_failover virtio_blk failover serio_raw [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 938910:0:(lov_object.c:1341:lov_layout_change()) lustre-clilov-ffff97fd6e105800: cannot apply new layout on [0x200000beb:0xc32:0x0] : rc = -5 LustreError: 938910:0:(lov_object.c:1341:lov_layout_change()) Skipped 3 previous similar messages Lustre: dir [0x200000beb:0x1170:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 26 previous similar messages Autotest: Test running for 215 minutes (lustre-reviews_review-dne-part-9_109245.11) Lustre: dir [0x200000bed:0x1c32:0x0] stripe 2 readdir failed: -2, directory is partially accessed! | Link to test |
racer test 2: racer rename: onyx-136vm1.onyx.whamcloud.com,onyx-136vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 845582 Comm: ll_sa_845413 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 a3 69 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa9a1c62b7e08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff8feffbf08370 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff8fef87787600 R11: 0000000000000000 R12: ffff8fef87787600 R13: ffff8fef87787698 R14: ffff8feffbf08090 R15: ffff8fef877876a8 FS: 0000000000000000(0000) GS:ffff8ff0bbd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000065810004 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev i2c_piix4 pcspkr virtio_balloon sunrpc ext4 mbcache jbd2 ata_generic ata_piix crc32c_intel libata serio_raw virtio_net virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x200000bec:0x203d:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 8 previous similar messages Lustre: dir [0x280000be7:0x23a8:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x280000be8:0x3252:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 12 previous similar messages | Link to test |
racer test 2: racer rename: trevis-66vm1.trevis.whamcloud.com,trevis-66vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 627623 Comm: ll_sa_627489 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 43 6b 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffff974684f97e08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100009 RDX: 000000000010000a RSI: ffff888e67757570 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff888e64701600 R11: 0000000000000000 R12: ffff888e64701600 R13: ffff888e64701698 R14: ffff888e67757290 R15: ffff888e647016a8 FS: 0000000000000000(0000) GS:ffff888e7fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000002da10002 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon joydev pcspkr i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_blk serio_raw virtio_net net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 595113:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000be9:0x4f8:0x0]: rc = -5 LustreError: 595113:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 6 previous similar messages LustreError: 595113:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 595113:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 6 previous similar messages Lustre: dir [0x280000be6:0xa46:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages LustreError: 334345:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for sleep [0x280000be9:0x201:0x0]: rc = -5 LustreError: 334345:0:(statahead.c:830:ll_statahead_interpret_work()) lustre: getattr callback for sleep [0x280000be9:0x201:0x0]: rc = -5 | Link to test |
racer test 2: racer rename: trevis-81vm1.trevis.whamcloud.com,trevis-81vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1122675 Comm: ll_sa_1122517 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 63 43 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffbbaac69cfe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100004 RDX: 0000000000100005 RSI: ffff9c5f320ee970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9c5f9ee4e600 R11: 0000000000000000 R12: ffff9c5f9ee4e600 R13: ffff9c5f9ee4e698 R14: ffff9c5f320ee690 R15: ffff9c5f9ee4e6a8 FS: 0000000000000000(0000) GS:ffff9c5fbfc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000086210004 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr sunrpc joydev virtio_balloon i2c_piix4 ext4 mbcache jbd2 ata_generic crc32c_intel ata_piix libata virtio_net serio_raw net_failover failover virtio_blk CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x280000be6:0xa13:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 15 previous similar messages Lustre: dir [0x200000bf1:0xc37:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 4 previous similar messages Lustre: dir [0x240000be9:0x1df9:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x200000bee:0x1ef3:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x2c0000beb:0x1fd9:0x0] stripe 2 readdir failed: -2, directory is partially accessed! LustreError: 1010152:0:(llite_nfs.c:446:ll_dir_get_parent_fid()) lustre: failure inode [0x200000bee:0x240e:0x0] get parent: rc = -116 Autotest: Test running for 260 minutes (lustre-reviews_review-dne-part-9_109212.11) Lustre: dir [0x240000beb:0x380a:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages | Link to test |
racer test 2: racer rename: trevis-107vm1.trevis.whamcloud.com,trevis-107vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1022337 Comm: ll_sa_1022183 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 23 5a 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb3fe4283be08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000005 RDX: ffff8a5b7fc38160 RSI: ffff8a5b57908970 RDI: 0000000000000008 RBP: 0000000000000008 R08: ffff8a5ac1003180 R09: fffffc2a804fca80 R10: 0000000000000000 R11: 000000000000000f R12: ffff8a5ac3f09800 R13: ffff8a5ac3f09898 R14: ffff8a5b57908690 R15: ffff8a5ac3f098a8 FS: 0000000000000000(0000) GS:ffff8a5b7fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000009c610002 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ata_generic ext4 mbcache jbd2 ata_piix libata crc32c_intel virtio_net virtio_blk serio_raw net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 1015636:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x2c0000be6:0x2d5:0x0]: rc = -5 LustreError: 1015636:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 52 previous similar messages LustreError: 1015636:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 1015636:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 52 previous similar messages | Link to test |
racer test 2: racer rename: trevis-107vm11.trevis.whamcloud.com,trevis-107vm12 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 4154045 Comm: ll_sa_4153892 Kdump: loaded Tainted: G OE --------- - - 4.18.0-477.27.1.el8_8.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 b4 05 53 5b e9 1d a2 42 00 48 89 df e8 b5 98 b0 ff 39 05 83 dc 5d 01 77 e3 5b e9 07 a2 42 00 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d e9 e5 RSP: 0018:ffffb19fc248be08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100007 RDX: 0000000000100008 RSI: ffff964147187b70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000000 R09: ffffffffc0f74400 R10: ffff9640ead70400 R11: 0000000000000001 R12: ffff9640ead70400 R13: ffff9640ead70498 R14: ffff964147187890 R15: ffff9640ead704a8 FS: 0000000000000000(0000) GS:ffff96417fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000004a610005 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ll_statahead_thread+0x658/0x2120 [lustre] ? __raw_spin_unlock_irq+0x5/0x20 ? finish_task_switch+0x86/0x2e0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: loop ib_core nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr nfs lockd grace fscache intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net virtio_blk serio_raw net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x280006994:0x2fa6:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 60 previous similar messages Autotest: Test running for 1020 minutes (lustre-master_full-dne-zfs-part-2_4592.11) Lustre: dir [0x200007167:0x5f9c:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 24 previous similar messages | Link to test |
racer test 2: racer rename: trevis-129vm1.trevis.whamcloud.com,trevis-129vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 911076 Comm: ll_sa_910915 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 43 4e 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb79002f7fe08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100003 RDX: 0000000000100004 RSI: ffff9c35f3106970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9c36631bc600 R11: 0000000000000000 R12: ffff9c36631bc600 R13: ffff9c36631bc698 R14: ffff9c35f3106690 R15: ffff9c36631bc6a8 FS: 0000000000000000(0000) GS:ffff9c367fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000063e10001 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_blk virtio_net net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 756188:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x280000be8:0x13b:0x0]: rc = -5 LustreError: 756188:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 6 previous similar messages LustreError: 756188:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 756188:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 6 previous similar messages Lustre: dir [0x280000be9:0x8f1:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 51 previous similar messages Lustre: dir [0x280000be6:0xbe8:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages Lustre: dir [0x2c0000be6:0xdde:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x280000be9:0x1972:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 6 previous similar messages | Link to test |
racer test 2: racer rename: onyx-24vm4.onyx.whamcloud.com,onyx-24vm5 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 655209 Comm: ll_sa_655096 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 63 4e 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffc061c6017e08 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100007 RDX: 0000000000100008 RSI: ffff9a75bdc6bf70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff9a75a3f86200 R11: 0000000000000000 R12: ffff9a75a3f86200 R13: ffff9a75a3f86298 R14: ffff9a75bdc6bc90 R15: ffff9a75a3f862a8 FS: 0000000000000000(0000) GS:ffff9a75ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000006b810003 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core lustre(OE) mgc(OE) mdc(OE) lov(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) osc(OE) ksocklnd(OE) ptlrpc(OE) obdecho(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel serio_raw net_failover failover virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 596266:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x200000be9:0x506:0x0]: rc = -5 LustreError: 596266:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 1 previous similar message LustreError: 596266:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 596266:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 1 previous similar message Lustre: dir [0x2c0000bea:0x6bc:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages | Link to test |
racer test 2: racer rename: trevis-47vm1.trevis.whamcloud.com,trevis-47vm2 DURATION=300 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 861218 Comm: ll_sa_861067 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 a3 64 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa211c44fbe08 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff94863f52b970 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000001 R10: ffff9486313d1200 R11: 0000000000000000 R12: ffff9486313d1200 R13: ffff9486313d1298 R14: ffff94863f52b690 R15: ffff9486313d12a8 FS: 0000000000000000(0000) GS:ffff94867fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 00000000bd210006 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl ib_core mgc(OE) lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel sunrpc pcspkr joydev virtio_balloon i2c_piix4 ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel serio_raw virtio_net virtio_blk net_failover failover CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=300 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 604881:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x240000be8:0xe8:0x0]: rc = -5 LustreError: 604881:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 56 previous similar messages LustreError: 604881:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 604881:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 56 previous similar messages Lustre: dir [0x200000bed:0xb31:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 3 previous similar messages Lustre: dir [0x2c0000bea:0x587:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 1 previous similar message Lustre: dir [0x240000be6:0x2326:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 8 previous similar messages Autotest: Test running for 260 minutes (lustre-reviews_review-dne-part-9_109135.29) | Link to test |
racer test 2: racer rename: trevis-24vm4.trevis.whamcloud.com,trevis-24vm5 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 747982 Comm: ll_sa_747826 Kdump: loaded Tainted: G W OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 43 60 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffac9b41c73e08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000010000c RDX: 000000000010000d RSI: ffff97fc9fd13f70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff97fc8310fc00 R11: 0000000000000000 R12: ffff97fc8310fc00 R13: ffff97fc8310fc98 R14: ffff97fc9fd13c90 R15: ffff97fc8310fca8 FS: 0000000000000000(0000) GS:ffff97fcbfd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000007f410005 CR4: 00000000000606e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) ib_core tcp_diag inet_diag loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr virtio_balloon joydev i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic virtio_net crc32c_intel ata_piix net_failover failover libata serio_raw virtio_blk [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x2c0006997:0x161e:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 38 previous similar messages Autotest: Test running for 1230 minutes (lustre-master-next_full-dne-zfs-part-2_839.35) Lustre: dir [0x2c0006997:0x4273:0x0] stripe 2 readdir failed: -2, directory is partially accessed! Lustre: Skipped 25 previous similar messages | Link to test |
racer test 2: racer rename: onyx-143vm7.onyx.whamcloud.com,onyx-143vm8 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 768693 Comm: ll_sa_768585 Kdump: loaded Tainted: G W OE --------- - - 4.18.0-513.24.1.el8_9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 73 78 84 50 5b c3 cc cc cc cc 48 89 df e8 05 d7 af ff 39 05 03 8c 5d 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffa96047a63e08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff8bbd05fbfb70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: ffffffffc10eb100 R10: ffff8bbe1cfa0000 R11: 0000000000000001 R12: ffff8bbe1cfa0000 R13: ffff8bbe1cfa0098 R14: ffff8bbd05fbf890 R15: ffff8bbe1cfa00a8 FS: 0000000000000000(0000) GS:ffff8bbe3bd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000022210004 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x16c/0x1c0 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? ll_sai_put+0xa0/0x2f0 [lustre] ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) ib_core tcp_diag inet_diag loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_balloon i2c_piix4 pcspkr joydev sunrpc ext4 mbcache jbd2 ata_generic crc32c_intel ata_piix libata serio_raw virtio_net virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x240007932:0x20dc:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 120 previous similar messages | Link to test |
racer test 2: racer rename: onyx-75vm1.onyx.whamcloud.com,onyx-75vm2 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1376224 Comm: ll_sa_1376108 Kdump: loaded Tainted: G W OE --------- - - 4.18.0-477.27.1.el8_8.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 b3 b4 25 4a 5b e9 1d a2 42 00 48 89 df e8 b5 98 b0 ff 39 05 83 dc 5d 01 77 e3 5b e9 07 a2 42 00 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d e9 e5 RSP: 0018:ffff9953c5af7e08 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100006 RDX: 0000000000100007 RSI: ffff88e9e3555770 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000000 R09: ffffffffc0d8f400 R10: ffff88e9940aae00 R11: 0000000000000001 R12: ffff88e9940aae00 R13: ffff88e9940aae98 R14: ffff88e9e3555490 R15: ffff88e9940aaea8 FS: 0000000000000000(0000) GS:ffff88e9ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000012410001 CR4: 00000000001706f0 Call Trace: ll_statahead_thread+0x658/0x2120 [lustre] ? __raw_spin_unlock_irq+0x5/0x20 ? finish_task_switch+0x86/0x2e0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) ib_core tcp_diag inet_diag loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs intel_rapl_msr lockd grace fscache intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr joydev virtio_balloon i2c_piix4 sunrpc ata_generic ext4 mbcache jbd2 ata_piix libata virtio_net crc32c_intel serio_raw virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x2c0006990:0x1f0d:0x0] stripe 0 readdir failed: -2, directory is partially accessed! Lustre: Skipped 10 previous similar messages Autotest: Test running for 1040 minutes (lustre-master-next_full-dne-zfs-part-2_839.11) Lustre: dir [0x2c0006998:0x4f3f:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 9 previous similar messages Autotest: Test running for 1045 minutes (lustre-master-next_full-dne-zfs-part-2_839.11) Autotest: Test running for 1050 minutes (lustre-master-next_full-dne-zfs-part-2_839.11) | Link to test |
racer test 2: racer rename: onyx-25vm1.onyx.whamcloud.com,onyx-25vm2 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 534928 Comm: ll_sa_534809 Kdump: loaded Tainted: G W OE -------- - - 4.18.0-553.16.1.el8_10.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 03 77 03 4c 5b c3 cc cc cc cc 48 89 df e8 25 22 af ff 39 05 13 a1 5c 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb1ec88a47e08 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100008 RDX: 0000000000100009 RSI: ffff91fce02ddd70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: 0000000000000000 R10: ffff91fc94651e00 R11: 0000000000000000 R12: ffff91fc94651e00 R13: ffff91fc94651e98 R14: ffff91fce02dda90 R15: ffff91fc94651ea8 FS: 0000000000000000(0000) GS:ffff91fcffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000057410006 CR4: 00000000000606f0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x157/0x180 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) ib_core tcp_diag inet_diag loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw net_failover virtio_blk failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c LustreError: 272347:0:(lcommon_cl.c:195:cl_file_inode_init()) lustre: failed to initialize cl_object [0x20000716a:0x1ef:0x0]: rc = -5 LustreError: 272347:0:(lcommon_cl.c:195:cl_file_inode_init()) Skipped 106 previous similar messages LustreError: 272347:0:(llite_lib.c:3731:ll_prep_inode()) lustre: new_inode - fatal error: rc = -5 LustreError: 272347:0:(llite_lib.c:3731:ll_prep_inode()) Skipped 106 previous similar messages Autotest: Test running for 1080 minutes (lustre-master-next_full-dne-zfs-part-2_838.35) Lustre: dir [0x200007168:0x32fa:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 11 previous similar messages | Link to test |
racer test 2: racer rename: onyx-76vm1.onyx.whamcloud.com,onyx-76vm2 DURATION=900 | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 826445 Comm: ll_sa_826294 Kdump: loaded Tainted: G W OE --------- - - 4.18.0-513.24.1.el8_9.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 73 78 24 47 5b c3 cc cc cc cc 48 89 df e8 05 d7 af ff 39 05 03 8c 5d 01 77 e3 5b c3 cc cc cc cc 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d c3 cc RSP: 0018:ffffb711424bbe08 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000100007 RDX: 0000000000100008 RSI: ffff8b371291ad70 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000000 R09: ffffffffc1087400 R10: ffff8b370cc52c00 R11: 0000000000000001 R12: ffff8b370cc52c00 R13: ffff8b370cc52c98 R14: ffff8b371291aa90 R15: ffff8b370cc52ca8 FS: 0000000000000000(0000) GS:ffff8b377fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 0000000060010003 CR4: 00000000001706e0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x1ba/0x3f0 ? __bad_area_nosemaphore+0x16c/0x1c0 ? do_page_fault+0x37/0x12d ? page_fault+0x1e/0x30 ? ll_sai_put+0x120/0x2f0 [lustre] ? _atomic_dec_and_lock+0x2/0x50 ll_statahead_thread+0x658/0x2120 [lustre] ? __switch_to_asm+0x43/0x80 ? finish_task_switch+0x86/0x2f0 ? __schedule+0x2d9/0x870 ? ll_statahead_handle.constprop.30+0x170/0x170 [lustre] kthread+0x134/0x150 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsv3 nfsd nfs_acl lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) ib_core tcp_diag inet_diag loop rpcsec_gss_krb5 auth_rpcgss intel_rapl_msr nfsv4 intel_rapl_common dns_resolver crct10dif_pclmul nfs lockd grace crc32_pclmul fscache ghash_clmulni_intel joydev pcspkr virtio_balloon i2c_piix4 sunrpc ext4 mbcache jbd2 ata_generic ata_piix libata crc32c_intel virtio_net serio_raw virtio_blk net_failover failover [last unloaded: libcfs] CR2: 0000000000000008 | Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: DEBUG MARKER: DURATION=900 MDSCOUNT=4 OSTCOUNT=8 RACER_MAX_MB=0 RACER_ENABLE_FLR=false RACER_ENABLE_DOM=false RACER_ENABLE_SEL=false RACER_ENABLE_MIGRATION=false RACER_MAX_CLEANUP_WAIT= RACER_EXTRA="" RACER_EXTRA_LAYOUT="" RACER_PROGS=dir_c Lustre: dir [0x2c0006996:0x3789:0x0] stripe 1 readdir failed: -2, directory is partially accessed! Lustre: Skipped 218 previous similar messages LustreError: lustre-MDT0000-mdc-ffff8b375dcfd800: operation ldlm_enqueue to node 10.240.26.106@tcp failed: rc = -107 Lustre: lustre-MDT0000-mdc-ffff8b375dcfd800: Connection to lustre-MDT0000 (at 10.240.26.106@tcp) was lost; in progress operations using this service will wait for recovery to complete LustreError: lustre-MDT0000-mdc-ffff8b375dcfd800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. LustreError: 448364:0:(file.c:5973:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200007162:0x11:0x0] error: rc = -108 Lustre: lustre-MDT0000-mdc-ffff8b375dcfd800: Connection restored to 10.240.26.106@tcp (at 10.240.26.106@tcp) Autotest: Test running for 875 minutes (lustre-master-next_full-dne-zfs-part-2_838.23) | Link to test |
parallel-scale-nfsv3 test racer_on_nfs: racer on NFS client | BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 0 P4D 0 Oops: 0000 [#1] SMP PTI CPU: 1 PID: 40515 Comm: ll_sa_21331 Kdump: loaded Tainted: G OE --------- - - 4.18.0-425.10.1.el8_lustre.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:_atomic_dec_and_lock+0x2/0x50 Code: 05 93 34 86 55 5b e9 3d a2 23 00 48 89 df e8 f5 f1 b0 ff 39 05 23 a6 3e 01 77 e3 5b e9 27 a2 23 00 90 90 90 90 90 90 90 55 53 <8b> 07 83 f8 01 74 12 8d 50 ff f0 0f b1 17 75 f2 31 c0 5b 5d e9 05 RSP: 0018:ffffb5fbc0bafe10 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000008010000e RDX: 000000008010000f RSI: ffff88be622ca770 RDI: 0000000000000008 RBP: 0000000000000008 R08: 0000000000000001 R09: ffffffffc11c3c00 R10: ffff88bec475a400 R11: 0000000000000001 R12: ffff88bec475a400 R13: ffff88bec475a4a8 R14: ffff88be622ca490 R15: ffff88bec475a498 FS: 0000000000000000(0000) GS:ffff88beffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000003c810002 CR4: 00000000000606e0 Call Trace: ll_statahead_thread+0x66c/0x2090 [lustre] ? __raw_spin_unlock_irq+0x5/0x20 ? finish_task_switch+0xaf/0x2e0 ? __schedule+0x2d9/0x860 ? ll_statahead_handle.constprop.31+0x170/0x170 [lustre] kthread+0x10b/0x130 ? set_kthread_struct+0x50/0x50 ret_from_fork+0x35/0x40 Modules linked in: nfsd nfs_acl osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) lmv(OE) mdc(OE) lov(OE) osc(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) dm_flakey rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel joydev virtio_balloon pcspkr i2c_piix4 sunrpc dm_mod ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel serio_raw net_failover failover virtio_blk CR2: 0000000000000008 | Link to test |