Editing crashreport #73796

ReasonCrashing FunctionWhere to cut BacktraceReports Count
BUG: unable to handle kernel NULL pointer dereference__wake_up_common__wake_up_common_lock
tcp_child_process
tcp_v4_rcv
ip_protocol_deliver_rcu
ip_local_deliver_finish
ip_local_deliver
ip_rcv
__netif_receive_skb_core
netif_receive_skb_internal
napi_gro_receive
receive_buf
virtnet_poll
__napi_poll
net_rx_action
__do_softirq
irq_exit_rcu
irq_exit
do_IRQ
common_interrupt
24

Added fields:

Match messages in logs
(every line would be required to be present in log output
Copy from "Messages before crash" column below):
Match messages in full crash
(every line would be required to be present in crash log output
Copy from "Full Crash" column below):
Limit to a test:
(Copy from below "Failing text"):
Delete these reports as invalid (real bug in review or some such)
Bug or comment:
Extra info:

Failures list (last 100):

Failing TestFull CrashMessages before crashComment
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.89.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff64da4940003a18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0822380 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0822378
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff64da4940003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff1311ff3fa00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000017aa10002 CR4: 0000000000771ef0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
LustreError: 781103:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fef8d31800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 781103:0:(lov_obd.c:783:lov_cleanup()) Skipped 151 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 781246:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 781246:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 781510:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
LustreError: 781457:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff1311ff04a9c000: import has been deactivated: rc = -22
LustreError: 781459:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1311ff04a9c000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 781628:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 781628:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 781890:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311ff04a9e800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 781890:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 782008:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 782008:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 782260:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec7fe7800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 782211:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec7fe7800: import has been deactivated: rc = -22
LustreError: 782260:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 782379:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 782379:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 782643:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec2111000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 782592:0:(gss_cli_upcall.c:255:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec2111000: obd has stopped: rc = -22
LustreError: 782643:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
LustreError: 782590:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1311fec2111000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 782762:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 782762:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 783022:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec7772000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 782973:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec7772000: import has been deactivated: rc = -22
LustreError: 783022:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 783139:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 783139:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 783347:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec7770800: import has been deactivated: rc = -22
LustreError: 783396:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 783513:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 783513:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 783776:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec41c0800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 783776:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 783892:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 783892:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 784091:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311ff39e1d000: import has been deactivated: rc = -22
LustreError: 784140:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
LustreError: 784089:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1311ff39e1d000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 784258:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 784258:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 784522:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec7777800: lov tgt 1 not cleaned! deathrow=1, lovrc=1
LustreError: 784522:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
LustreError: 784469:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1311fec7777800: import has been deactivated: rc = -22
LustreError: 784471:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec7777800: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 784640:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 784640:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 784895:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
LustreError: 784842:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff1311fec7fe0000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
LustreError: 784836:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0002-osc-ff1311fec7fe0000: no such obd: rc = -22
Key type lgssc unregistered
LNet: 785014:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 785014:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 785274:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec7777800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 785225:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec7777800: import has been deactivated: rc = -22
LustreError: 785274:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 785393:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 785393:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 785655:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec4017800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 785606:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fec4017800: import has been deactivated: rc = -22
LustreError: 785655:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 785772:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 785772:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 786033:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fec7773000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 786033:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
LustreError: 785981:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1311fec7773000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 786149:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786149:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 786404:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1311fed3d94800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 786355:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1311fed3d94800: import has been deactivated: rc = -22
LustreError: 786404:0:(obd_class.h:481:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 786522:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786522:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.236@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_93_g711650b
LNet: Added LNI 10.240.27.236@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.89.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff1e7e1c80003a18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0897380 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0897378
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff1e7e1c80003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff1599bd3ba00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000116a10001 CR4: 0000000000771ef0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 785845:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 785845:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 786123:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 786068:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0004-osc-ff1599bccfcdb000: import has been deactivated: rc = -22
LustreError: 786072:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1599bccfcdb000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 786241:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786241:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 786489:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1599bcf895b800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 786440:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1599bcf895b800: import has been deactivated: rc = -22
LustreError: 786489:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 786607:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786607:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 786867:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1599bcc7ba6800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 786818:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1599bcc7ba6800: import has been deactivated: rc = -22
LustreError: 786867:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 786986:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786986:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 787243:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1599bd136d0800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 787193:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1599bd136d0800: import has been deactivated: rc = -22
LustreError: 787243:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 787362:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 787362:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 787618:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1599bcc7a0f800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 787569:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1599bcc7a0f800: import has been deactivated: rc = -22
LustreError: 787618:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 787737:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 787737:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 787999:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 787948:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff1599bcc4413000: import has been deactivated: rc = -22
LustreError: 787936:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0000-osc-ff1599bcc4413000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 788118:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 788118:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 788369:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff1599bcc2e20000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 788369:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 788316:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff1599bcc2e20000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 788486:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 788486:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 788701:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1599bcd8ce9000: import has been deactivated: rc = -22
LustreError: 788750:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 788868:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 788868:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 789076:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff1599bcc7ba0000: import has been deactivated: rc = -22
LustreError: 789127:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 789246:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 789246:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.31.55@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.50_86_gaa23913
LNet: Added LNI 10.240.31.55@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.85.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff60483840003a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc09d8300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc09d82f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff60483840003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff38bb3ffba00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000158c10003 CR4: 0000000000771ef0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? srso_alias_return_thunk+0x5/0xfcdfd
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
LustreError: 764103:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup
LustreError: 764103:0:(obd_class.h:479:obd_check_dev()) Skipped 441 previous similar messages
Lustre: Unmounted lustre-client
Lustre: Skipped 16 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 764247:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 764247:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 764475:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3fbab10000: import has been deactivated: rc = -22
LustreError: 764524:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 764643:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 764643:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 764909:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff38bb3f8558e800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 764859:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3f8558e800: import has been deactivated: rc = -22
LustreError: 764909:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 765028:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 765028:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 765292:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff38bb3fda4cd800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 765292:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 765241:0:(gss_cli_upcall.c:255:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3fda4cd800: obd has stopped: rc = -22
LustreError: 765239:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff38bb3fda4cd800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 765411:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 765411:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
Autotest: Test running for 215 minutes (lustre-reviews_review-dne-selinux-ssk-part-2_120480.63)
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 765730:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff38bb3f83d6f000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 765673:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3f83d6f000: import has been deactivated: rc = -22
LustreError: 765730:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 765849:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 765849:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 766086:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff38bb3f8403e800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 766037:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3f8403e800: import has been deactivated: rc = -22
LustreError: 766086:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 766205:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 766205:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 766410:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3f85588000: import has been deactivated: rc = -22
LustreError: 766459:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 766578:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 766578:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 766839:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff38bb3fda4cc800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 766839:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 766958:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 766958:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 767170:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff38bb3f8558f800: import has been deactivated: rc = -22
LustreError: 767219:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 767168:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff38bb3f8558f800: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 767337:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 767337:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.44.174@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_9_g884d93f
LNet: Added LNI 10.240.44.174@tcp [8/256/0/180]
Link to test
Module load
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 102e53067 P4D 0
Oops: 0000 [#1] SMP NOPTI
CPU: 1 PID: 488 Comm: systemd-journal Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.89.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff557f03006f8a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0b1a300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0b1a2f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff557f03006f8a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f34827a0980(0000) GS:ff31382e3bb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000102ec4003 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? srso_alias_return_thunk+0x5/0xfcdfd
? update_load_avg+0x7e/0x710
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? enqueue_entity+0x102/0x6b0
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:0xffffffffc028d911
libcfs: loading out-of-tree module taints kernel.
libcfs: module verification failed: signature and/or required key missing - tainting kernel
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost5; mount -t lustre -o localrecov /dev/mapper/ost5_flakey /mnt/lustre-ost5
Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost3; mount -t lustre -o localrecov /dev/mapper/ost3_flakey /mnt/lustre-ost3
Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost6; mount -t lustre -o localrecov /dev/mapper/ost6_flakey /mnt/lustre-ost6
Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost1; mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1
Lustre: Lustre: Build Version: 2.17.50_28_g0f6e401
LNet: Added LNI 10.240.47.238@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff2aac04c06f8a18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0b26340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0b26338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff2aac04c06f8a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff23cf367bb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000153010002 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? srso_alias_return_thunk+0x5/0xfcdfd
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 809059:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 809059:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 809389:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf36234b7000: obd has stopped: rc = -22
LustreError: 809387:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff23cf36234b7000: import has been deactivated: rc = -22
LustreError: 809379:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0002-osc-ff23cf36234b7000: import has been deactivated: rc = -22
LustreError: 809379:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) Skipped 3 previous similar messages
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 809558:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 809558:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 809841:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf363c1f2000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 810009:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 810009:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 810309:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf361ec49800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 810478:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 810478:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 810772:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf360349f800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 810941:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 810941:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 811224:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf360349a000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 811392:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 811392:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 811690:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf36234b3800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 811858:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 811858:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 812156:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf3605244800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 812324:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 812324:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 812626:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf36525c6800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 812794:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 812794:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 813097:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf3605244000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 813269:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 813269:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 813561:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf361ec48000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 813729:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 813729:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 814032:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff23cf360349f000: import has been deactivated: rc = -22
LustreError: 814032:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) Skipped 1 previous similar message
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 814200:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 814200:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.134@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn239_1_gb0d51af
LNet: Added LNI 10.240.39.134@tcp [8/256/0/180]
Link to test
sanity-lnet test 221: Add routes w/DD disabled - check aliveness
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 2 Comm: kthreadd Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.89.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffff9e09006f4a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0bdc300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0bdc2f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffff9e09006f4a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8d0f3bd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000067a10006 CR4: 0000000000370ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? kvm_sched_clock_read+0xd/0x20
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:ktime_get_with_offset+0xa1/0xc0
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 1048011:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1048011:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.25.201@tcp1
LNetError: 1048011:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNetError: 1048011:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Skipped 1 previous similar message
LNet: Removed LNI 10.240.25.201@tcp2
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
Lustre: DEBUG MARKER: /sbin/lctl mark onyx-140vm10.onyx.whamcloud.com: executing load_lnet config_on_load=1
Lustre: DEBUG MARKER: /sbin/lctl mark onyx-140vm10.onyx.whamcloud.com: executing load_lnet config_on_load=1
Lustre: DEBUG MARKER: /usr/sbin/lctl lustre_build_version 2>/dev/null
Lustre: DEBUG MARKER: onyx-140vm10.onyx.whamcloud.com: executing load_lnet config_on_load=1
Lustre: DEBUG MARKER: onyx-140vm10.onyx.whamcloud.com: executing load_lnet config_on_load=1
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /sbin/lnetctl lnet configure -a
LNet: Added LNI 10.240.25.201@tcp [8/256/0/180]
Link to test
sanity-lnet test 100: Add route with single gw
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.89.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff4ff5e880003a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc095f300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc095f2f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ff4f93758851dc00 R11: 0000000000000000 R12: ff4ff5e880003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff4f9375fba00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000005ea10005 CR4: 0000000000771ef0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? srso_alias_return_thunk+0x5/0xfcdfd
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? srso_alias_return_thunk+0x5/0xfcdfd
? update_load_avg+0x7e/0x710
? srso_alias_return_thunk+0x5/0xfcdfd
? enqueue_task_fair+0xeb/0x3a0
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type lgssc unregistered
LNet: 771351:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 771351:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 773018:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 773018:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 775480:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 775480:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 777719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 777719:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 779981:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 779981:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 782465:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 782465:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 784727:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 784727:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 786722:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786722:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 788984:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 788984:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 791958:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 791958:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 794220:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 794220:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Autotest: Test running for 195 minutes (lustre-reviews_review-ldiskfs_120091.56)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 807592:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 807592:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 809750:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 808662:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 808662:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from up to down
LNetError: 808662:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 808662:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 808662:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 810836:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 811430:0:(timer.c:222:stt_shutdown()) waiting for 1 threads to terminate
LNet: 811430:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 811430:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 813423:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 813423:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 815580:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 815580:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 814492:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 814492:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from up to down
LNetError: 814492:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 814492:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
Autotest: Test running for 200 minutes (lustre-reviews_review-ldiskfs_120091.56)
LNetError: 814492:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 816218:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 816218:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 818212:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 818212:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 820369:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.90@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 820465:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.90@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 820561:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 819281:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 821329:0:(lib-move.c:2362:lnet_handle_find_routed_path()) no route to 10.240.42.90@tcp from 10.240.42.92@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 821531:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 821531:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 823525:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 823525:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 825682:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.90@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 825778:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.90@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 825874:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 825874:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 824594:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 824594:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
LNetError: 824594:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
LNetError: 824594:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
LNetError: 827218:0:(lib-move.c:2362:lnet_handle_find_routed_path()) no route to 10.240.42.90@tcp from 10.240.42.92@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 827420:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 827420:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 829415:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 829415:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 832446:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 832446:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 834440:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 834440:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 836597:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 837471:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 837471:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 839854:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 839854:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 842910:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 842910:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 842910:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 843013:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 843013:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 845199:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 845199:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 847549:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 847549:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 846365:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 846365:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from up to down
LNetError: 846365:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 846365:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 846365:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
LNet: 848029:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 848029:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 850512:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 850512:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Autotest: Test running for 205 minutes (lustre-reviews_review-ldiskfs_120091.56)
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 853620:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 853620:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 853620:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 853915:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 853915:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 857859:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 857859:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 860016:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 858928:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 858928:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from up to down
LNetError: 858928:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 858928:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 858928:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 858928:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp3 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp3 --if eth0
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 860698:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 860698:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 864493:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 864493:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.91@tcp1
LNet: 866651:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 865563:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 865563:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from up to down
LNetError: 865563:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 865563:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 865563:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.91@tcp is being used as a gateway but routing feature is not turned on
LNetError: 865563:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.91@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.90@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.91@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.91@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.91@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 867237:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 867237:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 869617:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 869617:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm50.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm50.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.92@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm49.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm49.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 872673:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 872673:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 872673:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm48.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-153vm51.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm48.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-153vm51.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/alive_router_check_interval
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/router_ping_timeout
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Disable routing on trevis-153vm49
Lustre: DEBUG MARKER: Disable routing on trevis-153vm49
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Enable routing on trevis-153vm49
Lustre: DEBUG MARKER: Enable routing on trevis-153vm49
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 873736:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 873736:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.92@tcp
LNet: Removed LNI 10.240.42.92@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Autotest: Test running for 210 minutes (lustre-reviews_review-ldiskfs_120091.56)
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
Lustre: DEBUG MARKER: e2label /dev/mapper/ost3_flakey
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: e2label /dev/mapper/ost6_flakey
Lustre: DEBUG MARKER: test -b /dev/mapper/ost5_flakey
Lustre: DEBUG MARKER: e2label /dev/mapper/ost7_flakey
Lustre: DEBUG MARKER: test -b /dev/mapper/ost2_flakey
Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost3; mount -t lustre -o localrecov /dev/mapper/ost3_flakey /mnt/lustre-ost3
Lustre: Lustre: Build Version: 2.17.0_RC4_5_g36e97ed
Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost1; mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1
LNet: Added LNI 10.240.42.92@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffb3e6806eca18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc07b0340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc07b0338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffffb3e6806eca70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff937efbd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000086010004 CR4: 00000000000606e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? kfree+0xd3/0x250
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: Unmounted lustre-client
Lustre: Skipped 17 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 794677:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 794677:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.172@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn238_1_g2b260f1
LNet: Added LNI 10.240.41.172@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 795007:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff937dd5e17800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 795175:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 795175:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.172@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn238_1_g2b260f1
LNet: Added LNI 10.240.41.172@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.85.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff1d7971006f8a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0a8e300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0a8e2f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff1d7971006f8a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff179c97fbb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000079210002 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
LustreError: 766426:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff179c97d213f800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 766426:0:(lov_obd.c:783:lov_cleanup()) Skipped 151 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 766570:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 766570:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 766838:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 766789:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c978dd1e000: no such obd: rc = -22
LustreError: 766787:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff179c978dd1e000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 766956:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 766956:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 767206:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff179c9784014000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 767206:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 767324:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 767324:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 767588:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff179c97a84dd000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 767539:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c97a84dd000: import has been deactivated: rc = -22
LustreError: 767588:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 767707:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 767707:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 767968:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff179c97a84da800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 767968:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 768087:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 768087:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 768342:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff179c97820ad000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 768293:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c97820ad000: import has been deactivated: rc = -22
LustreError: 768342:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 768461:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 768461:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 768666:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c9783622000: import has been deactivated: rc = -22
LustreError: 768715:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 768834:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 768834:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 769091:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff179c97a84d8000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 769091:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 769040:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff179c97a84d8000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 769209:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 769209:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 769402:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c97a84da800: import has been deactivated: rc = -22
LustreError: 769451:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 769398:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff179c97a84da800: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 769570:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 769570:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 769771:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c97820a8000: import has been deactivated: rc = -22
LustreError: 769822:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 769767:0:(gss_cli_upcall.c:255:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff179c97820a8000: obd has stopped: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 769941:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 769941:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 770148:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff179c9783622000: import has been deactivated: rc = -22
LustreError: 770197:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 770138:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0002-osc-ff179c9783622000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 770317:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 770317:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_28_g7476944
LNet: Added LNI 10.240.39.120@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.85.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff67586b00003a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0a40300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0a402f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff67586b00003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff32757e3ba00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000044210001 CR4: 0000000000771ef0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 764353:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 764353:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.171@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_18_g81a813c
LNet: Added LNI 10.240.45.171@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 764580:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff32757e071d5800: import has been deactivated: rc = -22
LustreError: 764629:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 764578:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff32757e071d5800: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 764748:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 764748:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.171@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_18_g81a813c
LNet: Added LNI 10.240.45.171@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 765005:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff32757dc86c0000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 764956:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff32757dc86c0000: import has been deactivated: rc = -22
LustreError: 765005:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 765124:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 765124:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.171@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_18_g81a813c
LNet: Added LNI 10.240.45.171@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 765388:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff32757dd78c0800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 765339:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff32757dd78c0800: import has been deactivated: rc = -22
LustreError: 765388:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 765507:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 765507:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.171@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_18_g81a813c
LNet: Added LNI 10.240.45.171@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 765773:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff32757dc44cb000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 765723:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff32757dc44cb000: import has been deactivated: rc = -22
LustreError: 765773:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 765892:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 765892:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.171@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.17.0_RC1_18_g81a813c
LNet: Added LNI 10.240.45.171@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffa7aac0003a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0647300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc06472f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffffa7aac0003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8d513bc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000007e210001 CR4: 00000000000606f0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? kfree+0xd3/0x250
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 800894:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 800894:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 801116:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff8d5049f97800: import has been deactivated: rc = -22
LustreError: 801165:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 801283:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 801283:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 801537:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff8d5101cae000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 801487:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff8d5101cae000: import has been deactivated: rc = -22
LustreError: 801537:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 801656:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 801656:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 801915:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff8d50150f4000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 801866:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff8d50150f4000: import has been deactivated: rc = -22
LustreError: 801915:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 802034:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 802034:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 802283:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff8d5024ce8000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 802283:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 802401:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 802401:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 802664:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff8d51027e6000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 802664:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 802615:0:(gss_cli_upcall.c:255:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff8d51027e6000: obd has stopped: rc = -22
LustreError: 802609:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0004-osc-ffff8d51027e6000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 802781:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 802781:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 803029:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
LustreError: 802968:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0001-osc-ffff8d51033ad000: no such obd: rc = -22
Key type lgssc unregistered
LNet: 803148:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 803148:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 803410:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff8d5102ecc800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 803410:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 803528:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 803528:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 803736:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff8d50175cc000: import has been deactivated: rc = -22
LustreError: 803785:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 803905:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 803905:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.12@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_gd24ac3d
LNet: Added LNI 10.240.41.12@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 0 PID: 840040 Comm: acceptor_000 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffb214c0003a18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0962340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0962338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ffff938c1e029100 R11: 0000000000000000 R12: ffffb214c0003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff938d3bc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000000b7010004 CR4: 00000000000606f0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? update_load_avg+0x7e/0x710
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:_raw_spin_unlock_irqrestore+0x11/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: Unmounted lustre-client
Lustre: Skipped 17 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 832893:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 832893:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 833234:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c220b3000: obd has stopped: rc = -22
LustreError: 833232:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff938c220b3000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 833404:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 833404:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 833688:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c1e355000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 833859:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 833859:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 834319:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 834319:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 834617:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c38657800: obd has stopped: rc = -22
LustreError: 834615:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff938c38657800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 834787:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 834787:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 835083:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c2955d000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 835253:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 834886:0:(lib-move.c:4844:lnet_parse()) 10.240.41.114@tcp, src 10.240.41.114@tcp: Dropping PUT (error -108 looking up sender)
LNetError: 835253:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 835542:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938d053ce800: import has been deactivated: rc = -22
LustreError: 835532:0:(gss_cli_upcall.c:267:gss_do_ctx_init_rpc()) lustre-OST0002-osc-ffff938d053ce800: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 835712:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 835712:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 836001:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938d04427800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 836172:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 836172:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 836467:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c38653000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 836637:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 836637:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 836928:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ffff938c1e357000: import has been deactivated: rc = -22
LustreError: 836926:0:(gss_cli_upcall.c:267:gss_do_ctx_init_rpc()) lustre-OST0004-osc-ffff938c1e357000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 837102:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 837102:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 837398:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c38657000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 837568:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 837568:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 838033:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 838033:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 838322:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c1378a800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 838492:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 838492:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 838782:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c38656800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 838953:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 838953:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Autotest: Test running for 250 minutes (lustre-b_es-reviews_review-dne-selinux-ssk-part-2_26232.1002)
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 839311:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938c1179d800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 839481:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 839481:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 839768:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff938d04427000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 839938:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 839938:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.41.112@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn232_1_g5490840
LNet: Added LNI 10.240.41.112@tcp [8/256/0/180]
Link to test
sanity-lnet test 100: Add route with single gw
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffaeeac06eca18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0829300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc08292f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffffaeeac06eca70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff9c5cbbd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000000a4a10003 CR4: 00000000000606e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? kfree+0xd3/0x250
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type lgssc unregistered
LNet: 736699:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 736699:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 738360:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 738360:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 740816:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 740816:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 743050:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 743050:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 745306:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 745306:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 747784:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 747784:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 750042:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 750042:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 752030:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 752030:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 754287:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 754287:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 757255:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 757255:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 759511:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 759511:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Autotest: Test running for 240 minutes (lustre-reviews_review-ldiskfs_118752.55)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 772864:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 772864:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 775016:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 773931:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 773931:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from up to down
LNetError: 773931:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 773931:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
Autotest: Test running for 245 minutes (lustre-reviews_review-ldiskfs_118752.55)
LNetError: 773931:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 773931:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 776130:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 776746:0:(timer.c:222:stt_shutdown()) waiting for 1 threads to terminate
LNet: 776746:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 776746:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 778733:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 778733:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 780885:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 780885:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 779799:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 779799:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from up to down
LNetError: 779799:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 779799:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 779799:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 781471:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 781471:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 783459:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 783459:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 785610:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.96@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 785706:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.96@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 785802:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 784525:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 784525:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from up to down
LNetError: 784525:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 784525:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 786571:0:(lib-move.c:2362:lnet_handle_find_routed_path()) no route to 10.240.42.96@tcp from 10.240.42.98@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 786773:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 786773:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 788761:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 788761:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 790912:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.96@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 791008:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.42.96@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 791104:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 791104:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 789827:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 789827:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
LNetError: 789827:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
LNetError: 789827:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
LNetError: 792448:0:(lib-move.c:2362:lnet_handle_find_routed_path()) no route to 10.240.42.96@tcp from 10.240.42.98@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 792650:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 792650:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 794640:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 794640:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 797665:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 797665:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 799653:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 799653:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 801805:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 802679:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 802679:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 805056:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 805056:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Autotest: Test running for 250 minutes (lustre-reviews_review-ldiskfs_118752.55)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 808156:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 808156:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 808156:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 808259:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 808259:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 810439:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 810439:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 812783:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 812783:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 811602:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 811602:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from up to down
LNetError: 811602:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 811602:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp1 is being used as a gateway but routing feature is not turned on
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
LNetError: 811602:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
LNet: 813552:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 813552:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 816029:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 816029:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 819077:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 819077:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 819077:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 819372:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 819372:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 823322:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 823322:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 825473:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 824388:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 824388:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from up to down
LNetError: 824388:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 824388:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 824388:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 824388:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp3 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp3 --if eth0
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 826156:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 826156:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 829946:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 829946:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.42.97@tcp1
LNet: 832097:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 831012:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 831012:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from up to down
LNetError: 831012:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 831012:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
Autotest: Test running for 255 minutes (lustre-reviews_review-ldiskfs_118752.55)
LNetError: 831012:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.42.97@tcp is being used as a gateway but routing feature is not turned on
LNetError: 831012:0:(lib-lnet.h:1337:lnet_set_route_aliveness()) route to tcp through 10.240.42.97@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.42.96@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.42.97@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.42.97@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.42.97@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 832736:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 832736:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 835110:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 835110:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.42.98@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 838157:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 838157:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 838157:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm6.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-76vm1.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm6.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-76vm1.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/alive_router_check_interval
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/router_ping_timeout
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Disable routing on trevis-76vm2
Lustre: DEBUG MARKER: Disable routing on trevis-76vm2
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Enable routing on trevis-76vm2
Lustre: DEBUG MARKER: Enable routing on trevis-76vm2
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 839221:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 839221:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.42.98@tcp
LNet: Removed LNI 10.240.42.98@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.61_40_g0660e94
LNet: Added LNI 10.240.42.98@tcp [8/256/0/180]
Link to test
sanity-lnet test 226: test missing route for 1 of 2 routers
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 513 Comm: systemd-udevd Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffb56c806f4a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0983300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc09832f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ffff98467aa88500 R11: 0000000000000000 R12: ffffb56c806f4a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007fbafc62e280(0000) GS:ffff98467bd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000001025a0004 CR4: 0000000000370ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? irqtime_account_irq+0x3d/0xc0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? receive_buf+0x39b/0xe70 [virtio_net]
? enqueue_entity+0x102/0x6b0
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:do_syscall_64+0x1e/0x1a0
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type lgssc unregistered
LNet: 844785:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 844785:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.27.6@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 846733:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 846733:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm5.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm5.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.27.6@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.11@tcp1
LNet: 849323:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 847964:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 847964:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.5@tcp1
LNetError: 847964:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 847964:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages
LNetError: 847964:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.5@tcp has gone from down to up
LNetError: 847964:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.27.4@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.11@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.11@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.11@tcp1; else
LNet: 849011:0:(lib-move.c:2247:lnet_handle_find_routed_path()) No peer NI for gateway 10.240.27.11@tcp1. Attempting to find an alternative route.
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.11@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.11@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.11@tcp1; else
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.5@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.5@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.5@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 850317:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 850317:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.27.6@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 852630:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 852630:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8
Lustre: DEBUG MARKER: onyx-144vm5.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.27.6@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.11@tcp1
LNet: 855113:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 855113:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 853861:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 853861:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp1 has gone from up to down
LNetError: 853861:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 853861:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 853861:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.27.4@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.11@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.11@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.11@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 855798:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 855798:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.27.6@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 858112:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 858112:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm5.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm5.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.27.6@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.11@tcp1
LNet: 860704:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 859342:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 859342:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.5@tcp1
LNetError: 859342:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 859342:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages
LNetError: 859342:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 859342:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 859342:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp has gone from down to up
LNetError: 859342:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.27.4@tcp
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm5.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: onyx-144vm5.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 861993:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
Lustre: DEBUG MARKER: Start LST rw
Autotest: Test running for 265 minutes (lustre-reviews_review-ldiskfs-ubuntu_118504.58)
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.11@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.11@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.11@tcp1; else
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.5@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.5@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.5@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 862589:0:(timer.c:222:stt_shutdown()) waiting for 1 threads to terminate
LNet: 862589:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 862589:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.27.6@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 869857:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 869857:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm5.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm12.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm12.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm6.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm6.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.27.6@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.11@tcp1
LNet: 872446:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 871087:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.27.5@tcp1
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.5@tcp is being used as a gateway but routing feature is not turned on
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 871087:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp has gone from down to up
LNetError: 871087:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.27.4@tcp
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm5.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-144vm7.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: onyx-144vm5.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: onyx-144vm7.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 873710:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/alive_router_check_interval
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/router_ping_timeout
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Disable routing on onyx-144vm12
Lustre: DEBUG MARKER: Disable routing on onyx-144vm12
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 871087:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.27.11@tcp has gone from up to down
LNetError: 871087:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: Wait 5s for LST to start
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.27.11@tcp is being used as a gateway but routing feature is not turned on
LNetError: 871087:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Enable routing on onyx-144vm12
Lustre: DEBUG MARKER: Enable routing on onyx-144vm12
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
LNet: 1 peer NIs in recovery (showing 1): 10.240.27.11@tcp1
Lustre: DEBUG MARKER: Wait for lst to finish
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.11@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.11@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.11@tcp1; else
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.27.5@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.27.5@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.27.5@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 875292:0:(timer.c:222:stt_shutdown()) waiting for 1 threads to terminate
LNet: 875292:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 875292:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.27.6@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Autotest: Test running for 270 minutes (lustre-reviews_review-ldiskfs-ubuntu_118504.58)
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_24_g24d81f5
LNet: Added LNI 10.240.27.6@tcp [8/256/0/180]
Link to test
sanity-lnet test 100: Add route with single gw
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff508729406f4a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc0a29300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0a292f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff508729406f4a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff4e1bb6fbb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000007ec10005 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type lgssc unregistered
LNet: 810463:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 810463:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 812200:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 812200:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 814586:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 814586:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 816958:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 816958:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 819344:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 819344:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 821987:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 821987:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 824374:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 824374:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 826475:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 826475:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 828861:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 828861:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Autotest: Test running for 230 minutes (lustre-reviews_review-ldiskfs-ubuntu_118467.58)
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 832102:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 832102:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 834489:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 834489:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 847805:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 847805:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 850075:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 848930:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 848930:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp has gone from up to down
LNetError: 848930:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 848930:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 848930:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 848930:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm16.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 851246:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
Lustre: DEBUG MARKER: Start LST rw
Autotest: Test running for 235 minutes (lustre-reviews_review-ldiskfs-ubuntu_118467.58)
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 851922:0:(timer.c:222:stt_shutdown()) waiting for 1 threads to terminate
LNet: 851922:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 851922:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 854022:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 854022:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 856292:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 856292:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 855147:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 855147:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from up to down
LNetError: 855147:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 855147:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 855147:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 856945:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 856945:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 859046:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 859046:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 861316:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.45.174@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 861423:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.45.174@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 861530:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 860171:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 860171:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 860171:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp has gone from up to down
LNetError: 860171:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) Skipped 1 previous similar message
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 862386:0:(lib-move.c:2362:lnet_handle_find_routed_path()) no route to 10.240.45.174@tcp from 10.240.45.176@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 862610:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 862610:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 864712:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 864712:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 866983:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.45.174@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 867091:0:(lib-move.c:2321:lnet_handle_find_routed_path()) peer 10.240.45.174@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 867198:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 867198:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 865837:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 865837:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
LNetError: 865837:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
LNetError: 865837:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
LNetError: 869019:0:(lib-move.c:2362:lnet_handle_find_routed_path()) no route to 10.240.45.174@tcp from 10.240.45.176@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 869243:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 869243:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 871344:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 871344:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 874588:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 874588:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 876689:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 876689:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 878960:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 879933:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 879933:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 882462:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 882462:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 885664:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 885664:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 885664:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 885779:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 885779:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 888092:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 888092:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffe
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buf
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Autotest: Test running for 240 minutes (lustre-reviews_review-ldiskfs-ubuntu_118467.58)
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 890629:0:(router.c:718:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 890629:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 889322:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 889322:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from up to down
LNetError: 889322:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 889322:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp1 is being used as a gateway but routing feature is not turned on
LNetError: 889322:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp1 has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
LNet: 891165:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 891165:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 893809:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 893809:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 897012:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 897012:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 897012:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm16.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 897339:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 897339:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 903058:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 903058:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.45.175@tcp1
LNet: 905330:0:(router.c:733:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
LNetError: 904185:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 904185:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp has gone from up to down
LNetError: 904185:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 904185:0:(router.c:398:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message
LNetError: 904185:0:(router.c:398:lnet_router_discovery_ping_reply()) Peer 10.240.45.175@tcp is being used as a gateway but routing feature is not turned on
LNetError: 904185:0:(lib-lnet.h:1315:lnet_set_route_aliveness()) route to tcp through 10.240.45.175@tcp has gone from down to up
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n routes
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.174@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.45.175@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.45.175@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.45.175@tcp1; e
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 905982:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 905982:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 908511:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 908511:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm18.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm18.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.45.176@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm17.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm17.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 911714:0:(router.c:1482:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 911714:0:(router.c:1504:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 911714:0:(router.c:1526:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
LNet: Message forwarding will be enabled in 10s
LNet: Message forwarding enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm16.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-155vm19.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm16.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-155vm19.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/alive_router_check_interval
Lustre: DEBUG MARKER: echo 5 > /sys/module/lnet/parameters/router_ping_timeout
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Disable routing on trevis-155vm17
Lustre: DEBUG MARKER: Disable routing on trevis-155vm17
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait 5s for LST to start
Lustre: DEBUG MARKER: Wait 5s for LST to start
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Enable routing on trevis-155vm17
Lustre: DEBUG MARKER: Enable routing on trevis-155vm17
Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for lst to finish
Lustre: DEBUG MARKER: Wait for lst to finish
Autotest: Test running for 245 minutes (lustre-reviews_review-ldiskfs-ubuntu_118467.58)
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 912954:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 912954:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.45.176@tcp
LNetError: 912954:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNetError: 912954:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Skipped 1 previous similar message
LNet: Removed LNI 10.240.45.176@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_23_gc895b4d
LNet: Added LNI 10.240.45.176@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 0 PID: 238 Comm: kworker/0:4 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014
Workqueue: events do_free_init
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff452dfb40003a18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffffc07f7300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc07f72f8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ff442cff4420e700 R11: 0000000000000000 R12: ff452dfb40003a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff442cff7ba00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000003f210005 CR4: 0000000000771ef0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? srso_alias_return_thunk+0x5/0xfcdfd
? update_load_avg+0x7e/0x710
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? enqueue_entity+0x102/0x6b0
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_write_msr+0x6/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
LustreError: 33936:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff442cff57633800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 33936:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup
Lustre: Unmounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 34079:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 34079:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 34292:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff442cff41c22000: import has been deactivated: rc = -22
LustreError: 34341:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 34288:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff442cff41c22000: no such obd: rc = -22
LustreError: 34282:0:(gss_cli_upcall.c:255:gss_do_ctx_init_rpc()) lustre-OST0002-osc-ff442cff41c22000: obd has stopped: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 34460:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 34460:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 34722:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff442cff56ce2000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 34673:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff442cff56ce2000: import has been deactivated: rc = -22
LustreError: 34722:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 34841:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 34841:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 35088:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff442cff4359f000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 35088:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 35037:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff442cff4359f000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 35207:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 35207:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 35467:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff442cff62042000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 35417:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff442cff62042000: import has been deactivated: rc = -22
LustreError: 35467:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 35586:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 35586:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 35832:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff442cff43594000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 35832:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 35779:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff442cff43594000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 35951:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 35951:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 36155:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff442cff41c20800: import has been deactivated: rc = -22
LustreError: 36204:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 36323:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 36323:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Autotest: Test running for 10 minutes (lustre-reviews_review-dne-selinux-ssk-part-2_118189.53)
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 36639:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ff442cff41c24800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 36532:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff442cff41c24800: import has been deactivated: rc = -22
LustreError: 36639:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 36758:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 36758:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 37006:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 36955:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ff442cff41c21000: no such obd: rc = -22
LustreError: 36953:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ff442cff41c21000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 37125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 37125:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 37328:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff442cff47fbb800: import has been deactivated: rc = -22
LustreError: 37379:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 37498:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 37498:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.24.30@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_g85696fa
LNet: Added LNI 10.240.24.30@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 469 Comm: systemd-journal Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffaf1a806f4a18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0816740 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0816738
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ffff901fc1dbb700 R11: 0000000000000000 R12: ffffaf1a806f4a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f291d1fc980(0000) GS:ffff901ffbd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000104be6001 CR4: 0000000000370ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? irqtime_account_irq+0x3d/0xc0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? ep_poll_callback+0x22a/0x2c0
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:seccomp_run_filters+0x68/0x130
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
LustreError: 810867:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup
LustreError: 810867:0:(obd_class.h:479:obd_check_dev()) Skipped 545 previous similar messages
Lustre: Unmounted lustre-client
Lustre: Skipped 20 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 811010:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 811010:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 811267:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 811214:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ffff901ffa169800: no such obd: rc = -22
LustreError: 811210:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0003-osc-ffff901ffa169800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 811386:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 811386:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 811645:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fc202b000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 811645:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 811764:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 811764:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 812026:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fc4045800: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 811977:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fc4045800: import has been deactivated: rc = -22
LustreError: 812026:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 812145:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 812145:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 812403:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901ff0bdc000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 812403:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 812522:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 812522:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 812778:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fc2029000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 812778:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 812896:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 812896:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 813153:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 813098:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0004-osc-ffff901fc202a000: import has been deactivated: rc = -22
LustreError: 813104:0:(gss_cli_upcall.c:242:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fc202a000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 813272:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 813272:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 813531:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fc35da000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 813531:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 813650:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 813650:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 813862:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fc202e800: import has been deactivated: rc = -22
LustreError: 813911:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 814030:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 814030:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 814239:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fc4554000: import has been deactivated: rc = -22
LustreError: 814290:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 814408:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 814408:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 814615:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fdf87b800: import has been deactivated: rc = -22
LustreError: 814664:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 814783:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 814783:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 815037:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fc36dd000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 814988:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fc36dd000: import has been deactivated: rc = -22
LustreError: 815037:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
LustreError: 814986:0:(gss_cli_upcall.c:255:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff901fc36dd000: obd has stopped: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 815155:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 815155:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 815411:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fdf878000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 815362:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fdf878000: import has been deactivated: rc = -22
LustreError: 815411:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 815529:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 815529:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 815794:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff901fc202e000: lov tgt 0 not cleaned! deathrow=0, lovrc=1
LustreError: 815743:0:(gss_cli_upcall.c:295:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff901fc202e000: import has been deactivated: rc = -22
LustreError: 815794:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 815913:0:(lib-ptl.c:970:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 815913:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.28.244@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.59_50_gcab2d33
LNet: Added LNI 10.240.28.244@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffa06ec06eca18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0a15340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0a15338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ffff9425c6e7ef00 R11: 0000000000000000 R12: ffffa06ec06eca70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff9425ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000087a10001 CR4: 00000000000606e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? receive_buf+0x39b/0xe70 [virtio_net]
? update_load_avg+0x7e/0x710
? enqueue_entity+0x102/0x6b0
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 823218:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 823218:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 823555:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9425759ae800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 823725:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 823725:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 824192:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 824192:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 824492:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9425759ae800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 824662:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 824662:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 824955:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9425446bc000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 825126:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 825126:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 825593:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 825593:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 825881:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9425446bc000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 826051:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 826051:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.39.88@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g89f9482
LNet: Added LNI 10.240.39.88@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0
Oops: 0000 [#1] SMP NOPTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.76.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.16.3-2.el9_5.1 04/01/2014
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ff6d7f71006f4a18 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0af3340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0af3338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ff6d7f71006f4a70
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ff4a5d77fbb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000011da10004 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? srso_alias_return_thunk+0x5/0xfcdfd
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x81/0xd0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
? srso_alias_return_thunk+0x5/0xfcdfd
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? srso_alias_return_thunk+0x5/0xfcdfd
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? srso_alias_return_thunk+0x5/0xfcdfd
? inet_gro_receive+0x223/0x2d0
? srso_alias_return_thunk+0x5/0xfcdfd
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? srso_alias_return_thunk+0x5/0xfcdfd
? kfree+0xd3/0x250
? srso_alias_return_thunk+0x5/0xfcdfd
? srso_alias_return_thunk+0x5/0xfcdfd
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: Unmounted lustre-client
Lustre: Skipped 17 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 824317:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 824317:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.46.23@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g20c61da
LNet: Added LNI 10.240.46.23@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 824644:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff4a5d77dadf5800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 824814:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 824814:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.46.23@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g20c61da
LNet: Added LNI 10.240.46.23@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 825111:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff4a5d77c4224000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 825281:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 825281:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.46.23@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g20c61da
LNet: Added LNI 10.240.46.23@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 825578:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff4a5d77eedb6800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 825748:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 825748:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.46.23@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g20c61da
LNet: Added LNI 10.240.46.23@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 826033:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff4a5d77c4222800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 826206:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 826206:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.46.23@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g20c61da
LNet: Added LNI 10.240.46.23@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 826489:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ff4a5d77c2a69800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 826659:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 826659:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.46.23@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn229_1_g20c61da
LNet: Added LNI 10.240.46.23@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.51.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffa940006f4a20 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0a00340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0a00338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ffff992244b27b00 R11: 0000000000000000 R12: ffffa940006f4a70
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff9922bfd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000006e010004 CR4: 00000000001706e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? __bad_area_nosemaphore+0x157/0x180
? call_function_single_interrupt+0xa/0x20
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x7c/0xc0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? enqueue_entity+0x102/0x6b0
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 836970:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 836970:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 837304:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff992206342000: import has been deactivated: rc = -22
LustreError: 837306:0:(gss_cli_upcall.c:267:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992206342000: no such obd: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 837476:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 837476:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Autotest: Test running for 220 minutes (lustre-b_es-reviews_review-dne-selinux-ssk-part-2_25780.68)
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 837774:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992206020000: obd has stopped: rc = -22
LustreError: 837770:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ffff992206020000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 838006:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 838006:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 838301:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992205168800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 838472:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 838472:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 838766:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992206021000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 838937:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 838937:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 839233:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992244ab9000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 839404:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 839404:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 839699:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992206022000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 839869:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 839869:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 840336:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 840336:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 840635:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff992244abb000: obd has stopped: rc = -22
LustreError: 840633:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff992244abb000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 840805:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 840805:0:(acceptor.c:292:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.43.218@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn227_1_g7745621
LNet: Added LNI 10.240.43.218@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 514 Comm: systemd-udevd Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.51.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffaff7006f4a20 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0657340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0657338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffffaff7006f4a70
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
FS: 00007ffb38fa2280(0000) GS:ffff9acb3fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000003456003 CR4: 00000000001706e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? fib_rules_lookup+0x143/0x1c0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x7c/0xc0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? receive_buf+0x39b/0xe70 [virtio_net]
? enqueue_task_fair+0x94/0x3a0
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:do_syscall_64+0x1e/0x1a0
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: Unmounted lustre-client
Lustre: Skipped 14 previous similar messages
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 996975:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 996975:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 997313:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9acaa2f31800: import has been deactivated: rc = -22
LustreError: 997313:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) Skipped 1 previous similar message
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 997483:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 997483:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 997782:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9aca83680800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 997953:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 997953:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 998242:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9aca82e66800: import has been deactivated: rc = -22
LustreError: 998240:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff9aca82e66800: obd has stopped: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 998414:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 998414:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 998882:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 998882:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 999171:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9aca84b27800: import has been deactivated: rc = -22
LustreError: 999169:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff9aca84b27800: obd has stopped: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 999342:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 999342:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 999639:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9acab5f7e000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 999809:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 999809:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1000105:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0005-osc-ffff9aca85abf800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1000280:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1000280:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1000580:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9acaa2f32800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1000751:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1000751:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1001045:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9acab5f7d800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1001219:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1001219:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1001512:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9aca83b5f000: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1001682:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1001682:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1001983:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9aca83b5a800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1002153:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1002153:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1002445:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9aca83b5a800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1002615:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1002615:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1003084:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1003084:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1003553:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1003553:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1003852:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff9acaa2f35800: obd has stopped: rc = -22
LustreError: 1003850:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff9acaa2f35800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1004022:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1004022:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.26.105@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_1_gd95b00f
LNet: Added LNI 10.240.26.105@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.51.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffba58006eca20 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0a74340 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0a74338
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffffba58006eca70
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff920bffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000015810004 CR4: 00000000000606e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? fib_rules_lookup+0x143/0x1c0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x7c/0xc0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? kfree+0xd3/0x250
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 1002209:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1002209:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_2_g5338f65
LNet: Added LNI 10.240.38.120@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 1002545:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff920b44e73800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 1002715:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 1002715:0:(acceptor.c:280:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.120@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn224_2_g5338f65
LNet: Added LNI 10.240.38.120@tcp [8/256/0/180]
Link to test
sanity-lnet test 100: Add route with single gw
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: P OE -------- - - 4.18.0-553.58.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffb32c00003a20 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc0f3c180 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc0f3c178
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: 0000000000000000 R11: 0000000000000000 R12: ffffb32c00003a70
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff89157fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000098c10003 CR4: 00000000000606f0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? fib_rules_lookup+0x143/0x1c0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x7c/0xc0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? napi_consume_skb+0x45/0xd0
? kfree+0xd3/0x250
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type lgssc unregistered
LNet: 919342:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 919342:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 921006:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 921006:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 923271:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 923271:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 925516:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 925516:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 927782:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 927782:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 930272:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 930272:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 932537:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 932537:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 934537:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 934537:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 936802:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 936802:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 939783:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 939783:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 942048:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 942048:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Autotest: Test running for 230 minutes (lustre-reviews_review-dne-zfs-part-2_115950.36)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 952228:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 952228:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 954390:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
LNet: 955384:0:(rpc.c:648:srpc_service_add_buffers()) waiting for adding buffer
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: Start LST rw
Lustre: DEBUG MARKER: lst stop brw_rw
Lustre: DEBUG MARKER: Stop LST rw
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 955977:0:(timer.c:222:stt_shutdown()) waiting for 1 threads to terminate
LNet: 955977:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 955977:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 957976:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 957976:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 960137:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 960137:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 960627:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 960627:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 962627:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 962627:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 964787:0:(lib-move.c:2294:lnet_handle_find_routed_path()) peer 10.240.38.20@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 964883:0:(lib-move.c:2294:lnet_handle_find_routed_path()) peer 10.240.38.20@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 964979:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 963699:0:(lib-lnet.h:1330:lnet_set_route_aliveness()) route to tcp through 10.240.38.21@tcp has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 965555:0:(lib-move.c:2335:lnet_handle_find_routed_path()) no route to 10.240.38.20@tcp from 10.240.38.22@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 965757:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 965757:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 967763:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 967763:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 969924:0:(lib-move.c:2294:lnet_handle_find_routed_path()) peer 10.240.38.20@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 970020:0:(lib-move.c:2294:lnet_handle_find_routed_path()) peer 10.240.38.20@tcp has no available nets
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 970116:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 970116:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
LNetError: 968836:0:(lib-lnet.h:1330:lnet_set_route_aliveness()) route to tcp through 10.240.38.21@tcp1 has gone from up to down
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 971279:0:(lib-move.c:2335:lnet_handle_find_routed_path()) no route to 10.240.38.20@tcp from 10.240.38.22@tcp1
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 971481:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 971481:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 973481:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 973481:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 976324:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 976324:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 978324:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 978324:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 980484:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
Autotest: Test running for 235 minutes (lustre-reviews_review-dne-zfs-part-2_115950.36)
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 981223:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 981223:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n cpu_partition_table | wc -l
Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 983612:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 983612:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl set routing 1
LNet: 986671:0:(router.c:1403:lnet_nrb_tiny_calculate()) tiny_router_buffers=9 less than recommended minimum 512
LNet: 986671:0:(router.c:1425:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096
LNet: 986671:0:(router.c:1447:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 986774:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 986774:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 988966:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 988966:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffe
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm6.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm6.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 991319:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
LNet: 991319:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
LNet: 991703:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 991703:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing lnet_if_list
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 996098:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 996098:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm1.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm1.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm2.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm2.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-21vm3.trevis.whamcloud.com: executing \/usr\/sbin\/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
Lustre: DEBUG MARKER: trevis-21vm3.trevis.whamcloud.com: executing /usr/sbin/lnetctl net add --net tcp1 --if eth0
LNet: Added LNI 10.240.38.22@tcp1 [8/256/0/180]
Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 10.240.38.21@tcp1
LNet: 998258:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.38.20@tcp
LNetError: 997170:0:(lib-lnet.h:1330:lnet_set_route_aliveness()) route to tcp through 10.240.38.21@tcp has gone from up to down
LNetError: 997170:0:(lib-lnet.h:1330:lnet_set_route_aliveness()) route to tcp through 10.240.38.21@tcp has gone from down to up
Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp --gateway 10.240.38.21@tcp1 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp via 10.240.38.21@tcp1"; /usr/sbin/lnetctl route del --net tcp --gateway 10.240.38.21@tcp1; else
Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
LNet: 998748:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 998748:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.38.22@tcp1
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
Autotest: Test running for 240 minutes (lustre-reviews_review-dne-zfs-part-2_115950.36)
Key type ._llcrypt registered
Key type .llcrypt registered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.16.57_97_g55be0a7
LNet: Added LNI 10.240.38.22@tcp [8/256/0/180]
Link to test
sanity-selinux test 20d: [atomicity] avoid getxattr for security context
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G OE -------- - - 4.18.0-553.58.1.el8_10.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:__wake_up_common+0x4c/0x190
Code: 24 10 4d 85 c9 74 0a 41 f6 01 04 0f 85 a1 00 00 00 48 8b 43 08 48 83 c3 08 4c 8d 48 e8 49 8d 41 18 48 39 c3 0f 84 f4 00 00 00 <49> 8b 41 18 89 54 24 08 31 ed 4c 8d 70 e8 45 8b 29 41 f6 c5 04 75
RSP: 0018:ffffb3d7406eca20 EFLAGS: 00010082
RAX: 0000000000000000 RBX: ffffffffc093a2e0 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffc093a2d8
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffffffffffe8
R10: ffff99ac43bf3500 R11: 0000000000000000 R12: ffffb3d7406eca70
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff99acffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000008fa10004 CR4: 00000000000606e0
Call Trace:
<IRQ>
? __die_body+0x1a/0x60
? no_context+0x1ba/0x3f0
? fib_rules_lookup+0x143/0x1c0
? __bad_area_nosemaphore+0x157/0x180
? do_page_fault+0x37/0x12d
? page_fault+0x1e/0x30
? __wake_up_common+0x4c/0x190
__wake_up_common_lock+0x7c/0xc0
tcp_child_process+0x181/0x1b0
tcp_v4_rcv+0xa47/0xd90
ip_protocol_deliver_rcu+0x2c/0x1d0
ip_local_deliver_finish+0x4d/0x60
ip_local_deliver+0x6b/0xf0
? ip_rcv_finish+0xa9/0x430
ip_rcv+0x27a/0x3b5
__netif_receive_skb_core+0xc08/0xd10
? inet_gro_receive+0x223/0x2d0
? ktime_get_with_offset+0x54/0xc0
netif_receive_skb_internal+0x3d/0xb0
napi_gro_receive+0x118/0x180
receive_buf+0x39b/0xe70 [virtio_net]
? kvm_sched_clock_read+0xd/0x20
? detach_buf_split+0x6c/0x130
virtnet_poll+0x164/0x42b [virtio_net]
__napi_poll+0x2d/0x130
net_rx_action+0x252/0x320
__do_softirq+0xdc/0x2cf
irq_exit_rcu+0xc6/0xd0
irq_exit+0xa/0x10
do_IRQ+0x7f/0xd0
common_interrupt+0xf/0xf
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x20
Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
Key type lgssc unregistered
LNet: 966742:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 966742:0:(acceptor.c:269:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.40.28@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn221_1_gde926d4
LNet: Added LNI 10.240.40.28@tcp [8/256/0/180]
Key type lgssc registered
Lustre: Mounted lustre-client
Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
Lustre: DEBUG MARKER: lsof -t /mnt/lustre
Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
LustreError: 967081:0:(gss_cli_upcall.c:280:gss_do_ctx_init_rpc()) lustre-OST0007-osc-ffff99ac43732800: obd has stopped: rc = -22
LustreError: 967079:0:(gss_cli_upcall.c:317:gss_do_ctx_init_rpc()) lustre-OST0006-osc-ffff99ac43732800: import has been deactivated: rc = -22
Lustre: Unmounted lustre-client
Key type lgssc unregistered
LNet: 967251:0:(lib-ptl.c:988:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
LNetError: 967251:0:(acceptor.c:269:lnet_acceptor_remove_socket()) Interface eth0 not found
LNet: Removed LNI 10.240.40.28@tcp
Key type .llcrypt unregistered
Key type ._llcrypt unregistered
libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
alg: No test for adler32 (adler32-zlib)
Key type ._llcrypt registered
Key type .llcrypt registered
Lustre: Lustre: Build Version: 2.14.0_ddn221_1_gde926d4
LNet: Added LNI 10.240.40.28@tcp [8/256/0/180]
Link to test
Return to new crashes list