Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 0 PID: 34904 Comm: ll_ost00_043 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
vpanic+0x129/0x2f0
panic+0x58/0x60
out_of_memory+0x2ae/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
? __entry_text_end+0x1026f9/0x1026fd
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>

obd_memory max: 187663863, obd_memory current: 156092295
obd_memory max: 187663863, obd_memory current: 156092295
obd_memory max: 187663863, obd_memory current: 156092295
ll_ost00_043 invoked oom-killer: gfp_mask=0x42cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP), order=0, oom_score_adj=0
CPU: 0 PID: 34904 Comm: ll_ost00_043 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
dump_header+0x49/0x240
out_of_memory+0x289/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
? __entry_text_end+0x1026f9/0x1026fd
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>
Mem-Info:
active_anon:0 inactive_anon:0 isolated_anon:0
Node 0 active_anon:0kB inactive_anon:0kB active_file:152kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:0kB writeback:0kB shmem:0kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10064kB pagetables:2072kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:14644kB boost:0kB min:280kB low:348kB high:416kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1678 3598 3598 3598
Node 0 DMA32 free:40340kB boost:0kB min:31388kB low:39232kB high:47076kB reserved_highatomic:4096KB active_anon:0kB inactive_anon:0kB active_file:24kB inactive_file:72kB unevictable:0kB writepending:0kB present:2080624kB managed:1752944kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 1920 1920 1920
Node 0 Normal free:23488kB boost:0kB min:35912kB low:44888kB high:53864kB reserved_highatomic:2048KB active_anon:0kB inactive_anon:0kB active_file:16kB inactive_file:32kB unevictable:0kB writepending:0kB present:2097152kB managed:1972912kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 3*4kB (U) 1*8kB (U) 0*16kB 1*32kB (U) 0*64kB 0*128kB 1*256kB (U) 0*512kB 0*1024kB 1*2048kB (M) 3*4096kB (M) = 14644kB
Node 0 DMA32: 1276*4kB (UME) 660*8kB (UME) 434*16kB (UME) 285*32kB (UME) 86*64kB (UME) 12*128kB (UME) 27*256kB (UME) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 40400kB
Node 0 Normal: 1507*4kB (UME) 708*8kB (UME) 297*16kB (UME) 99*32kB (UME) 33*64kB (UME) 11*128kB (UME) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 23388kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
36 total pagecache pages
0 pages in swap cache
Free swap = 3761628kB
Total swap = 3809276kB
1048442 pages RAM
0 pages HighMem/MovableOnly
113138 pages reserved
0 pages cma reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
ofd_obj 85KB 255KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 142KB 485KB
vvp_thread_kmem 15KB 15KB
vvp_session_kmem 7KB 7KB
ll_thread_kmem 7KB 7KB
lov_session_kmem 15KB 15KB
lov_thread_kmem 7KB 7KB
osc_session_kmem 7KB 7KB
osc_thread_kmem 61KB 61KB
tgt_fmd_cache 27KB 43KB
tgt_session_kmem 46KB 46KB
tgt_thread_kmem 226KB 226KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 175KB 472KB
ldlm_locks 364KB 976KB
ldlm_resources 168KB 480KB
lnet_msg 71KB 71KB
lnet_rspt 1356KB 1356KB
lnet_small_MDs 31KB 31KB
lnet_MEs 51KB 51KB
ptlrpc_cache 845KB 895KB
cl_env_kmem 3KB 3KB
ll_obd_dev_cache 374KB 374KB
bio-184 31KB 31KB
bio-264 73KB 90KB
zfs_znode_hold_cache 7KB 7KB
zap_attr_long_cache 30KB 30KB
zap_attr_cache 7KB 7KB
zap_name 23KB 23KB
dbuf_dirty_record_t 9138KB 9138KB
dmu_buf_impl_t 10662KB 10677KB
arc_buf_t 763KB 768KB
zio_buf_comb_16384 5264KB 5728KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_8192 6216KB 6592KB
zio_buf_comb_6144 60KB 60KB
zio_buf_comb_4096 192KB 320KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 63KB 63KB
zio_link_cache 128KB 131KB
zio_cache 3120KB 3188KB
zfs_btree_leaf_cache 6732KB 6912KB
nfs_commit_data 15KB 15KB
nfs_write_data 95KB 175KB
nfs_read_data 31KB 31KB
nfs_page 44KB 44KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
ext4_io_end_vec 8KB 8KB
jbd2_inode 8KB 8KB
scsi_sense_cache 24KB 24KB
bio-144 7KB 7KB
fib6_nodes 4KB 4KB
ip6_dst_cache 7KB 7KB
PINGv6 15KB 15KB
RAWv6 31KB 31KB
UDPv6 63KB 63KB
TCPv6 59KB 59KB
mqueue_inode_cache 16KB 16KB
kioctx 15KB 15KB
aio_kiocb 7KB 7KB
posix_timers_cache 7KB 7KB
UNIX-STREAM 168KB 195KB
UNIX 75KB 75KB
tcp_bind_bucket 8KB 8KB
ip_fib_trie 3KB 3KB
ip_fib_alias 3KB 3KB
ip_dst_cache 48KB 48KB
PING 15KB 15KB
RAW 15KB 15KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 150KB 150KB
hugetlbfs_inode_cache 15KB 15KB
bio-272 18KB 18KB
ep_head 8KB 8KB
eventpoll_pwq 16KB 16KB
eventpoll_epi 28KB 28KB
inotify_inode_mark 15KB 15KB
sgpool-128 128KB 128KB
sgpool-64 64KB 64KB
sgpool-32 32KB 32KB
sgpool-16 16KB 16KB
sgpool-8 8KB 8KB
bio_integrity_data 3KB 3KB
request_queue 31KB 31KB
blkdev_ioc 7KB 7KB
bio-208 40KB 60KB
biovec-max 320KB 320KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
khugepaged_mm_slot 7KB 7KB
user_namespace 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 112KB 112KB
skbuff_head_cache 44KB 44KB
configfs_dir_cache 3KB 3KB
file_lock_cache 7KB 7KB
file_lock_ctx 7KB 7KB
fsnotify_mark_connector 8KB 8KB
taskstats 15KB 15KB
proc_dir_entry 220KB 220KB
pde_opener 7KB 7KB
seq_file 7KB 7KB
sigqueue 7KB 7KB
shmem_inode_cache 1011KB 1054KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4448KB 4469KB
mnt_cache 225KB 225KB
filp 262KB 340KB
names_cache 96KB 96KB
net_namespace 30KB 30KB
lsm_inode_cache 615KB 844KB
lsm_file_cache 26KB 32KB
key_jar 28KB 28KB
uts_namespace 15KB 15KB
nsproxy 7KB 7KB
vma_lock 190KB 227KB
vm_area_struct 966KB 980KB
fs_cache 16KB 16KB
files_cache 110KB 110KB
signal_cache 737KB 771KB
sighand_cache 1278KB 1330KB
task_struct 5578KB 5763KB
cred_jar 146KB 157KB
anon_vma_chain 160KB 184KB
anon_vma 186KB 190KB
pid 132KB 133KB
Acpi-Operand 118KB 118KB
Acpi-ParseExt 7KB 7KB
Acpi-Parse 7KB 7KB
Acpi-State 23KB 23KB
Acpi-Namespace 19KB 19KB
numa_policy 3KB 3KB
perf_event 30KB 30KB
trace_event_file 193KB 193KB
ftrace_event_field 274KB 274KB
pool_workqueue 96KB 96KB
maple_node 151KB 240KB
task_group 94KB 94KB
mm_struct 123KB 189KB
vmap_area 459KB 507KB
kmalloc-cg-8k 32KB 32KB
kmalloc-cg-4k 128KB 128KB
kmalloc-cg-2k 480KB 480KB
kmalloc-cg-1k 99KB 128KB
kmalloc-cg-512 32KB 32KB
kmalloc-cg-256 8KB 8KB
kmalloc-cg-192 31KB 31KB
kmalloc-cg-128 8KB 8KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 8KB 8KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 12KB 12KB
kmalloc-8k 6744KB 7360KB
kmalloc-4k 14720KB 14720KB
kmalloc-2k 2664KB 2816KB
kmalloc-1k 10552KB 10752KB
kmalloc-512 3159KB 4656KB
kmalloc-256 950KB 956KB
kmalloc-192 2630KB 2740KB
kmalloc-128 556KB 904KB
kmalloc-96 1021KB 1764KB
kmalloc-64 1350KB 1724KB
kmalloc-32 425KB 716KB
kmalloc-16 147KB 152KB
kmalloc-8 55KB 56KB
kmem_cache_node 24KB 24KB
kmem_cache 84KB 84KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 513] 0 513 7174 96 86016 288 -250 systemd-journal
[ 527] 0 527 9185 64 90112 544 -1000 systemd-udevd
[ 579] 32 579 3708 32 73728 224 0 rpcbind
[ 580] 0 580 4963 39 57344 224 -1000 auditd
[ 612] 81 612 2684 32 61440 160 -900 dbus-broker-lau
[ 613] 81 613 1262 32 49152 96 -900 dbus-broker
[ 614] 0 614 65218 32 147456 736 0 NetworkManager
[ 617] 0 617 20687 32 61440 96 0 irqbalance
[ 619] 0 619 5261 32 81920 320 0 systemd-logind
[ 622] 997 622 21450 42 69632 128 0 chronyd
[ 635] 0 635 20267 32 57344 32 0 rhsmcertd
[ 637] 0 637 2974 64 65536 256 -1000 sshd
[ 641] 996 641 20093 37 69632 320 0 munged
[ 646] 0 646 14170 48 81920 256 0 gssproxy
[ 670] 0 670 41279 96 94208 480 0 rsyslogd
[ 671] 29 671 2857 41 57344 192 0 rpc.statd
[ 733] 0 733 85089 58 122880 480 0 automount
[ 741] 0 741 766 32 40960 32 0 agetty
[ 743] 0 743 1408 32 57344 32 0 agetty
[ 774] 0 774 2166 64 57344 192 0 crond
[ 1223] 0 1223 4454 64 77824 480 0 sshd-session
[ 1227] 0 1227 5695 32 86016 480 100 systemd
[ 1229] 0 1229 43292 50 102400 896 100 (sd-pam)
[ 1236] 0 1236 4519 64 77824 576 0 sshd-session
[ 78102] 0 78102 4453 64 69632 448 0 sshd-session
[ 78105] 0 78105 4518 61 69632 544 0 sshd-session
[ 184418] 0 184418 4453 64 73728 480 0 sshd-session
[ 184421] 0 184421 4551 63 73728 576 0 sshd-session
[ 201377] 0 201377 1052 64 49152 32 0 bash
[ 201391] 0 201391 1052 45 49152 64 0 bash
[ 201392] 0 201392 745 0 36864 0 0 find
[ 201393] 0 201393 741 32 45056 0 0 xargs
[ 201394] 0 201394 672 32 45056 0 0 tr

Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 0 PID: 34647 Comm: ll_ost00_031 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
vpanic+0x129/0x2f0
panic+0x58/0x60
out_of_memory+0x2ae/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
? __entry_text_end+0x1026f9/0x1026fd
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x159/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x125/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>

Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanityn test_71b ORI-366\/LU-1941: FIEMAP unimplemented on ZFS
Lustre: DEBUG MARKER: SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
obd_memory max: 179706015, obd_memory current: 147211391
ll_ost00_031 invoked oom-killer: gfp_mask=0x42cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP), order=0, oom_score_adj=0
CPU: 0 PID: 34647 Comm: ll_ost00_031 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
dump_header+0x49/0x240
out_of_memory+0x289/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
? __entry_text_end+0x1026f9/0x1026fd
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x159/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e5/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x125/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2b/0x50
</TASK>
Mem-Info:
active_anon:247 inactive_anon:1846 isolated_anon:0
Node 0 active_anon:988kB inactive_anon:7384kB active_file:2128kB inactive_file:564kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2616kB dirty:0kB writeback:280kB shmem:80kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10256kB pagetables:2068kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:14644kB boost:0kB min:280kB low:348kB high:416kB reserved_highatomic:0KB active_anon:0kB inactive_anon:32kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1625 3599 3599 3599
Node 0 DMA32 free:53236kB boost:16384kB min:46768kB low:54364kB high:61960kB reserved_highatomic:0KB active_anon:0kB inactive_anon:648kB active_file:0kB inactive_file:328kB unevictable:0kB writepending:0kB present:2080624kB managed:1706276kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 1974 1974 1974
Node 0 Normal free:24416kB boost:30720kB min:67636kB low:76864kB high:86092kB reserved_highatomic:0KB active_anon:928kB inactive_anon:7412kB active_file:1920kB inactive_file:736kB unevictable:0kB writepending:448kB present:2097152kB managed:2021592kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 3*4kB (UME) 3*8kB (UME) 1*16kB (E) 2*32kB (ME) 3*64kB (UME) 2*128kB (ME) 3*256kB (UME) 2*512kB (ME) 2*1024kB (ME) 1*2048kB (E) 2*4096kB (M) = 14644kB
Node 0 DMA32: 989*4kB (ME) 762*8kB (UME) 717*16kB (UME) 504*32kB (UME) 180*64kB (UME) 33*128kB (UME) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 53396kB
Node 0 Normal: 2045*4kB (UME) 822*8kB (UME) 341*16kB (UME) 97*32kB (UME) 16*64kB (UME) 1*128kB (M) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 24724kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
897 total pagecache pages
157 pages in swap cache
Free swap = 3770392kB
Total swap = 3809276kB
1048442 pages RAM
0 pages HighMem/MovableOnly
112635 pages reserved
0 pages cma reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
ofd_obj 79KB 266KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 140KB 478KB
vvp_thread_kmem 15KB 15KB
vvp_session_kmem 7KB 7KB
ll_thread_kmem 7KB 7KB
lov_session_kmem 15KB 15KB
lov_thread_kmem 7KB 7KB
osc_session_kmem 7KB 7KB
osc_thread_kmem 61KB 61KB
tgt_fmd_cache 23KB 39KB
tgt_session_kmem 35KB 35KB
tgt_thread_kmem 211KB 211KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 237KB 472KB
ldlm_locks 506KB 976KB
ldlm_resources 240KB 480KB
lnet_msg 71KB 71KB
lnet_rspt 1320KB 1320KB
lnet_small_MDs 35KB 35KB
lnet_MEs 36KB 55KB
ptlrpc_cache 739KB 741KB
cl_env_kmem 3KB 3KB
ll_obd_dev_cache 320KB 320KB
bio-184 31KB 31KB
bio-264 73KB 86KB
zfs_znode_hold_cache 7KB 7KB
zap_attr_long_cache 30KB 30KB
zap_attr_cache 7KB 7KB
zap_name 23KB 23KB
dbuf_dirty_record_t 8831KB 8831KB
dmu_buf_impl_t 10108KB 10108KB
arc_buf_t 748KB 748KB
zio_buf_comb_16384 5392KB 5664KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_8192 4352KB 4544KB
zio_buf_comb_6144 60KB 60KB
zio_buf_comb_4096 224KB 320KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 63KB 63KB
zio_link_cache 19KB 31KB
zio_cache 261KB 324KB
zfs_btree_leaf_cache 4880KB 4960KB
nfs_commit_data 15KB 15KB
nfs_write_data 47KB 47KB
nfs_read_data 31KB 31KB
nfs_page 8KB 8KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
ext4_io_end_vec 8KB 8KB
jbd2_inode 8KB 8KB
0000:02:00.0-mlx5_fs_ftes 15KB 15KB
0000:02:00.0-mlx5_fs_fgs 15KB 15KB
scsi_sense_cache 24KB 24KB
bio-144 23KB 23KB
fib6_nodes 8KB 8KB
ip6_dst_cache 7KB 7KB
PINGv6 15KB 15KB
RAWv6 31KB 31KB
UDPv6 63KB 63KB
TCPv6 59KB 59KB
mqueue_inode_cache 16KB 16KB
kioctx 15KB 15KB
aio_kiocb 7KB 7KB
posix_timers_cache 7KB 7KB
UNIX-STREAM 167KB 195KB
UNIX 75KB 75KB
tcp_bind_bucket 8KB 8KB
ip_fib_trie 7KB 7KB
ip_fib_alias 7KB 7KB
ip_dst_cache 52KB 52KB
PING 15KB 15KB
RAW 31KB 31KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 150KB 150KB
hugetlbfs_inode_cache 15KB 15KB
bio-272 15KB 15KB
ep_head 8KB 8KB
eventpoll_pwq 16KB 16KB
eventpoll_epi 28KB 28KB
inotify_inode_mark 15KB 15KB
sgpool-128 160KB 160KB
sgpool-64 64KB 64KB
sgpool-32 32KB 32KB
sgpool-16 16KB 16KB
sgpool-8 8KB 8KB
bio_integrity_data 3KB 3KB
request_queue 31KB 31KB
blkdev_ioc 7KB 7KB
bio-208 48KB 48KB
biovec-max 312KB 352KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
khugepaged_mm_slot 7KB 7KB
user_namespace 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 80KB 80KB
skbuff_head_cache 32KB 32KB
configfs_dir_cache 3KB 3KB
file_lock_cache 7KB 7KB
file_lock_ctx 7KB 7KB
fsnotify_mark_connector 8KB 8KB
taskstats 15KB 15KB
proc_dir_entry 232KB 232KB
pde_opener 7KB 7KB
seq_file 7KB 7KB
sigqueue 7KB 7KB
shmem_inode_cache 1046KB 1054KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4694KB 4713KB
mnt_cache 225KB 225KB
filp 271KB 348KB
names_cache 128KB 128KB
net_namespace 30KB 30KB
lsm_inode_cache 910KB 1060KB
lsm_file_cache 27KB 36KB
key_jar 24KB 24KB
uts_namespace 15KB 15KB
nsproxy 7KB 7KB
vma_lock 185KB 247KB
vm_area_struct 963KB 988KB
fs_cache 20KB 20KB
files_cache 94KB 94KB
signal_cache 757KB 802KB
sighand_cache 1311KB 1361KB
task_struct 4683KB 4872KB
cred_jar 143KB 165KB
anon_vma_chain 159KB 196KB
anon_vma 192KB 194KB
pid 132KB 137KB
Acpi-Operand 122KB 122KB
Acpi-ParseExt 7KB 7KB
Acpi-Parse 7KB 7KB
Acpi-State 23KB 23KB
Acpi-Namespace 15KB 15KB
numa_policy 3KB 3KB
perf_event 30KB 30KB
trace_event_file 201KB 201KB
ftrace_event_field 286KB 286KB
pool_workqueue 120KB 120KB
maple_node 151KB 232KB
task_group 79KB 79KB
mm_struct 189KB 189KB
vmap_area 458KB 543KB
kmalloc-cg-4k 128KB 128KB
kmalloc-cg-2k 512KB 512KB
kmalloc-cg-1k 128KB 128KB
kmalloc-cg-512 32KB 32KB
kmalloc-cg-256 12KB 12KB
kmalloc-cg-192 31KB 31KB
kmalloc-cg-128 8KB 8KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 8KB 8KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 16KB 16KB
kmalloc-8k 7464KB 8192KB
kmalloc-4k 14656KB 14656KB
kmalloc-2k 2792KB 2880KB
kmalloc-1k 9485KB 9536KB
kmalloc-512 3731KB 6000KB
kmalloc-256 631KB 708KB
kmalloc-192 2757KB 2835KB
kmalloc-128 456KB 912KB
kmalloc-96 1010KB 1630KB
kmalloc-64 1396KB 1728KB
kmalloc-32 582KB 872KB
kmalloc-16 162KB 168KB
kmalloc-8 59KB 60KB
kmem_cache_node 24KB 24KB
kmem_cache 88KB 88KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 528] 0 528 7141 160 90112 224 -250 systemd-journal
[ 543] 0 543 9078 224 90112 256 -1000 systemd-udevd
[ 601] 32 601 3708 192 65536 128 0 rpcbind
[ 602] 0 602 4963 169 53248 160 -1000 auditd
[ 634] 81 634 2684 224 57344 0 -900 dbus-broker-lau
[ 635] 81 635 1262 128 49152 32 -900 dbus-broker
[ 636] 0 636 65240 544 143360 704 0 NetworkManager
[ 639] 0 639 20695 256 61440 0 0 irqbalance
[ 641] 0 641 5262 224 77824 192 0 systemd-logind
[ 645] 997 645 21450 42 65536 160 0 chronyd
[ 670] 0 670 20267 96 57344 32 0 rhsmcertd
[ 673] 0 673 2973 256 57344 192 -1000 sshd
[ 676] 0 676 14170 179 86016 160 0 gssproxy
[ 678] 996 678 20093 165 73728 320 0 munged
[ 709] 0 709 41214 256 90112 416 0 rsyslogd
[ 715] 29 715 2857 133 61440 160 0 rpc.statd
[ 764] 0 764 85089 168 122880 448 0 automount
[ 774] 0 774 766 64 45056 0 0 agetty
[ 775] 0 775 1408 32 53248 32 0 agetty
[ 813] 0 813 2166 192 53248 96 0 crond
[ 1254] 0 1254 4454 384 73728 352 0 sshd-session
[ 1258] 0 1258 5695 256 81920 256 100 systemd
[ 1260] 0 1260 26908 68 90112 896 100 (sd-pam)
[ 1267] 0 1267 4519 333 77824 480 0 sshd-session
[ 73206] 0 73206 4453 288 73728 448 0 sshd-session
[ 73209] 0 73209 4518 269 73728 544 0 sshd-session
[ 179067] 0 179067 4453 256 77824 448 0 sshd-session
[ 179070] 0 179070 4592 286 77824 576 0 sshd-session
[ 196328] 0 196328 1052 96 57344 0 0 bash
[ 196337] 0 196337 1052 75 57344 32 0 bash
[ 196338] 0 196338 1052 44 53248 32 0 bash
[ 196339] 0 196339 741 32 49152 0 0 xargs
[ 196340] 0 196340 672 32 45056 0 0 tr

Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 1 PID: 13445 Comm: ll_ost00_000 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
vpanic+0x129/0x2f0
panic+0x58/0x60
out_of_memory+0x2ae/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>

Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanityn test_71b ORI-366\/LU-1941: FIEMAP unimplemented on ZFS
Lustre: DEBUG MARKER: SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS
Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 10:08:49 \(1784196529\)
obd_memory max: 174080419, obd_memory current: 152729579
obd_memory max: 174080419, obd_memory current: 152729579
ll_ost00_000 invoked oom-killer: gfp_mask=0x42cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP), order=0, oom_score_adj=0
CPU: 1 PID: 13445 Comm: ll_ost00_000 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
dump_header+0x49/0x240
out_of_memory+0x289/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>
Mem-Info:
active_anon:3481 inactive_anon:5240 isolated_anon:0
Node 0 active_anon:13924kB inactive_anon:20960kB active_file:8944kB inactive_file:18076kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:17944kB dirty:60kB writeback:12kB shmem:36kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:9888kB pagetables:2008kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:14648kB boost:0kB min:280kB low:348kB high:416kB reserved_highatomic:0KB active_anon:20kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1626 3598 3598 3598
Node 0 DMA32 free:43452kB boost:26624kB min:57040kB low:64644kB high:72248kB reserved_highatomic:2048KB active_anon:2168kB inactive_anon:2440kB active_file:996kB inactive_file:396kB unevictable:0kB writepending:0kB present:2080624kB managed:1706276kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 1972 1972 1972
Node 0 Normal free:21672kB boost:82984kB min:119868kB low:129088kB high:138308kB reserved_highatomic:0KB active_anon:11968kB inactive_anon:18540kB active_file:7948kB inactive_file:18016kB unevictable:0kB writepending:324kB present:2097152kB managed:2019572kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 2*4kB (UM) 2*8kB (UM) 2*16kB (UE) 3*32kB (UME) 2*64kB (ME) 2*128kB (ME) 3*256kB (UME) 2*512kB (ME) 2*1024kB (ME) 1*2048kB (E) 2*4096kB (M) = 14616kB
Node 0 DMA32: 887*4kB (UME) 535*8kB (UME) 438*16kB (UME) 330*32kB (UME) 168*64kB (UME) 60*128kB (UME) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 43828kB
Node 0 Normal: 2009*4kB (UME) 814*8kB (UME) 317*16kB (UME) 71*32kB (UM) 5*64kB (UM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 22212kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
7025 total pagecache pages
261 pages in swap cache
Free swap = 3789456kB
Total swap = 3807228kB
1048442 pages RAM
0 pages HighMem/MovableOnly
113140 pages reserved
0 pages cma reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
ofd_obj 92KB 298KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 121KB 485KB
vvp_thread_kmem 15KB 15KB
vvp_session_kmem 7KB 7KB
ll_thread_kmem 7KB 7KB
lov_session_kmem 15KB 15KB
lov_thread_kmem 7KB 7KB
osc_session_kmem 7KB 7KB
osc_thread_kmem 61KB 61KB
tgt_fmd_cache 27KB 43KB
tgt_session_kmem 62KB 62KB
tgt_thread_kmem 226KB 226KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 289KB 492KB
ldlm_locks 566KB 1016KB
ldlm_resources 283KB 484KB
lnet_msg 63KB 63KB
lnet_rspt 1532KB 1532KB
lnet_small_MDs 12KB 35KB
lnet_MEs 47KB 47KB
ptlrpc_cache 802KB 802KB
cl_env_kmem 3KB 3KB
ll_obd_dev_cache 320KB 320KB
bio-184 31KB 31KB
bio-264 72KB 93KB
zfs_znode_hold_cache 7KB 7KB
zap_attr_long_cache 30KB 30KB
zap_attr_cache 7KB 7KB
zap_name 11KB 11KB
dbuf_dirty_record_t 8867KB 8867KB
dmu_buf_impl_t 9934KB 9934KB
arc_buf_t 744KB 744KB
zio_buf_comb_16384 4944KB 5280KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_8192 5256KB 5600KB
zio_buf_comb_6144 60KB 60KB
zio_buf_comb_4096 188KB 288KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 63KB 63KB
zio_link_cache 15KB 31KB
zio_cache 252KB 339KB
zfs_btree_leaf_cache 5048KB 5216KB
nfs_commit_data 15KB 15KB
nfs_write_data 47KB 47KB
nfs_read_data 31KB 31KB
nfs_page 8KB 8KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
ext4_io_end_vec 8KB 8KB
jbd2_inode 8KB 8KB
scsi_sense_cache 24KB 24KB
bio-144 19KB 19KB
fib6_nodes 8KB 8KB
ip6_dst_cache 7KB 7KB
PINGv6 15KB 15KB
RAWv6 31KB 31KB
UDPv6 63KB 63KB
TCPv6 59KB 59KB
mqueue_inode_cache 16KB 16KB
kioctx 15KB 15KB
aio_kiocb 7KB 7KB
posix_timers_cache 7KB 7KB
UNIX-STREAM 133KB 180KB
UNIX 60KB 60KB
tcp_bind_bucket 8KB 8KB
inet_peer_cache 3KB 3KB
ip_fib_trie 7KB 7KB
ip_fib_alias 7KB 7KB
ip_dst_cache 36KB 36KB
PING 15KB 15KB
RAW 15KB 15KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 120KB 120KB
hugetlbfs_inode_cache 15KB 15KB
bio-272 18KB 18KB
ep_head 8KB 8KB
eventpoll_pwq 16KB 16KB
eventpoll_epi 28KB 28KB
inotify_inode_mark 15KB 15KB
sgpool-128 128KB 128KB
sgpool-64 64KB 64KB
sgpool-32 32KB 32KB
sgpool-16 16KB 16KB
sgpool-8 8KB 8KB
bio_integrity_data 3KB 3KB
request_queue 31KB 31KB
blkdev_ioc 3KB 3KB
bio-208 44KB 60KB
biovec-max 320KB 320KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
khugepaged_mm_slot 7KB 7KB
user_namespace 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 72KB 72KB
skbuff_head_cache 32KB 32KB
configfs_dir_cache 3KB 3KB
file_lock_cache 7KB 7KB
file_lock_ctx 7KB 7KB
fsnotify_mark_connector 8KB 8KB
taskstats 15KB 15KB
proc_dir_entry 225KB 232KB
pde_opener 7KB 7KB
seq_file 7KB 7KB
sigqueue 7KB 7KB
shmem_inode_cache 1006KB 1070KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4439KB 4465KB
mnt_cache 221KB 221KB
filp 268KB 344KB
names_cache 128KB 128KB
net_namespace 60KB 60KB
lsm_inode_cache 778KB 940KB
lsm_file_cache 26KB 36KB
key_jar 28KB 28KB
uts_namespace 15KB 15KB
nsproxy 7KB 7KB
vma_lock 187KB 231KB
vm_area_struct 957KB 1015KB
fs_cache 16KB 16KB
files_cache 110KB 110KB
signal_cache 732KB 771KB
sighand_cache 1284KB 1330KB
task_struct 5472KB 5657KB
cred_jar 141KB 153KB
anon_vma_chain 163KB 188KB
anon_vma 180KB 182KB
pid 129KB 137KB
Acpi-Operand 122KB 122KB
Acpi-ParseExt 7KB 7KB
Acpi-Parse 7KB 7KB
Acpi-State 23KB 23KB
Acpi-Namespace 15KB 15KB
numa_policy 3KB 3KB
perf_event 30KB 30KB
trace_event_file 189KB 189KB
ftrace_event_field 270KB 270KB
pool_workqueue 104KB 104KB
maple_node 135KB 224KB
task_group 79KB 79KB
mm_struct 189KB 189KB
vmap_area 463KB 547KB
kmalloc-cg-4k 96KB 96KB
kmalloc-cg-2k 460KB 512KB
kmalloc-cg-1k 89KB 128KB
kmalloc-cg-512 32KB 32KB
kmalloc-cg-256 12KB 12KB
kmalloc-cg-192 35KB 35KB
kmalloc-cg-128 8KB 8KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 8KB 8KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 16KB 16KB
kmalloc-8k 6544KB 7072KB
kmalloc-4k 14368KB 14368KB
kmalloc-2k 2698KB 2816KB
kmalloc-1k 9485KB 9600KB
kmalloc-512 3002KB 3928KB
kmalloc-256 621KB 704KB
kmalloc-192 2666KB 2748KB
kmalloc-128 450KB 920KB
kmalloc-96 1001KB 1602KB
kmalloc-64 1340KB 1512KB
kmalloc-32 409KB 640KB
kmalloc-16 146KB 152KB
kmalloc-8 54KB 56KB
kmem_cache_node 24KB 24KB
kmem_cache 88KB 88KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 512] 0 512 7141 992 90112 0 -250 systemd-journal
[ 527] 0 527 9051 1152 90112 0 -1000 systemd-udevd
[ 578] 32 578 3708 800 69632 0 0 rpcbind
[ 579] 0 579 4963 452 57344 0 -1000 auditd
[ 609] 81 609 2684 608 57344 0 -900 dbus-broker-lau
[ 610] 81 610 1262 320 45056 0 -900 dbus-broker
[ 611] 0 611 65214 2080 147456 0 0 NetworkManager
[ 614] 0 614 20687 576 57344 0 0 irqbalance
[ 616] 0 616 5262 1088 81920 0 0 systemd-logind
[ 619] 997 619 21450 426 65536 0 0 chronyd
[ 639] 0 639 2974 1184 65536 64 -1000 sshd
[ 640] 0 640 20267 352 53248 0 0 rhsmcertd
[ 643] 996 643 20093 1095 73728 32 0 munged
[ 645] 0 645 14170 312 86016 224 0 gssproxy
[ 664] 0 664 41279 1216 94208 0 0 rsyslogd
[ 665] 29 665 2857 393 61440 64 0 rpc.statd
[ 724] 0 724 85088 1481 131072 0 0 automount
[ 732] 0 732 766 256 45056 0 0 agetty
[ 733] 0 733 1408 288 49152 0 0 agetty
[ 761] 0 761 2166 512 57344 0 0 crond
[ 1214] 0 1214 4453 1536 73728 256 0 sshd-session
[ 1218] 0 1218 5692 1152 86016 0 100 systemd
[ 1220] 0 1220 26904 582 90112 576 100 (sd-pam)
[ 1227] 0 1227 4518 1340 73728 256 0 sshd-session
[ 73072] 0 73072 4453 1824 69632 0 0 sshd-session
[ 73075] 0 73075 4551 1439 69632 128 0 sshd-session
[ 178770] 0 178770 4453 1696 77824 128 0 sshd-session
[ 178773] 0 178773 4551 1375 77824 256 0 sshd-session
[ 196308] 0 196308 1819 576 61440 0 0 bash
[ 196334] 0 196334 1819 355 57344 0 0 bash
[ 196335] 0 196335 2573 1920 57344 0 0 tclsh

Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 1 PID: 1332216 Comm: ll_ost00_016 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
vpanic+0x129/0x2f0
panic+0x58/0x60
out_of_memory+0x2ae/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>

Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanityn test_71d ORI-366\/LU-1941: FIEMAP unimplemented on ZFS
Lustre: DEBUG MARKER: SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS
obd_memory max: 152591679, obd_memory current: 149076855
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149093759
obd_memory max: 152591679, obd_memory current: 149093759
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149078359
obd_memory max: 152591679, obd_memory current: 149078359
ll_ost00_016 invoked oom-killer: gfp_mask=0x42cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP), order=0, oom_score_adj=0
CPU: 1 PID: 1332216 Comm: ll_ost00_016 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
dump_header+0x49/0x240
out_of_memory+0x289/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>
Mem-Info:
active_anon:1 inactive_anon:0 isolated_anon:0
Node 0 active_anon:4kB inactive_anon:0kB active_file:92kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:0kB writeback:0kB shmem:0kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:8848kB pagetables:1900kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:14672kB boost:0kB min:280kB low:348kB high:416kB reserved_highatomic:0KB active_anon:4kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1626 3598 3598 3598
Node 0 DMA32 free:37452kB boost:0kB min:30416kB low:38020kB high:45624kB reserved_highatomic:2048KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:2080624kB managed:1706276kB mlocked:0kB bounce:0kB free_pcp:800kB local_pcp:800kB free_cma:0kB
lowmem_reserve[]: 0 0 1972 1972 1972
Node 0 Normal free:18032kB boost:0kB min:36884kB low:46104kB high:55324kB reserved_highatomic:2048KB active_anon:0kB inactive_anon:0kB active_file:12kB inactive_file:24kB unevictable:0kB writepending:0kB present:2097152kB managed:2019572kB mlocked:0kB bounce:0kB free_pcp:672kB local_pcp:424kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 2*4kB (UE) 3*8kB (UE) 5*16kB (UE) 4*32kB (UE) 4*64kB (UE) 3*128kB (UE) 4*256kB (UE) 3*512kB (UE) 3*1024kB (UE) 2*2048kB (UM) 1*4096kB (M) = 14704kB
Node 0 DMA32: 349*4kB (UME) 324*8kB (UME) 233*16kB (UME) 252*32kB (UME) 127*64kB (UME) 44*128kB (UME) 8*256kB (UM) 4*512kB (M) 4*1024kB (M) 0*2048kB 0*4096kB = 37732kB
Node 0 Normal: 39*4kB (UE) 3*8kB (UE) 8*16kB (E) 2*32kB (UE) 2*64kB (UE) 1*128kB (E) 1*256kB (E) 1*512kB (E) 2*1024kB (UE) 7*2048kB (UE) 0*4096kB = 17780kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
41 total pagecache pages
0 pages in swap cache
Free swap = 3756540kB
Total swap = 3809276kB
1048442 pages RAM
0 pages HighMem/MovableOnly
113140 pages reserved
0 pages cma reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
ofd_obj 56KB 127KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 102KB 192KB
upd_kmem 3KB 3KB
lqe_kmem 39KB 39KB
vvp_thread_kmem 15KB 15KB
vvp_session_kmem 7KB 7KB
ll_thread_kmem 7KB 7KB
lov_session_kmem 15KB 15KB
lov_thread_kmem 7KB 7KB
osc_session_kmem 7KB 7KB
osc_thread_kmem 61KB 61KB
tgt_fmd_cache 7KB 7KB
tgt_session_kmem 46KB 46KB
tgt_thread_kmem 181KB 181KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 64KB 140KB
ldlm_locks 135KB 392KB
ldlm_resources 89KB 144KB
lnet_msg 63KB 63KB
lnet_rspt 20KB 40KB
lnet_small_MDs 43KB 43KB
lnet_MEs 48KB 51KB
ptlrpc_cache 670KB 787KB
cl_env_kmem 3KB 3KB
ll_obd_dev_cache 213KB 213KB
bio-184 35KB 35KB
bio-264 83KB 86KB
zfs_znode_hold_cache 7KB 7KB
zap_attr_long_cache 30KB 30KB
zap_attr_cache 7KB 7KB
zap_name 7KB 7KB
zil_zcw_cache 7KB 7KB
zil_lwb_cache 15KB 15KB
dbuf_dirty_record_t 9323KB 9323KB
dmu_buf_impl_t 10120KB 10155KB
arc_buf_t 779KB 780KB
zio_buf_comb_16384 3616KB 3744KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_8192 4192KB 4192KB
zio_buf_comb_6144 60KB 60KB
zio_buf_comb_4096 184KB 288KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 63KB 63KB
zio_link_cache 107KB 127KB
zio_cache 2319KB 2759KB
zfs_btree_leaf_cache 3580KB 3776KB
nfs_commit_data 15KB 15KB
nfs_write_data 47KB 47KB
nfs_read_data 31KB 31KB
nfs_page 8KB 8KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
ext4_io_end_vec 8KB 8KB
jbd2_inode 8KB 8KB
scsi_sense_cache 24KB 24KB
bio-144 11KB 11KB
fib6_nodes 8KB 8KB
ip6_dst_cache 7KB 7KB
PINGv6 31KB 31KB
RAWv6 31KB 31KB
UDPv6 63KB 63KB
TCPv6 59KB 59KB
mqueue_inode_cache 16KB 16KB
kioctx 15KB 15KB
aio_kiocb 7KB 7KB
posix_timers_cache 7KB 7KB
UNIX-STREAM 145KB 180KB
UNIX 60KB 60KB
tcp_bind_bucket 8KB 8KB
ip_fib_trie 7KB 7KB
ip_fib_alias 7KB 7KB
ip_dst_cache 28KB 28KB
PING 31KB 31KB
RAW 15KB 15KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 180KB 300KB
hugetlbfs_inode_cache 15KB 15KB
bio-272 15KB 15KB
ep_head 8KB 8KB
eventpoll_pwq 28KB 28KB
eventpoll_epi 52KB 52KB
inotify_inode_mark 15KB 15KB
sgpool-128 128KB 128KB
sgpool-64 64KB 64KB
sgpool-32 32KB 32KB
sgpool-16 16KB 16KB
sgpool-8 8KB 8KB
bio_integrity_data 3KB 3KB
request_queue 31KB 31KB
blkdev_ioc 7KB 7KB
bio-208 36KB 56KB
biovec-max 296KB 352KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
khugepaged_mm_slot 7KB 7KB
user_namespace 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 80KB 80KB
skbuff_head_cache 28KB 44KB
configfs_dir_cache 3KB 3KB
file_lock_cache 7KB 7KB
file_lock_ctx 19KB 19KB
fsnotify_mark_connector 8KB 8KB
taskstats 15KB 15KB
proc_dir_entry 192KB 192KB
pde_opener 7KB 7KB
seq_file 7KB 7KB
sigqueue 7KB 7KB
shmem_inode_cache 1032KB 1131KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4316KB 4350KB
mnt_cache 210KB 210KB
filp 258KB 336KB
names_cache 160KB 160KB
net_namespace 30KB 30KB
lsm_inode_cache 610KB 832KB
lsm_file_cache 25KB 32KB
key_jar 28KB 28KB
uts_namespace 15KB 15KB
nsproxy 7KB 7KB
vma_lock 177KB 227KB
vm_area_struct 866KB 868KB
fs_cache 16KB 16KB
files_cache 116KB 126KB
signal_cache 663KB 663KB
sighand_cache 1144KB 1144KB
task_struct 4935KB 4996KB
cred_jar 129KB 133KB
anon_vma_chain 153KB 172KB
anon_vma 165KB 178KB
pid 118KB 118KB
Acpi-Operand 122KB 122KB
Acpi-ParseExt 7KB 7KB
Acpi-Parse 7KB 7KB
Acpi-State 23KB 23KB
Acpi-Namespace 19KB 19KB
numa_policy 3KB 3KB
perf_event 30KB 30KB
trace_event_file 193KB 193KB
ftrace_event_field 274KB 274KB
pool_workqueue 120KB 120KB
maple_node 142KB 240KB
task_group 79KB 79KB
mm_struct 189KB 189KB
vmap_area 392KB 547KB
kmalloc-cg-4k 96KB 96KB
kmalloc-cg-2k 544KB 544KB
kmalloc-cg-1k 128KB 128KB
kmalloc-cg-512 56KB 56KB
kmalloc-cg-256 8KB 8KB
kmalloc-cg-192 27KB 27KB
kmalloc-cg-128 8KB 8KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 12KB 12KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 14KB 16KB
kmalloc-8k 5224KB 5504KB
kmalloc-4k 14560KB 14560KB
kmalloc-2k 2252KB 2304KB
kmalloc-1k 7641KB 7920KB
kmalloc-512 3743KB 4376KB
kmalloc-256 867KB 912KB
kmalloc-192 5251KB 5288KB
kmalloc-128 611KB 828KB
kmalloc-96 931KB 1386KB
kmalloc-64 1174KB 1372KB
kmalloc-32 369KB 524KB
kmalloc-16 149KB 152KB
kmalloc-8 51KB 52KB
kmem_cache_node 28KB 28KB
kmem_cache 100KB 100KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 512] 0 512 11410 64 102400 480 -250 systemd-journal
[ 527] 0 527 9116 12 98304 544 -1000 systemd-udevd
[ 577] 32 577 3708 64 65536 224 0 rpcbind
[ 578] 0 578 4963 67 61440 224 -1000 auditd
[ 608] 81 608 2684 64 61440 128 -900 dbus-broker-lau
[ 609] 81 609 1262 32 45056 96 -900 dbus-broker
[ 610] 0 610 65251 32 143360 832 0 NetworkManager
[ 613] 0 613 20687 32 57344 96 0 irqbalance
[ 615] 0 615 7311 64 81920 320 0 systemd-logind
[ 621] 997 621 21450 79 69632 128 0 chronyd
[ 635] 0 635 20267 64 53248 0 0 rhsmcertd
[ 637] 0 637 2974 64 61440 256 -1000 sshd
[ 643] 996 643 20093 69 73728 320 0 munged
[ 645] 0 645 14170 79 81920 224 0 gssproxy
[ 660] 0 660 45311 0 94208 512 0 rsyslogd
[ 662] 29 662 2857 71 65536 160 0 rpc.statd
[ 731] 0 731 85090 68 122880 480 0 automount
[ 740] 0 740 766 32 49152 32 0 agetty
[ 741] 0 741 1408 32 53248 32 0 agetty
[ 773] 0 773 2167 64 53248 192 0 crond
[ 1221] 0 1221 4454 64 77824 480 0 sshd-session
[ 1225] 0 1225 5694 32 81920 448 100 systemd
[ 1227] 0 1227 26906 47 94208 896 100 (sd-pam)
[ 1234] 0 1234 4519 64 77824 576 0 sshd-session
[1350278] 0 1350278 4453 64 81920 480 0 sshd-session
[1350281] 0 1350281 4551 62 77824 576 0 sshd-session
[1367257] 0 1367257 1819 32 53248 64 0 bash
[1367298] 0 1367298 1819 52 45056 64 0 bash
[1367299] 0 1367299 1819 0 45056 100 0 bash

Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 1 PID: 34508 Comm: ll_ost00_008 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
vpanic+0x129/0x2f0
panic+0x58/0x60
out_of_memory+0x2ae/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>

obd_memory max: 177877471, obd_memory current: 152545339
ll_ost00_008 invoked oom-killer: gfp_mask=0x42cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP), order=0, oom_score_adj=0
CPU: 1 PID: 34508 Comm: ll_ost00_008 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
dump_header+0x49/0x240
out_of_memory+0x289/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>
Mem-Info:
active_anon:0 inactive_anon:16 isolated_anon:0
Node 0 active_anon:0kB inactive_anon:64kB active_file:64kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:52kB dirty:0kB writeback:28kB shmem:32kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:9872kB pagetables:1880kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:14660kB boost:0kB min:280kB low:348kB high:416kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1678 3598 3598 3598
Node 0 DMA32 free:38440kB boost:0kB min:31388kB low:39232kB high:47076kB reserved_highatomic:0KB active_anon:0kB inactive_anon:60kB active_file:0kB inactive_file:500kB unevictable:0kB writepending:0kB present:2080624kB managed:1752944kB mlocked:0kB bounce:0kB free_pcp:1600kB local_pcp:828kB free_cma:0kB
lowmem_reserve[]: 0 0 1920 1920 1920
Node 0 Normal free:24304kB boost:4096kB min:40008kB low:48984kB high:57960kB reserved_highatomic:2048KB active_anon:0kB inactive_anon:92kB active_file:16kB inactive_file:168kB unevictable:0kB writepending:448kB present:2097152kB managed:1972912kB mlocked:0kB bounce:0kB free_pcp:1528kB local_pcp:952kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 1*4kB (E) 2*8kB (UE) 3*16kB (UE) 2*32kB (UE) 1*64kB (E) 1*128kB (U) 2*256kB (UE) 1*512kB (E) 1*1024kB (E) 2*2048kB (ME) 2*4096kB (M) = 14660kB
Node 0 DMA32: 575*4kB (UME) 363*8kB (UME) 305*16kB (ME) 250*32kB (UME) 162*64kB (UME) 56*128kB (UME) 2*256kB (UM) 5*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 38692kB
Node 0 Normal: 1409*4kB (UME) 759*8kB (UME) 356*16kB (UME) 131*32kB (UME) 29*64kB (ME) 1*128kB (U) 4*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 24604kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
38 total pagecache pages
0 pages in swap cache
Free swap = 3762408kB
Total swap = 3809276kB
1048442 pages RAM
0 pages HighMem/MovableOnly
113138 pages reserved
0 pages cma reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
ofd_obj 76KB 231KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 126KB 385KB
vvp_thread_kmem 15KB 15KB
vvp_session_kmem 7KB 7KB
ll_thread_kmem 7KB 7KB
lov_session_kmem 15KB 15KB
lov_thread_kmem 7KB 7KB
osc_session_kmem 7KB 7KB
osc_thread_kmem 61KB 61KB
tgt_fmd_cache 27KB 43KB
tgt_session_kmem 54KB 54KB
tgt_thread_kmem 226KB 226KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 283KB 352KB
ldlm_locks 602KB 856KB
ldlm_resources 283KB 356KB
lnet_msg 71KB 71KB
lnet_rspt 432KB 432KB
lnet_small_MDs 43KB 43KB
lnet_MEs 47KB 51KB
ptlrpc_cache 752KB 787KB
cl_env_kmem 3KB 3KB
ll_obd_dev_cache 347KB 347KB
bio-184 31KB 31KB
bio-264 77KB 105KB
zfs_znode_hold_cache 7KB 7KB
zap_attr_long_cache 30KB 30KB
zap_attr_cache 7KB 7KB
zap_name 19KB 19KB
dbuf_dirty_record_t 8890KB 8890KB
dmu_buf_impl_t 9992KB 10040KB
arc_buf_t 760KB 764KB
zio_buf_comb_16384 4816KB 5472KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_8192 4384KB 5504KB
zio_buf_comb_6144 60KB 60KB
zio_buf_comb_4096 256KB 352KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 63KB 63KB
zio_link_cache 47KB 51KB
zio_cache 1169KB 1269KB
zfs_btree_leaf_cache 3544KB 4480KB
nfs_commit_data 31KB 31KB
nfs_write_data 95KB 175KB
nfs_read_data 31KB 31KB
nfs_page 44KB 44KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
ext4_io_end_vec 8KB 8KB
jbd2_inode 8KB 8KB
scsi_sense_cache 24KB 24KB
bio-144 7KB 7KB
fib6_nodes 4KB 4KB
ip6_dst_cache 7KB 7KB
PINGv6 15KB 15KB
RAWv6 31KB 31KB
UDPv6 63KB 63KB
TCPv6 59KB 59KB
mqueue_inode_cache 16KB 16KB
kioctx 15KB 15KB
aio_kiocb 7KB 7KB
posix_timers_cache 7KB 7KB
UNIX-STREAM 112KB 180KB
UNIX 75KB 75KB
tcp_bind_bucket 8KB 8KB
ip_fib_trie 3KB 3KB
ip_fib_alias 3KB 3KB
ip_dst_cache 32KB 32KB
PING 15KB 15KB
RAW 15KB 15KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 150KB 150KB
hugetlbfs_inode_cache 7KB 7KB
bio-272 15KB 15KB
ep_head 8KB 8KB
eventpoll_pwq 16KB 16KB
eventpoll_epi 32KB 32KB
inotify_inode_mark 15KB 15KB
sgpool-128 128KB 128KB
sgpool-64 64KB 64KB
sgpool-32 32KB 32KB
sgpool-16 16KB 16KB
sgpool-8 8KB 8KB
bio_integrity_data 3KB 3KB
request_queue 31KB 31KB
blkdev_ioc 7KB 7KB
bio-208 44KB 64KB
biovec-max 320KB 320KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
khugepaged_mm_slot 7KB 7KB
user_namespace 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 88KB 88KB
skbuff_head_cache 40KB 40KB
configfs_dir_cache 3KB 3KB
file_lock_cache 7KB 7KB
file_lock_ctx 7KB 7KB
fsnotify_mark_connector 8KB 8KB
taskstats 15KB 15KB
proc_dir_entry 228KB 228KB
pde_opener 7KB 7KB
seq_file 7KB 7KB
sigqueue 7KB 7KB
shmem_inode_cache 1019KB 1054KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4453KB 4473KB
mnt_cache 243KB 243KB
filp 264KB 328KB
names_cache 128KB 192KB
net_namespace 30KB 30KB
lsm_inode_cache 623KB 832KB
lsm_file_cache 28KB 36KB
key_jar 24KB 24KB
uts_namespace 15KB 15KB
nsproxy 7KB 7KB
vma_lock 190KB 227KB
vm_area_struct 954KB 1080KB
fs_cache 16KB 16KB
files_cache 94KB 94KB
signal_cache 737KB 771KB
sighand_cache 1266KB 1330KB
task_struct 5516KB 5684KB
cred_jar 148KB 157KB
anon_vma_chain 157KB 188KB
anon_vma 179KB 190KB
pid 126KB 129KB
Acpi-Operand 122KB 122KB
Acpi-ParseExt 7KB 7KB
Acpi-Parse 7KB 7KB
Acpi-State 15KB 15KB
Acpi-Namespace 19KB 19KB
numa_policy 3KB 3KB
perf_event 30KB 30KB
trace_event_file 193KB 193KB
ftrace_event_field 270KB 270KB
pool_workqueue 96KB 96KB
maple_node 125KB 208KB
task_group 79KB 79KB
mm_struct 189KB 189KB
vmap_area 463KB 519KB
kmalloc-cg-8k 32KB 32KB
kmalloc-cg-4k 128KB 128KB
kmalloc-cg-2k 512KB 512KB
kmalloc-cg-1k 128KB 128KB
kmalloc-cg-512 48KB 48KB
kmalloc-cg-256 12KB 12KB
kmalloc-cg-192 31KB 31KB
kmalloc-cg-128 8KB 8KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 8KB 8KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 12KB 12KB
kmalloc-8k 6520KB 7456KB
kmalloc-4k 14704KB 14720KB
kmalloc-2k 2752KB 2816KB
kmalloc-1k 8817KB 9360KB
kmalloc-512 2798KB 4536KB
kmalloc-256 620KB 768KB
kmalloc-192 2630KB 2744KB
kmalloc-128 386KB 856KB
kmalloc-96 985KB 1661KB
kmalloc-64 1376KB 1708KB
kmalloc-32 397KB 700KB
kmalloc-16 147KB 152KB
kmalloc-8 58KB 60KB
kmem_cache_node 28KB 28KB
kmem_cache 88KB 88KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 512] 0 512 7141 64 86016 320 -250 systemd-journal
[ 528] 0 528 9152 32 102400 544 -1000 systemd-udevd
[ 579] 32 579 3708 32 65536 224 0 rpcbind
[ 580] 0 580 4963 32 53248 224 -1000 auditd
[ 610] 81 610 2684 32 61440 160 -900 dbus-broker-lau
[ 611] 81 611 1262 32 45056 96 -900 dbus-broker
[ 612] 0 612 65216 64 143360 768 0 NetworkManager
[ 615] 0 615 20687 64 57344 64 0 irqbalance
[ 617] 0 617 5263 64 90112 320 0 systemd-logind
[ 621] 997 621 21450 40 69632 128 0 chronyd
[ 636] 0 636 20267 32 53248 32 0 rhsmcertd
[ 637] 0 637 2974 32 61440 288 -1000 sshd
[ 643] 996 643 20093 37 73728 320 0 munged
[ 646] 0 646 14170 82 86016 224 0 gssproxy
[ 665] 0 665 41279 96 86016 480 0 rsyslogd
[ 667] 29 667 2857 39 53248 192 0 rpc.statd
[ 734] 0 734 85090 31 122880 512 0 automount
[ 742] 0 742 766 32 40960 32 0 agetty
[ 743] 0 743 1408 32 53248 32 0 agetty
[ 795] 0 795 2167 64 57344 192 0 crond
[ 1223] 0 1223 4454 64 69632 480 0 sshd-session
[ 1227] 0 1227 5694 32 81920 480 100 systemd
[ 1229] 0 1229 26905 62 90112 864 100 (sd-pam)
[ 1236] 0 1236 4519 32 69632 576 0 sshd-session
[ 111506] 0 111506 4453 32 77824 480 0 sshd-session
[ 111509] 0 111509 4518 39 77824 576 0 sshd-session
[ 184628] 0 184628 4453 64 73728 480 0 sshd-session
[ 184632] 0 184632 4551 61 73728 576 0 sshd-session

Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 0 PID: 1314850 Comm: ll_ost00_001 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
vpanic+0x129/0x2f0
panic+0x58/0x60
out_of_memory+0x2ae/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>

Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanityn test_71d ORI-366\/LU-1941: FIEMAP unimplemented on ZFS
obd_memory max: 157959488, obd_memory current: 151711290
obd_memory max: 157959488, obd_memory current: 151710530
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151710386
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151713394
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151714898
obd_memory max: 157959488, obd_memory current: 151716498
obd_memory max: 157959488, obd_memory current: 151716498
obd_memory max: 157959488, obd_memory current: 151716498
obd_memory max: 157959488, obd_memory current: 151716498
obd_memory max: 157959488, obd_memory current: 151716498
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
obd_memory max: 157959488, obd_memory current: 151716354
ll_ost00_001 invoked oom-killer: gfp_mask=0x42cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP), order=0, oom_score_adj=0
CPU: 0 PID: 1314850 Comm: ll_ost00_001 Kdump: loaded Tainted: P OE ------ --- 5.14.0-687.17.1.el9_8_lustre.x86_64 #1
Hardware name: Red Hat KVM/RHEL, BIOS 1.16.3-4.el9 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x33/0x50
dump_header+0x49/0x240
out_of_memory+0x289/0x330
__alloc_pages_slowpath.constprop.0+0xa70/0xc90
__alloc_pages+0x21c/0x250
alloc_pages_mpol+0x93/0x1f0
allocate_slab+0x387/0x4c0
? srso_alias_return_thunk+0x5/0xfbef5
? get_any_partial+0x6b/0x190
___slab_alloc+0x33c/0x7c0
? schedule_timeout+0x91/0x160
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
? spl_kmem_cache_free+0xff/0x1b0 [spl]
? spl_kmem_cache_alloc+0xcf/0x150 [spl]
kmem_cache_alloc+0x2d4/0x310
? srso_alias_return_thunk+0x5/0xfbef5
spl_kmem_cache_alloc+0xcf/0x150 [spl]
dbuf_dirty+0xf1/0x980 [zfs]
dnode_free_range+0x56a/0x6e0 [zfs]
dmu_free_long_range_impl+0x389/0x480 [zfs]
dmu_free_long_range+0x72/0xb0 [zfs]
osd_unlinked_object_free+0x46/0x3c0 [osd_zfs]
osd_unlinked_list_emptify+0xb0/0xc0 [osd_zfs]
osd_trans_stop+0x3d7/0x530 [osd_zfs]
ofd_destroy+0x39e/0x740 [ofd]
ofd_destroy_by_fid+0x2ef/0x480 [ofd]
ofd_destroy_hdl+0x17c/0x9e0 [ofd]
tgt_handle_request0+0x156/0x820 [ptlrpc]
? srso_alias_return_thunk+0x5/0xfbef5
tgt_request_handle+0x3b7/0xd40 [ptlrpc]
ptlrpc_server_handle_request.isra.0+0x2e2/0xd80 [ptlrpc]
? lprocfs_counter_add+0x121/0x180 [obdclass]
ptlrpc_main+0x983/0xe50 [ptlrpc]
? __pfx_ptlrpc_main+0x10/0x10 [ptlrpc]
kthread+0x122/0x240
? __pfx_kthread+0x10/0x10
? __pfx_kthread+0x10/0x10
ret_from_fork+0x28/0x50
</TASK>
Mem-Info:
active_anon:0 inactive_anon:0 isolated_anon:0
Node 0 active_anon:0kB inactive_anon:0kB active_file:44kB inactive_file:20kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:0kB writeback:0kB shmem:0kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:8864kB pagetables:1844kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:14660kB boost:0kB min:280kB low:348kB high:416kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1678 3598 3598 3598
Node 0 DMA32 free:47324kB boost:0kB min:31388kB low:39232kB high:47076kB reserved_highatomic:12288KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:188kB unevictable:0kB writepending:0kB present:2080624kB managed:1752944kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 1920 1920 1920
Node 0 Normal free:17940kB boost:0kB min:35912kB low:44888kB high:53864kB reserved_highatomic:10240KB active_anon:0kB inactive_anon:0kB active_file:32kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1972912kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 1*4kB (E) 1*8kB (U) 2*16kB (E) 1*32kB (U) 2*64kB (UE) 1*128kB (E) 2*256kB (UE) 1*512kB (E) 1*1024kB (E) 2*2048kB (UM) 2*4096kB (ME) = 14668kB
Node 0 DMA32: 190*4kB (UME) 161*8kB (UM) 200*16kB (UME) 203*32kB (UME) 122*64kB (UME) 24*128kB (ME) 10*256kB (UM) 10*512kB (UME) 13*1024kB (ME) 2*2048kB (M) 0*4096kB = 47712kB
Node 0 Normal: 467*4kB (UME) 295*8kB (UME) 140*16kB (UME) 30*32kB (UE) 14*64kB (UME) 5*128kB (UME) 0*256kB 2*512kB (ME) 2*1024kB (ME) 3*2048kB (E) 0*4096kB = 18180kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
60 total pagecache pages
0 pages in swap cache
Free swap = 3755316kB
Total swap = 3807228kB
1048442 pages RAM
0 pages HighMem/MovableOnly
113138 pages reserved
0 pages cma reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
ofd_obj 73KB 163KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 97KB 293KB
upd_kmem 3KB 3KB
lqe_kmem 47KB 47KB
vvp_thread_kmem 15KB 15KB
vvp_session_kmem 7KB 7KB
ll_thread_kmem 7KB 7KB
lov_session_kmem 15KB 15KB
lov_thread_kmem 7KB 7KB
osc_session_kmem 7KB 7KB
osc_thread_kmem 61KB 61KB
tgt_fmd_cache 7KB 7KB
tgt_session_kmem 50KB 50KB
tgt_thread_kmem 196KB 196KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 173KB 280KB
ldlm_locks 384KB 712KB
ldlm_resources 182KB 288KB
lnet_msg 63KB 63KB
lnet_rspt 960KB 960KB
lnet_small_MDs 27KB 27KB
lnet_MEs 48KB 51KB
ptlrpc_cache 849KB 849KB
cl_env_kmem 3KB 3KB
ll_obd_dev_cache 213KB 213KB
bio-184 31KB 31KB
bio-264 87KB 105KB
zfs_znode_hold_cache 7KB 7KB
zap_attr_long_cache 30KB 30KB
zap_attr_cache 7KB 7KB
zap_name 7KB 7KB
zil_zcw_cache 7KB 7KB
zil_lwb_cache 15KB 15KB
dbuf_dirty_record_t 9221KB 9221KB
dmu_buf_impl_t 10518KB 10673KB
arc_buf_t 767KB 784KB
zio_buf_comb_16384 3680KB 3680KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_8192 5312KB 5312KB
zio_buf_comb_6144 60KB 60KB
zio_buf_comb_4096 216KB 320KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 63KB 63KB
zio_link_cache 248KB 258KB
zio_cache 5700KB 6011KB
zfs_btree_leaf_cache 4068KB 4224KB
nfs_commit_data 15KB 15KB
nfs_write_data 47KB 47KB
nfs_read_data 31KB 31KB
nfs_page 8KB 8KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
ext4_io_end_vec 8KB 8KB
jbd2_inode 8KB 8KB
scsi_sense_cache 24KB 24KB
bio-144 11KB 11KB
fib6_nodes 8KB 8KB
ip6_dst_cache 7KB 7KB
PINGv6 31KB 31KB
RAWv6 31KB 31KB
UDPv6 63KB 63KB
TCPv6 59KB 59KB
mqueue_inode_cache 16KB 16KB
kioctx 15KB 15KB
aio_kiocb 7KB 7KB
posix_timers_cache 7KB 7KB
UNIX-STREAM 152KB 180KB
UNIX 75KB 75KB
tcp_bind_bucket 8KB 8KB
ip_fib_trie 7KB 7KB
ip_fib_alias 7KB 7KB
ip_dst_cache 36KB 36KB
PING 31KB 31KB
RAW 15KB 15KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 200KB 330KB
hugetlbfs_inode_cache 15KB 15KB
bio-272 15KB 15KB
ep_head 8KB 8KB
eventpoll_pwq 24KB 24KB
eventpoll_epi 48KB 48KB
inotify_inode_mark 15KB 15KB
sgpool-128 128KB 128KB
sgpool-64 64KB 64KB
sgpool-32 32KB 32KB
sgpool-16 16KB 16KB
sgpool-8 8KB 8KB
bio_integrity_data 3KB 3KB
request_queue 31KB 31KB
blkdev_ioc 7KB 7KB
bio-208 60KB 60KB
biovec-max 264KB 320KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
khugepaged_mm_slot 7KB 7KB
user_namespace 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 80KB 80KB
skbuff_head_cache 40KB 40KB
configfs_dir_cache 7KB 7KB
file_lock_cache 7KB 7KB
file_lock_ctx 15KB 15KB
fsnotify_mark_connector 8KB 8KB
taskstats 15KB 15KB
proc_dir_entry 189KB 189KB
pde_opener 7KB 7KB
seq_file 7KB 7KB
sigqueue 7KB 7KB
shmem_inode_cache 1018KB 1116KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4312KB 4350KB
mnt_cache 210KB 210KB
filp 261KB 324KB
names_cache 160KB 160KB
net_namespace 60KB 60KB
lsm_inode_cache 612KB 812KB
lsm_file_cache 25KB 32KB
key_jar 24KB 24KB
uts_namespace 15KB 15KB
nsproxy 7KB 7KB
vma_lock 174KB 215KB
vm_area_struct 857KB 891KB
fs_cache 20KB 20KB
files_cache 121KB 126KB
signal_cache 663KB 663KB
sighand_cache 1144KB 1144KB
task_struct 4996KB 5049KB
cred_jar 137KB 137KB
anon_vma_chain 144KB 168KB
anon_vma 170KB 178KB
pid 114KB 114KB
Acpi-Operand 122KB 122KB
Acpi-ParseExt 7KB 7KB
Acpi-Parse 7KB 7KB
Acpi-State 15KB 15KB
Acpi-Namespace 19KB 19KB
numa_policy 3KB 3KB
perf_event 30KB 30KB
trace_event_file 189KB 189KB
ftrace_event_field 270KB 270KB
pool_workqueue 120KB 120KB
maple_node 138KB 212KB
task_group 94KB 94KB
mm_struct 189KB 189KB
vmap_area 398KB 543KB
kmalloc-cg-4k 128KB 128KB
kmalloc-cg-2k 464KB 544KB
kmalloc-cg-1k 89KB 128KB
kmalloc-cg-512 56KB 56KB
kmalloc-cg-256 12KB 12KB
kmalloc-cg-192 43KB 43KB
kmalloc-cg-128 8KB 8KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 16KB 16KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 16KB 16KB
kmalloc-8k 5264KB 5632KB
kmalloc-4k 14432KB 14432KB
kmalloc-2k 2288KB 2336KB
kmalloc-1k 8160KB 8256KB
kmalloc-512 3842KB 4408KB
kmalloc-256 1228KB 1264KB
kmalloc-192 5260KB 5292KB
kmalloc-128 803KB 904KB
kmalloc-96 945KB 1508KB
kmalloc-64 1211KB 1352KB
kmalloc-32 373KB 584KB
kmalloc-16 146KB 152KB
kmalloc-8 53KB 56KB
kmem_cache_node 24KB 24KB
kmem_cache 100KB 100KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 511] 0 511 11438 103 94208 480 -250 systemd-journal
[ 525] 0 525 9112 79 94208 480 -1000 systemd-udevd
[ 576] 32 576 3708 32 69632 224 0 rpcbind
[ 577] 0 577 4963 32 57344 224 -1000 auditd
[ 607] 81 607 2684 64 61440 160 -900 dbus-broker-lau
[ 608] 81 608 1262 64 45056 64 -900 dbus-broker
[ 609] 0 609 65219 64 151552 768 0 NetworkManager
[ 612] 0 612 20687 64 61440 64 0 irqbalance
[ 614] 0 614 7311 64 77824 352 0 systemd-logind
[ 618] 997 618 21450 78 77824 160 0 chronyd
[ 647] 0 647 14170 26 90112 256 0 gssproxy
[ 648] 0 648 2974 64 57344 288 -1000 sshd
[ 649] 0 649 20267 32 57344 32 0 rhsmcertd
[ 662] 996 662 20093 69 73728 320 0 munged
[ 680] 0 680 61695 85 106496 320 0 rsyslogd
[ 683] 29 683 2857 44 57344 192 0 rpc.statd
[ 722] 0 722 85088 33 122880 480 0 automount
[ 733] 0 733 766 64 40960 0 0 agetty
[ 734] 0 734 1408 32 49152 32 0 agetty
[ 751] 0 751 2166 64 57344 192 0 crond
[ 1214] 0 1214 4454 64 73728 480 0 sshd-session
[ 1218] 0 1218 5692 64 81920 480 100 systemd
[ 1220] 0 1220 26907 66 94208 896 100 (sd-pam)
[ 1227] 0 1227 4519 45 73728 544 0 sshd-session
[1362864] 0 1362864 4453 32 73728 512 0 sshd-session
[1362867] 0 1362867 4551 32 73728 608 0 sshd-session
[1379849] 0 1379849 1052 32 40960 64 0 bash
[1379852] 0 1379852 1052 32 49152 32 0 grepconf.sh