Editing crashreport #70791

ReasonCrashing FunctionWhere to cut BacktraceReports Count
Kernel panic - not syncing: Out of memorypanic
out_of_memory
__alloc_pages_slowpath
__alloc_pages_nodemask
alloc_pages_vma
__read_swap_cache_async
read_swap_cache_async
swap_cluster_readahead
swapin_readahead
do_swap_page
__handle_mm_fault
handle_mm_fault
__do_page_fault
do_page_fault
page_fault
1

Added fields:

Match messages in logs
(every line would be required to be present in log output
Copy from "Messages before crash" column below):
Match messages in full crash
(every line would be required to be present in crash log output
Copy from "Full Crash" column below):
Limit to a test:
(Copy from below "Failing text"):
Delete these reports as invalid (real bug in review or some such)
Bug or comment:
Extra info:

Failures list (last 100):

Failing TestFull CrashMessages before crashComment
obdfilter-survey test 1a: Object Storage Targets survey
Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled
CPU: 0 PID: 433206 Comm: lctl Kdump: loaded Tainted: P OE --------- - - 4.18.0-477.27.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Call Trace:
dump_stack+0x41/0x60
panic+0xe7/0x2ac
out_of_memory.cold.36+0x2f/0x7e
__alloc_pages_slowpath+0xbe7/0xcd0
? blk_mq_sched_insert_request+0xd3/0x140
__alloc_pages_nodemask+0x2e2/0x330
alloc_pages_vma+0x74/0x1d0
__read_swap_cache_async+0xf4/0x2b0
read_swap_cache_async+0x28/0x70
swap_cluster_readahead+0x267/0x2f0
? xas_store+0x446/0x5a0
? __mod_memcg_lruvec_state+0x4a/0xc0
swapin_readahead+0x5c/0x501
? pagecache_get_page+0x30/0x310
do_swap_page+0x45b/0x710
? pmd_devmap_trans_unstable+0x2e/0x40
? handle_pte_fault+0x5d/0x880
__handle_mm_fault+0x453/0x6c0
handle_mm_fault+0xca/0x2a0
__do_page_fault+0x1f0/0x460
do_page_fault+0x37/0x130
? page_fault+0x8/0x30
page_fault+0x1e/0x30
RIP: 0033:0x7f79bf88ab90
Lustre: Echo OBD driver; http://www.lustre.org/
obd_memory max: 130128111, obd_memory current: 118290411
obd_memory max: 130128111, obd_memory current: 118274939
obd_memory max: 130128111, obd_memory current: 118274939
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258067
obd_memory max: 130128111, obd_memory current: 118258027
obd_memory max: 130128111, obd_memory current: 118258027
obd_memory max: 130128111, obd_memory current: 119360827
obd_memory max: 130128111, obd_memory current: 119360827
obd_memory max: 130128111, obd_memory current: 119140971
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 119140163
obd_memory max: 130128111, obd_memory current: 118979099
obd_memory max: 130128111, obd_memory current: 118979099
obd_memory max: 130128111, obd_memory current: 118979099
obd_memory max: 130128111, obd_memory current: 118979099
obd_memory max: 130128111, obd_memory current: 118979099
obd_memory max: 130128111, obd_memory current: 118979099
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118876827
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118875459
obd_memory max: 130128111, obd_memory current: 118881499
lctl invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
CPU: 0 PID: 433206 Comm: lctl Kdump: loaded Tainted: P OE --------- - - 4.18.0-477.27.1.el8_lustre.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Call Trace:
dump_stack+0x41/0x60
dump_header+0x4a/0x1df
out_of_memory.cold.36+0xa/0x7e
__alloc_pages_slowpath+0xbe7/0xcd0
? blk_mq_sched_insert_request+0xd3/0x140
__alloc_pages_nodemask+0x2e2/0x330
alloc_pages_vma+0x74/0x1d0
__read_swap_cache_async+0xf4/0x2b0
read_swap_cache_async+0x28/0x70
swap_cluster_readahead+0x267/0x2f0
? xas_store+0x446/0x5a0
? __mod_memcg_lruvec_state+0x4a/0xc0
swapin_readahead+0x5c/0x501
? pagecache_get_page+0x30/0x310
do_swap_page+0x45b/0x710
? pmd_devmap_trans_unstable+0x2e/0x40
? handle_pte_fault+0x5d/0x880
__handle_mm_fault+0x453/0x6c0
handle_mm_fault+0xca/0x2a0
__do_page_fault+0x1f0/0x460
do_page_fault+0x37/0x130
? page_fault+0x8/0x30
page_fault+0x1e/0x30
RIP: 0033:0x7f79bf88ab90
Code: Unable to access opcode bytes at RIP 0x7f79bf88ab66.
RSP: 002b:00007ffc2705fca8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 00007f79bfac5d10 RCX: 00007ffc27060210
RDX: 00007ffc2705ffd0 RSI: 00000000c008667e RDI: 0000000000000003
RBP: 00000000c008667e R08: 0000000000000240 R09: 000000000065d2e6
R10: 00007ffc271ea000 R11: 0000000000000202 R12: 0000561bd6b014ec
R13: 00007ffc2705ffd0 R14: 0000000000000000 R15: 0000000000000000
Mem-Info:
active_anon:3 inactive_anon:142 isolated_anon:75
Node 0 active_anon:12kB inactive_anon:568kB active_file:52kB inactive_file:104kB unevictable:0kB isolated(anon):300kB isolated(file):0kB mapped:1044kB dirty:20kB writeback:100kB shmem:156kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:9360kB pagetables:16808kB all_unreclaimable? no
Node 0 DMA free:10692kB min:256kB low:320kB high:384kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2609 2609 2609 2609
Node 0 DMA32 free:48840kB min:48892kB low:60088kB high:71284kB active_anon:12kB inactive_anon:592kB active_file:52kB inactive_file:104kB unevictable:0kB writepending:120kB present:3129320kB managed:2706896kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 4*16kB (U) 2*32kB (U) 3*64kB (U) 1*128kB (U) 2*256kB (U) 1*512kB (U) 1*1024kB (U) 2*2048kB (UM) 1*4096kB (M) = 10692kB
Node 0 DMA32: 571*4kB (UME) 446*8kB (UME) 409*16kB (UME) 584*32kB (UME) 216*64kB (UME) 13*128kB (UM) 3*256kB (UM) 3*512kB (UM) 0*1024kB 0*2048kB 0*4096kB = 48876kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
153 total pagecache pages
75 pages in swap cache
Swap cache stats: add 33551, delete 33481, find 548/2557
Free swap = 2732380kB
Total swap = 2853884kB
786328 pages RAM
0 pages HighMem/MovableOnly
105764 pages reserved
0 pages hwpoisoned
Unreclaimable slab info:
Name Used Total
echo_session_kmem 8KB 8KB
echo_thread_kmem 124KB 124KB
echo_object_kmem 7KB 7KB
ofd_obj 90KB 148KB
osd_zapit_cache 15KB 15KB
zfs_osd_obj 130KB 229KB
lqe_kmem 40KB 40KB
vvp_thread_kmem 52KB 52KB
vvp_session_kmem 19KB 19KB
ll_thread_kmem 15KB 15KB
lov_session_kmem 55KB 55KB
lov_thread_kmem 19KB 19KB
osc_session_kmem 30KB 30KB
osc_thread_kmem 272KB 272KB
ldlm_glimpse_work_kmem 7KB 7KB
interval_tree 80KB 80KB
interval_node 112KB 112KB
ldlm_locks 84KB 110KB
ldlm_resources 126KB 126KB
lnet_msg 104KB 111KB
lnet_rspt 31KB 31KB
lnet_small_MDs 135KB 135KB
lnet_MEs 134KB 134KB
ptlrpc_cache 175KB 189KB
tgt_fmd_cache 87KB 87KB
tgt_session_kmem 82KB 82KB
tgt_thread_kmem 246KB 246KB
cl_env_kmem 15KB 15KB
ll_obd_dev_cache 315KB 315KB
zfs_znode_hold_cache 7KB 7KB
zfs_znode_cache 31KB 31KB
zil_zcw_cache 3KB 3KB
zil_lwb_cache 7KB 7KB
dmu_buf_impl_t 1551KB 1738KB
arc_buf_t 193KB 223KB
arc_buf_hdr_t_full 2570KB 2812KB
dnode_t 2989KB 3983KB
sa_cache 106KB 168KB
abd_t 203KB 352KB
lz4_cache 64KB 64KB
zio_buf_comb_16384 3408KB 4384KB
zio_buf_comb_12288 48KB 48KB
zio_buf_comb_10240 48KB 48KB
zio_buf_comb_8192 128KB 256KB
zio_buf_comb_7168 64KB 64KB
zio_buf_comb_6144 64KB 64KB
zio_buf_comb_5120 64KB 64KB
zio_buf_comb_4096 288KB 480KB
zio_buf_comb_3584 63KB 63KB
zio_buf_comb_3072 60KB 60KB
zio_buf_comb_2560 60KB 60KB
zio_buf_comb_2048 64KB 64KB
zio_buf_comb_1536 94KB 94KB
zio_buf_comb_1024 178KB 752KB
zio_buf_comb_512 118KB 224KB
zio_link_cache 107KB 107KB
zio_cache 1876KB 1901KB
zfs_btree_leaf_cache 2176KB 2880KB
mod_hash_entries 3KB 3KB
bio-200 36KB 36KB
bio-272 108KB 112KB
nfs_commit_data 15KB 15KB
nfs_write_data 47KB 47KB
nfs_read_data 31KB 31KB
nfs_page 8KB 8KB
fscache_cookie_jar 7KB 7KB
rpc_buffers 64KB 64KB
rpc_tasks 8KB 8KB
ext4_system_zone 3KB 3KB
jbd2_inode 40KB 40KB
scsi_sense_cache 4KB 4KB
bio-160 11KB 11KB
fib6_nodes 8KB 8KB
ip6_dst_cache 7KB 7KB
RAWv6 15KB 15KB
UDPv6 63KB 63KB
TCPv6 61KB 61KB
sgpool-128 32KB 32KB
sgpool-64 32KB 32KB
sgpool-32 16KB 16KB
sgpool-16 8KB 8KB
sgpool-8 4KB 4KB
mqueue_inode_cache 16KB 16KB
kioctx 31KB 31KB
aio_kiocb 7KB 7KB
bio-248 8KB 8KB
posix_timers_cache 7KB 7KB
UNIX 330KB 330KB
tcp_bind_bucket 8KB 8KB
ip_fib_trie 3KB 3KB
ip_fib_alias 3KB 3KB
ip_dst_cache 68KB 68KB
RAW 31KB 31KB
UDP 30KB 30KB
tw_sock_TCP 7KB 7KB
request_sock_TCP 7KB 7KB
TCP 95KB 95KB
hugetlbfs_inode_cache 15KB 15KB
bio-280 11KB 11KB
eventpoll_pwq 19KB 19KB
eventpoll_epi 36KB 36KB
inotify_inode_mark 7KB 7KB
request_queue 63KB 63KB
blkdev_ioc 31KB 31KB
bio-224 36KB 56KB
biovec-max 544KB 672KB
biovec-128 64KB 64KB
biovec-64 32KB 32KB
biovec-16 8KB 8KB
bio_integrity_payload 4KB 4KB
khugepaged_mm_slot 7KB 7KB
uid_cache 7KB 7KB
dmaengine-unmap-256 30KB 30KB
dmaengine-unmap-128 15KB 15KB
dmaengine-unmap-16 3KB 3KB
dmaengine-unmap-2 4KB 4KB
audit_buffer 7KB 7KB
skbuff_fclone_cache 55KB 63KB
skbuff_head_cache 41KB 56KB
configfs_dir_cache 3KB 3KB
file_lock_cache 7KB 7KB
file_lock_ctx 7KB 7KB
fsnotify_mark_connector 7KB 7KB
net_namespace 31KB 31KB
task_delay_info 91KB 91KB
taskstats 15KB 15KB
proc_dir_entry 248KB 248KB
pde_opener 7KB 7KB
seq_file 8KB 8KB
sigqueue 7KB 7KB
shmem_inode_cache 859KB 859KB
kernfs_iattrs_cache 15KB 15KB
kernfs_node_cache 4581KB 4605KB
mnt_cache 275KB 275KB
filp 602KB 648KB
names_cache 96KB 96KB
avc_node 7KB 7KB
lsm_inode_cache 1152KB 1748KB
lsm_file_cache 60KB 60KB
key_jar 24KB 24KB
nsproxy 7KB 7KB
vm_area_struct 3162KB 3162KB
mm_struct 252KB 252KB
fs_cache 16KB 16KB
files_cache 110KB 110KB
signal_cache 720KB 720KB
sighand_cache 1175KB 1175KB
task_struct 5910KB 5910KB
cred_jar 169KB 189KB
anon_vma_chain 672KB 672KB
anon_vma 598KB 598KB
pid 140KB 140KB
Acpi-Operand 78KB 78KB
Acpi-ParseExt 3KB 3KB
Acpi-Parse 3KB 3KB
Acpi-State 11KB 11KB
Acpi-Namespace 19KB 19KB
numa_policy 3KB 3KB
perf_event 15KB 15KB
trace_event_file 154KB 154KB
ftrace_event_field 211KB 211KB
pool_workqueue 8KB 8KB
task_group 157KB 157KB
vmap_area 239KB 372KB
kmalloc-cg-4k 96KB 96KB
kmalloc-cg-2k 96KB 96KB
kmalloc-cg-1k 212KB 240KB
kmalloc-cg-512 24KB 24KB
kmalloc-cg-256 8KB 8KB
kmalloc-cg-192 23KB 23KB
kmalloc-cg-96 7KB 7KB
kmalloc-cg-64 8KB 8KB
kmalloc-cg-32 8KB 8KB
kmalloc-cg-16 8KB 8KB
kmalloc-cg-8 8KB 8KB
kmalloc-8k 4072KB 4768KB
kmalloc-4k 4756KB 5664KB
kmalloc-2k 3604KB 3808KB
kmalloc-1k 7401KB 7596KB
kmalloc-512 4470KB 4740KB
kmalloc-256 316KB 364KB
kmalloc-192 2494KB 2638KB
kmalloc-128 676KB 1056KB
kmalloc-96 953KB 1008KB
kmalloc-64 1055KB 1164KB
kmalloc-32 1250KB 1428KB
kmalloc-16 148KB 148KB
kmalloc-8 48KB 48KB
kmem_cache_node 24KB 24KB
kmem_cache 149KB 149KB
Tasks state (memory values in pages):
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 478] 0 478 22409 1 262144 270 0 systemd-journal
[ 504] 0 504 24378 1 217088 465 -1000 systemd-udevd
[ 574] 32 574 16812 0 180224 194 0 rpcbind
[ 577] 0 577 14276 0 135168 164 -1000 auditd
[ 603] 0 603 31227 0 147456 185 0 irqbalance
[ 605] 70 605 14599 0 159744 200 0 avahi-daemon
[ 606] 81 606 14142 1 159744 225 -900 dbus-daemon
[ 607] 0 607 95182 0 368640 684 0 NetworkManager
[ 610] 0 610 20392 0 184320 239 0 systemd-logind
[ 615] 995 615 35015 0 180224 189 0 chronyd
[ 618] 70 618 14533 0 147456 110 0 avahi-daemon
[ 629] 0 629 19162 1 196608 233 -1000 sshd
[ 631] 0 631 30231 0 266240 356 0 cupsd
[ 632] 0 632 121255 142 430080 4253 0 tuned
[ 636] 0 636 10449 0 102400 137 0 rhsmcertd
[ 638] 994 638 29747 0 155648 177 0 munged
[ 644] 0 644 26274 1 176128 176 0 gssproxy
[ 687] 0 687 52381 0 180224 336 0 rsyslogd
[ 694] 29 694 11291 0 126976 149 0 rpc.statd
[ 766] 0 766 12229 0 143360 145 0 xinetd
[ 795] 0 795 102322 0 286720 975 0 automount
[ 802] 0 802 6533 0 94208 50 0 atd
[ 804] 0 804 6466 1 98304 219 0 crond
[ 818] 0 818 2371 0 61440 37 0 agetty
[ 820] 0 820 1729 0 61440 32 0 agetty
[ 1072] 998 1072 402487 0 327680 2114 0 polkitd
[ 1181] 0 1181 26833 0 172032 267 0 master
[ 1190] 89 1190 30849 0 225280 273 0 qmgr
[ 5472] 0 5472 22418 1 208896 384 0 systemd
[ 5474] 0 5474 34214 0 282624 699 0 (sd-pam)
[ 7399] 0 7399 31609 1 286720 317 0 sshd
[ 7401] 0 7401 31653 1 278528 381 0 sshd
[ 10620] 0 10620 31609 1 290816 315 0 sshd
[ 10679] 0 10679 31609 1 282624 316 0 sshd
[ 33173] 0 33173 31609 1 290816 317 0 sshd
[ 33175] 0 33175 31609 1 286720 318 0 sshd
[ 271276] 89 271276 30836 0 221184 292 0 pickup
[ 402607] 0 402607 47776 0 409600 7986 0 sssd_kcm
[ 402610] 0 402610 12108 0 126976 211 0 ssh
[ 402663] 0 402663 31609 1 294912 315 0 sshd
[ 402667] 0 402667 12108 0 126976 210 0 ssh
[ 402668] 0 402668 31609 1 286720 316 0 sshd
[ 402892] 0 402892 31609 1 290816 315 0 sshd
[ 402930] 0 402930 12108 0 131072 210 0 ssh
[ 402935] 0 402935 12108 0 126976 211 0 ssh
[ 402944] 0 402944 12648 0 126976 213 0 ssh
[ 402945] 0 402945 31609 1 282624 316 0 sshd
[ 430005] 0 430005 31609 1 290816 315 0 sshd
[ 430099] 0 430099 31609 1 290816 316 0 sshd
[ 430937] 0 430937 31609 1 286720 316 0 sshd
[ 430939] 0 430939 31609 1 278528 319 0 sshd
[ 431133] 0 431133 2484 1 65536 117 0 bash
[ 431227] 0 431227 10488 0 126976 101 0 vmstat
[ 433149] 0 433149 2484 1 61440 116 0 bash
[ 433196] 0 433196 2451 1 61440 75 0 bash
[ 433197] 0 433197 8103 5 114688 78 0 lctl
[ 433198] 0 433198 8103 0 114688 90 0 lctl
[ 433199] 0 433199 8103 2 102400 80 0 lctl
[ 433200] 0 433200 8103 0 114688 104 0 lctl
[ 433201] 0 433201 8103 0 106496 85 0 lctl
[ 433202] 0 433202 8103 2 102400 81 0 lctl
[ 433203] 0 433203 8103 0 102400 85 0 lctl
[ 433204] 0 433204 8103 0 110592 84 0 lctl
[ 433205] 0 433205 8103 0 98304 91 0 lctl
[ 433206] 0 433206 8103 1 98304 84 0 lctl
[ 433207] 0 433207 8103 0 98304 92 0 lctl
[ 433208] 0 433208 8103 0 98304 84 0 lctl
[ 433209] 0 433209 8103 0 98304 88 0 lctl
[ 433210] 0 433210 8103 0 98304 94 0 lctl
[ 433211] 0 433211 8103 0 98304 94 0 lctl
[ 433213] 0 433213 8103 0 106496 111 0 lctl
[ 433214] 0 433214 8103 0 106496 104 0 lctl
[ 433215] 0 433215 8103 0 106496 105 0 lctl
[ 433216] 0 433216 8103 0 106496 97 0 lctl
[ 433217] 0 433217 8103 0 106496 107 0 lctl
[ 433218] 0 433218 8103 0 106496 112 0 lctl
[ 433219] 0 433219 8103 0 106496 97 0 lctl
[ 433220] 0 433220 8103 0 106496 114 0 lctl
[ 433221] 0 433221 8103 0 106496 103 0 lctl
[ 433222] 0 433222 8103 1 106496 87 0 lctl
[ 433224] 0 433224 8103 0 106496 114 0 lctl
[ 433225] 0 433225 8103 0 106496 113 0 lctl
[ 433226] 0 433226 8103 0 106496 100 0 lctl
[ 433227] 0 433227 8103 0 106496 114 0 lctl
[ 433228] 0 433228 8103 0 106496 125 0 lctl
[ 433229] 0 433229 8103 0 106496 84 0 lctl
[ 433230] 0 433230 8103 0 102400 96 0 lctl
[ 433231] 0 433231 8103 0 98304 84 0 lctl
[ 433232] 0 433232 8103 0 102400 96 0 lctl
[ 433233] 0 433233 8103 0 98304 84 0 lctl
[ 433234] 0 433234 8103 0 102400 95 0 lctl
[ 433235] 0 433235 8103 0 98304 89 0 lctl
[ 433236] 0 433236 8103 0 102400 95 0 lctl
[ 433237] 0 433237 8103 0 98304 102 0 lctl
[ 433238] 0 433238 8103 0 102400 89 0 lctl
[ 433239] 0 433239 8103 0 98304 87 0 lctl
[ 433240] 0 433240 8103 0 102400 85 0 lctl
[ 433241] 0 433241 8103 0 98304 92 0 lctl
[ 433242] 0 433242 8103 0 102400 95 0 lctl
[ 433243] 0 433243 8103 0 98304 110 0 lctl
[ 433244] 0 433244 8103 0 102400 85 0 lctl
[ 433245] 0 433245 8103 0 98304 93 0 lctl
[ 433246] 0 433246 8103 0 98304 95 0 lctl
[ 433247] 0 433247 8103 0 98304 98 0 lctl
[ 433248] 0 433248 8103 0 98304 84 0 lctl
[ 433249] 0 433249 8103 0 98304 84 0 lctl
[ 433250] 0 433250 8103 0 98304 105 0 lctl
[ 433251] 0 433251 8103 0 98304 97 0 lctl
[ 433253] 0 433253 8103 0 98304 93 0 lctl
[ 433254] 0 433254 8103 0 98304 110 0 lctl
[ 433256] 0 433256 8103 5 98304 80 0 lctl
[ 433257] 0 433257 8103 0 106496 100 0 lctl
[ 433258] 0 433258 8103 0 98304 103 0 lctl
[ 433259] 0 433259 8103 0 106496 96 0 lctl
[ 433260] 0 433260 8103 6 98304 79 0 lctl
[ 433261] 0 433261 8103 0 106496 84 0 lctl
[ 433262] 0 433262 8103 0 98304 108 0 lctl
[ 433263] 0 433263 8103 0 106496 102 0 lctl
[ 433264] 0 433264 8103 0 98304 101 0 lctl
[ 433265] 0 433265 8103 0 106496 99 0 lctl
[ 433266] 0 433266 8103 0 98304 94 0 lctl
[ 433267] 0 433267 8103 0 106496 84 0 lctl
[ 433268] 0 433268 8103 0 106496 84 0 lctl
Link to test
Return to new crashes list