lkml.org 
[lkml]   [2009]   [Mar]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectPROBLEM: kernel BUG at mm/slab.c:3002!
Running blktrace & I/O loads cause a kernel BUG at mm/slab.c:3002!.

I'm running some moderate I/O loads (using 12 devices behind a Smart
Array on a 16-way x86_64 box, I'm doing asynchronous direct sequential
reads to each disk in parallel), and whilst attempting to get blktrace
data I routinely run into this.

I first started seeing this in 2.6.29-rc6, so I bumped to 2.6.29-rc7 and
made a couple of successful runs, then ran into it again. (I've
attached the script I was using, but I'm not sure it's very helpful...)

The environment the system under test is in is rather difficult to
bisect in, but if need be, I can certainly go through the (painful)
motions to do so...

I only ran this a couple of times on 2.6.27.7-4 (SLESS 11 b6 kernel),
and both times it worked there - not sure how far back the problem occurs...

I'm open to any SLAB debug tracing options that may help with this...

Alan D.Brunelle
Hewlett-Packard
If some fields are empty or look unusual you may have an old version.
Compare to the current minimal requirements in Documentation/Changes.

Linux seatpost 2.6.29-rc7 #3 SMP Tue Mar 10 10:15:07 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

Gnu C 4.3
Gnu make 3.81
binutils 2.19
util-linux 2.14.1
mount support
module-init-tools 3.4
e2fsprogs 1.41.1
reiserfsprogs 3.6.19
xfsprogs 2.10.1
quota-tools 3.16.
PPP 2.4.5
Linux C Library 2.9
Dynamic linker (ldd) 2.9
Procps 3.2.7
Net-tools 1.60
Kbd 1.14.1
oprofile 0.9.4
Sh-utils 6.12
udev 128
wireless-tools 30
Modules Loaded fuse ext2 loop dm_mod sd_mod crc_t10dif bnx2 qla2xxx sg scsi_transport_fc rtc_cmos ipmi_si shpchp sr_mod rtc_core ipmi_msghandler pcspkr rtc_lib serio_raw hpwdt pci_hotplug cdrom button scsi_tgt container hpilo usbhid hid ehci_hcd uhci_hcd ohci_hcd usbcore edd ext3 mbcache jbd fan ide_pci_generic amd74xx ide_core pata_amd thermal processor thermal_sys hwmon cciss ata_generic libata scsi_mod
#! /usr/bin/env python

import os, sys, tempfile, time
import parse_aiod

dsfs = 'sdy sdz sdaa sdab sdac sdad sdae sdaf sdag sdah sdai sdaj'.split()

def run_aiod(bs, dsfs, rnd):
def gen_aiod_cmd(bs, dsfs):
aiod_cmd= '/usr/local/bin/aiod'
opts = '-A 256 -D -I -i 10000000 -l -v'
xopts = '-R -b %dk -T 30' % bs
cmd = '%s %s %s' % (aiod_cmd, opts, xopts)
if rnd:
cmd = '%s -z' % cmd
for path in ['/dev/%s' % dsf for dsf in dsfs]:
cmd = '%s %s' % (cmd, path)
return cmd

(fd, fn) = tempfile.mkstemp()
fo = os.fdopen(fd, 'w')

pid = os.fork()
if pid == 0:
os.dup2(fo.fileno(), sys.stdout.fileno())
os.dup2(fo.fileno(), sys.stderr.fileno())
fo.close()

cmd = gen_aiod_cmd(bs, dsfs).split(None)
os.execvp(cmd[0], cmd)
sys.exit(1)
else:
fo.close()
os.waitpid(pid, 0)

pa = parse_aiod.parse_aiod(fn)
mbps = pa.get_mb_per_sec()
del pa

os.unlink(fn)
return mbps

if __name__ == '__main__':
def start_bt(dir, dsfs):
if os.path.exists(dir):
os.system('/bin/rm -rf ' + dir)
os.mkdir(dir)
cmd = '/usr/local/bin/blktrace -D %s -b 1024 -n 6' % dir
for dsf in dsfs:
cmd = '%s /dev/%s' % (cmd, dsf)

pid = os.fork()
if pid > 0:
time.sleep(30)
return pid

fo = open('%s/bt.out' % dir, 'w')
os.dup2(fo.fileno(), sys.stdout.fileno())
os.dup2(fo.fileno(), sys.stderr.fileno())
fo.close()

cmd = cmd.split(None)
os.execvp(cmd[0], cmd)
sys.exit(1)

def stop_bt(pid):
os.kill(pid, 1)
os.waitpid(pid, 0)

#bss = [ 4, 8, 16, 32, 64, 128, 256 ]
bss = [ 32, 64, 128 ]

print '%3s:' % 'run',
for bs in bss:
print '%6d ' % bs,
print '\n----',
for bs in bss:
print '%6s ' % '------',
print ''
for run in range(0, 3):
print '%3d:' % run,
sys.stdout.flush()
for bs in bss:
pid = start_bt('%03dk_%03d' % (bs, run), dsfs)
print '%6.2f ' % run_aiod(bs, dsfs, False),
sys.stdout.flush()
stop_bt(pid)
print ''

sys.exit(0)
\
 
 \ /
  Last update: 2009-03-10 16:19    [W:0.525 / U:0.068 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site