lkml.org 
[lkml]   [2012]   [Jul]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: RAID extremely slow
On 07/25/2012 10:00 PM, Kevin Ross wrote:
>
>>>
>>>> unused devices:<none>
>>>>
>>>> # cat /proc/sys/dev/raid/speed_limit_min
>>>> 10000
>>> MD is unable to reach its minimum rebuild rate while other system
>>> activity is ongoing. You might want to lower this number to see if
>>> that
>>> gets you out of the stalls.
>>>
>>> Or temporarily shut down mythtv.
>>
>> I will try lowering those numbers next time this happens, which will
>> probably be within the next day or two. That's about how often this
>> happens.
>
> Unfortunately, it has happened again, with speeds at near zero.
>
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md0 : active raid6 sdh1[0] sdd1[9] sde1[10] sdb1[6] sdi1[7] sdc1[4]
> sdf1[3] sdg1[8] sdj1[1]
> 6837311488 blocks super 1.2 level 6, 512k chunk, algorithm 2
> [9/9] [UUUUUUUUU]
> [=>...................] resync = 8.3% (81251712/976758784)
> finish=1057826.4min speed=14K/sec
>
> unused devices: <none>
>
> atop doesn't show ANY activity on the raid device or the individual
> drives.
> http://img687.imageshack.us/img687/2913/screenshotfrom201207252.png
>
> Also, I tried writing to a test file with the following command, and
> it hangs. I let it go for about 30 minutes, with no change.
>
> # dd if=/dev/zero of=test bs=1M count=1
>
> dmesg only reports hung tasks. It doesn't report any other problems.
> Here's my dmesg output:
> http://pastebin.ca/2174778
>
> I'm going to try rebooting into single user mode, and see if the
> rebuild succeeds without stalling.
>
> -- Kevin

It rebuilt fine in single user mode, with speeds usually around
50MB/sec. But after exiting single user mode, and allowing MythTV and
other programs to start, within 30 minutes I had the problem again.
Basically a hung filesystem. I couldn't even "cat /proc/mdstat", that
just hung. Lots of hung task warnings in dmesg.

Because Phil suggested that fsync calls might cause stalls, I commented
out the fsync in MythTV. I'll run with that for awhile, and see how
things work out. So far it isn't adversely affecting MythTV.

Thanks!
-- Kevin



\
 
 \ /
  Last update: 2012-07-27 01:21    [W:0.097 / U:1.912 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site