lkml.org 
[lkml]   [2007]   [Dec]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: QUEUE_FLAG_CLUSTER: not working in 2.6.24 ?
Jens Axboe wrote:
> On Thu, Dec 13 2007, Jens Axboe wrote:
>> On Thu, Dec 13 2007, Mark Lord wrote:
>>> Jens Axboe wrote:
>>>> On Thu, Dec 13 2007, Mark Lord wrote:
>>>>> Mark Lord wrote:
>>>>>> Jens Axboe wrote:
>>>>>>> On Thu, Dec 13 2007, Mark Lord wrote:
>>>>>>>> Matthew Wilcox wrote:
>>>>>>>>> On Thu, Dec 13, 2007 at 01:48:18PM -0500, Mark Lord wrote:
>>>>>>>>>> Problem confirmed. 2.6.23.8 regularly generates segments up to
>>>>>>>>>> 64KB for libata,
>>>>>>>>>> but 2.6.24 uses only 4KB segments and a *few* 8KB segments.
>>>>>>>>> Just a suspicion ... could this be slab vs slub? ie check your
>>>>>>>>> configs
>>>>>>>>> are the same / similar between the two kernels.
>>>>>>>> ..
>>>>>>>>
>>>>>>>> Mmmm.. a good thought, that one.
>>>>>>>> But I just rechecked, and both have CONFIG_SLAB=y
>>>>>>>>
>>>>>>>> My guess is that something got changed around when Jens
>>>>>>>> reworked the block layer for 2.6.24.
>>>>>>>> I'm going to dig around in there now.
>>>>>>> I didn't rework the block layer for 2.6.24 :-). The core block layer
>>>>>>> changes since 2.6.23 are:
>>>>>>>
>>>>>>> - Support for empty barriers. Not a likely candidate.
>>>>>>> - Shared tag queue fixes. Totally unlikely.
>>>>>>> - sg chaining support. Not likely.
>>>>>>> - The bio changes from Neil. Of the bunch, the most likely suspects in
>>>>>>> this area, since it changes some of the code involved with merges and
>>>>>>> blk_rq_map_sg().
>>>>>>> - Lots of simple stuff, again very unlikely.
>>>>>>>
>>>>>>> Anyway, it sounds odd for this to be a block layer problem if you do see
>>>>>>> occasional segments being merged. So it sounds more like the input data
>>>>>>> having changed.
>>>>>>>
>>>>>>> Why not just bisect it?
>>>>>> ..
>>>>>>
>>>>>> Because the early 2.6.24 series failed to boot on this machine
>>>>>> due to bugs in the block layer -- so the code that caused this regression
>>>>>> is probably in the stuff from before the kernels became usable here.
>>>>> ..
>>>>>
>>>>> That sounds more harsh than intended --> the earlier 2.6.24 kernels (up to
>>>>> the first couple of -rc* ones failed here because of incompatibilities
>>>>> between the block/bio changes and libata.
>>>>>
>>>>> That's better, I think!
>>>> No worries, I didn't pick it up as harsh just as an odd conclusion :-)
>>>>
>>>> If I were you, I'd just start from the first -rc that booted for you. If
>>>> THAT has the bug, then we'll think of something else. If you don't get
>>>> anywhere, I can run some tests tomorrow and see if I can reproduce it
>>>> here.
>>> ..
>>>
>>> I believe that *anyone* can reproduce it, since it's broken long before
>>> the requests ever get to SCSI or libata. Which also means that *anyone*
>>> who wants to can bisect it, as well.
>>>
>>> I don't do "bisects".
>> It was just a suggestion on how to narrow it down, do as you see fit.
>>
>>> But I will dig a bit more and see if I can find the culprit.
>> Sure, I'll dig around as well.
>
> Just tried something simple. I only see one 12kb segment so far, so not
> a lot by any stretch. I also DONT see any missed merges signs, so it
> would appear that the pages in the request are simply not contigious
> physically.
>
> diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
> index e30b1a4..1e34b6f 100644
> --- a/block/ll_rw_blk.c
> +++ b/block/ll_rw_blk.c
> @@ -1330,6 +1330,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> goto new_segment;
>
> sg->length += nbytes;
> + if (sg->length > 8192)
> + printk("sg_len=%d\n", sg->length);
> } else {
> new_segment:
> if (!sg)
> @@ -1349,6 +1351,8 @@ new_segment:
> sg = sg_next(sg);
> }
>
> + if (bvprv && (page_address(bvprv->bv_page) + bvprv->bv_len == page_address(bvec->bv_page)))
> + printk("missed merge\n");
> sg_set_page(sg, bvec->bv_page, nbytes, bvec->bv_offset);
> nsegs++;
> }
>
..

Yeah, the first part is similar to my own hack.

For testing, try "dd if=/dev/sda of=/dev/null bs=4096k".
That *really* should end up using contiguous pages on most systems.

I figured out the git thing, and am now building some in-between kernels to try.

Cheers


\
 
 \ /
  Last update: 2007-12-13 21:09    [from the cache]
©2003-2011 Jasper Spaans