lkml.org 
[lkml]   [2006]   [May]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: [OOPS] amrestore dies in kmem_cache_free 2.6.16.18 - cannot restore backups!
    On Thu, 25 May 2006, Kai Makisara wrote:

    > I am adding linux-scsi to recipients (and quoting the whole message for
    > readers of that list).
    >
    > On Tue, 23 May 2006, James Lamanna wrote:
    >
    > > So I was able to recreate this problem on a vanilla 2.6.16.18 with the
    > > following oops..
    > > I'd say this is a serious regression since I cannot restore backups
    > > anymore (I could with 2.6.14.x, but that kernel series had other
    > > issues...)
    > >
    > > amrestore does manage to read 1 32k block from tape before dying.
    > >
    > > Any help would be greatly appreciated.
    > >
    > I have tried 'amrestore' on my machine with 2.6.16.18 but was not able to
    > reproduce the problem.

    OK. Now I think I have found something, thanks to Mike Christie's reminder
    yesterday in another thread that the patch at the end of this message has
    not been merged into 2.6.16 (and 2.6.17-rcx) ;-)

    I did strace amrestore here and found out that the tape buffer addresses
    were not aligned at 512 byte boundaries:

    read(0x3, 0x523380, 0x8000) = 0x8000
    read(0x3, 0x51b370, 0x8000) = 0x8000

    This meant that st used the internal driver buffer aligned at page
    boundary as input to scsi_execute_async and the problem fixed by the patch
    did not occur.

    (BTW, I hate this. The SCSI HBA would be perfectly capable to do direct
    transfers from/to these addresses but the default alignment restrictions
    prevent this and the HBA driver does not modify the defaults.)

    Next I made a test program reading to a buffer with start address I could
    control. When the offset from page boundary was 0 or not a multiple of
    512, no errors occurred. When I set the offset to 512, I got the following
    OOPS (this is from 2.6.17-rc5 with CONFIG_DEBUG_SLAB set but the
    similarity is obvious):

    kfree_debugcheck: out of range ptr fffffffffffffff8h.
    ----------- [cut here ] --------- [please bite here ] ---------
    Kernel BUG at mm/slab.c:2590
    invalid opcode: 0000 [1]
    CPU 0
    Modules linked in: st snd_seq snd_pcm_oss snd_mixer_oss w83627hf hwmon_vid
    i2c_isa snd_via82xx snd_ac97_codec snd_ac97_bus snd_pcm snd_timer
    snd_page_alloc snd_mpu401_uart snd_rawmidi snd_seq_device snd i2c_viapro
    i2c_core ohci1394 ieee1394
    Pid: 11174, comm: talign Not tainted 2.6.17-rc5-g705af309 #7
    RIP: 0010:[<ffffffff8025dd2a>] <ffffffff8025dd2a>{kfree_debugcheck+70}
    RSP: 0018:ffff810011057c68 EFLAGS: 00010096
    RAX: 0000000000000039 RBX: fffffffffffffff8 RCX: ffffffff80558f98
    RDX: ffff810039dbae60 RSI: 0000000000000046 RDI: ffffffff80558f80
    RBP: ffff81003ff991c0 R08: ffffffff80558f98 R09: 0000000000000020
    R10: 0000000000000010 R11: 0000000000000010 R12: fffffffffffffff8
    R13: 0000000000000246 R14: ffffffff80266d70 R15: ffff8100390f28e0
    FS: 00002ba29f0aeb00(0000) GS:ffffffff806c8000(0000)
    knlGS:00000000563b80c0
    CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
    CR2: 00002ba29eed407b CR3: 000000000cdbb000 CR4: 00000000000006e0
    Process talign (pid: 11174, threadinfo ffff810011056000, task
    ffff810039dbae60)
    Stack: 0000000000000000 ffffffff8025e634 0000000000000000 ffff81003ff991c0
    ffff810001fee098 0000000000000246 0000000000000200 ffffffff8025f2a8
    ffff81003ff991c0 ffff81000b94e8b0
    Call Trace: <ffffffff8025e634>{cache_free_debugcheck+35}
    <ffffffff8025f2a8>{kmem_cache_free+41}
    <ffffffff80266d70>{bio_free+48}
    <ffffffff803fa0e5>{scsi_execute_async+374}
    <ffffffff880edec7>{:st:st_do_scsi+504}
    <ffffffff880ed045>{:st:st_sleep_done+0}
    <ffffffff880ed89e>{:st:setup_buffering+516}
    <ffffffff880f17cd>{:st:st_read+845}
    <ffffffff8022381d>{__wake_up+54}
    <ffffffff80262af3>{vfs_read+168} <ffffffff802634b0>{sys_read+69}
    <ffffffff802095de>{system_call+126}
    Code: 0f 0b 68 37 bd 4e 80 c2 1e 0a 48 b8 ff ff ff 7f ff ff ff ff
    RIP <ffffffff8025dd2a>{kfree_debugcheck+70} RSP <ffff810011057c68>

    Next thing was to patch 2.6.16.18 with the patch at the end: No more
    oopses with any alignment.

    James, does this fix your problem ?

    Kai

    --------------------------------8<------------------------------------------
    Excerpt from a message from Brian Holty to linux-scsi and linux-kernel on
    Wed, 22 Mar 2006 06:35:39:

    ...
    Based on above, I think the most intuitive fix would be the offset addition of
    the first entry to the initialization of nr_pages.

    Without this change, for instance, with 4K io's every sg io that is
    dma_aligned for direct io, but not page aligned will cause slab corruption
    and an oops

    I am able to run a number of tests with sg that cause the boundary to be
    crossed, and with this fix there is no slab corruption or data corruption.

    Thanks Dan, I had been hunting for this for a couple of days!!

    Thoughts??

    Signed-off-by: Bryan Holty <lgeek@frontiernet.net>

    --- a/drivers/scsi/scsi_lib.c 2006-03-03 13:17:22.000000000 -0600
    +++ b/drivers/scsi/scsi_lib.c 2006-03-22 06:09:09.669599539 -0600
    @@ -368,7 +368,7 @@
    int nsegs, unsigned bufflen, gfp_t gfp)
    {
    struct request_queue *q = rq->q;
    - int nr_pages = (bufflen + PAGE_SIZE - 1) >> PAGE_SHIFT;
    + int nr_pages = (bufflen + sgl[0].offset + PAGE_SIZE - 1) >> PAGE_SHIFT;
    unsigned int data_len = 0, len, bytes, off;
    struct page *page;
    struct bio *bio = NULL;
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/
    \
     
     \ /
      Last update: 2006-05-27 11:37    [from the cache]
    ©2003-2014 Jasper Spaans. hosted at Digital Ocean