lkml.org 
[lkml]   [2010]   [Jul]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: 2.6.35 Regression/oops from virtio: return ENOMEM on out of memory patch
    On Mon, Jul 19, 2010 at 06:16:10PM +0300, Michael S. Tsirkin wrote:
    > On Mon, Jul 19, 2010 at 11:02:16AM -0400, Chris Mason wrote:
    > > Hi everyone,
    > >
    > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=686d363786a53ed28ee875b84ef24e6d5126ef6f
    > >
    > > I've been having problems with my long running stress runs and tracked
    > > it down to the above commit. Under load I get a couple of GFP_ATOMIC
    > > allocation failures from virtio per day (not really surprising), and in
    > > the past it would carry on happily.
    > >
    > > Now I get the atomic allocation failure followed by this:
    > >
    > > BUG: unable to handle kernel paging request at ffff88087c37e458
    > > IP: [<ffffffff812e3752>] virtqueue_add_buf_gfp+0x305/0x353
    > >
    > > (Full oops below).
    > >
    > > Looking at virtqueue_add_buf_gfp, it does:
    > >
    > > /* If the host supports indirect descriptor tables, and we have multiple
    > > * buffers, then go indirect. FIXME: tune this threshold */
    > > if (vq->indirect && (out + in) > 1 && vq->num_free) {
    > > head = vring_add_indirect(vq, sg, out, in, gfp);
    > > if (head != vq->vring.num)
    > > goto add_head;
    > > }
    > > [ ... ]
    > >
    > > add_head:
    > > /* Set token. */
    > > vq->data[head] = data;
    > >
    > > Since vring_add_indirect is returning -ENOMEM, head is -ENOMEM and things
    > > go bad pretty quickly. Full oops below, afraid I don't know the virtio
    > > code well enough to provide the clean and obvious fix (outside of
    > > reverting) at this late rc.
    >
    > Good catch! Can you verify this fix please?
    >
    > virtio: fix oops on OOM
    >
    > virtio ring was changed to return an error code on OOM,
    > but one caller was missed and still checks for vq->vring.num.
    > The fix is just to check for <0 error code.
    >
    > Long term it might make sense to change goto add_head to
    > just return an error on oom instead, but let's apply
    > a minimal fix for 2.6.35.

    I haven't been able to trigger the allocation failure since putting this in,
    but it has run well for the last two days. I'll let you know if there
    are problems.

    -chris



    \
     
     \ /
      Last update: 2010-07-21 16:05    [W:0.029 / U:42.940 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site