lkml.org 
[lkml]   [2009]   [Aug]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: abnormal OOM killer message
    From
    On Wed, Aug 19, 2009 at 7:58 PM, Mel Gorman<mel@csn.ul.ie> wrote:
    > On Wed, Aug 19, 2009 at 07:52:42PM +0900, Minchan Kim wrote:
    >> Thanks for good comment, Mel.
    >>
    >> On Wed, 19 Aug 2009 11:36:11 +0100
    >> Mel Gorman <mel@csn.ul.ie> wrote:
    >>
    >> > On Wed, Aug 19, 2009 at 03:49:58PM +0900, Minchan Kim wrote:
    >> > > On Wed, 19 Aug 2009 15:24:54 +0900
    >> > > ????????? <chungki.woo@gmail.com> wrote:
    >> > >
    >> > > > Thank you very much for replys.
    >> > > >
    >> > > > But I think it seems not to relate with stale data problem in compcache.
    >> > > > My question was why last chance to allocate memory was failed.
    >> > > > When OOM killer is executed, memory state is not a condition to
    >> > > > execute OOM killer.
    >> > > > Specially, there are so many pages of order 0. And allocating order is zero.
    >> > > > I think that last allocating memory should have succeeded.
    >> > > > That's my worry.
    >> > >
    >> > > Yes. I agree with you.
    >> > > Mel. Could you give some comment in this situation ?
    >> > > Is it possible that order 0 allocation is failed
    >> > > even there are many pages in buddy ?
    >> > >
    >> >
    >> > Not ordinarily. If it happens, I tend to suspect that the free list data
    >> > is corrupted and would put a check in __rmqueue() that looked like
    >> >
    >> >     BUG_ON(list_empty(&area->free_list) && area->nr_free);
    >>
    >> If memory is corrupt, it would be not satisfied with both condition.
    >> It would be better to ORed condition.
    >>
    >> BUG_ON(list_empty(&area->free_list) || area->nr_free);
    >>
    >
    > But it's perfectly reasonable to have nr_free a positive value. The
    > point of the check is ensure the counters make sense. If nr_free > 0 and
    > the list is empty, it means accounting is all messed up and the values
    > reported for "free" in the OOM message are fiction.
    >
    >> > The second question is, why are we in direct reclaim this far above the
    >> > watermark? It should only be kswapd that is doing any reclaim at that
    >> > point. That makes me wonder again are the free lists corrupted.
    >>
    >> It does make sense!
    >

    'Corrupted free list' makes sense. Thank you very much.
    Inserting BUG_ON code is also good idea to check corruption of free list.

    I have one more question.
    As you know, before and after executing direct reclaim
    routine(try_to_free_pages)
    cond_resched() routine is also executed.
    In other words, it can be scheduled at that time.
    Is there no possibility executing kswapd or try_to_free_pages at other
    context at that time?
    I think this fact maybe can explain that gap(between watermark and
    free memory) also.
    How do you think about this?
    But I know this can't explain why last chance to allocate memory was failed.
    I think your idea makes sense.

    Anyway, I will try to test again with following BUG_ON code.

    BUG_ON(list_empty(&area->free_list) && area->nr_free);

    Thanks
    Mel, Minchan

    >> > The other possibility is that the zonelist used for allocation in the
    >> > troubled path contains no populated zones. I would put a BUG_ON check in
    >> > get_page_from_freelist() to check if the first zone in the zonelist has no
    >> > pages. If that bug triggers, it might explain why OOMs are triggering for
    >> > no good reason.
    >>
    >> Yes. Chungki. Could you put the both BUG_ON in each function and
    >> try to reproduce the problem ?
    >>
    >> > I consider both of those possibilities abnormal though.
    >> >
    >> > > >
    >> > > > -----------------------------------------------------------------------------------------------------------------------------------------------
    >> > > >       page = get_page_from_freelist(gfp_mask|__GFP_HARDWALL, order,
    >> > > > <== this is last chance
    >> > > >                            zonelist, ALLOC_WMARK_HIGH|ALLOC_CPUSET);
    >> > > > <== uses ALLOC_WMARK_HIGH
    >> > > >       if (page)
    >> > > >       goto got_pg;
    >> > > >
    >> > > >       out_of_memory(zonelist, gfp_mask, order);
    >> > > >       goto restart;
    >> > > > -----------------------------------------------------------------------------------------------------------------------------------------------
    >> > > >
    >> > > > > Let me have a question.
    >> > > > > Now the system has 79M as total swap.
    >> > > > > It's bigger than system memory size.
    >> > > > > Is it possible in compcache?
    >> > > > > Can we believe the number?
    >> > > >
    >> > > > Yeah, It's possible. 79Mbyte is data size can be swap.
    >> > > > It's not compressed data size. It's just original data size.
    >> > >
    >> > > You means your pages with 79M are swap out in compcache's reserved
    >> > > memory?
    >> > >
    >> >
    >> > --
    >> > Mel Gorman
    >> > Part-time Phd Student                          Linux Technology Center
    >> > University of Limerick                         IBM Dublin Software Lab
    >>
    >>
    >> --
    >> Kind regards,
    >> Minchan Kim
    >>
    >
    > --
    > Mel Gorman
    > Part-time Phd Student                          Linux Technology Center
    > University of Limerick                         IBM Dublin Software Lab
    >
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2009-08-19 14:09    [W:3.467 / U:0.008 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site