lkml.org 
[lkml]   [2007]   [Oct]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    SubjectRe: [PATCH] remove throttle_vm_writeout()
    From
    Date
    On Thu, 2007-10-04 at 16:09 -0700, Andrew Morton wrote:
    > On Fri, 05 Oct 2007 00:39:16 +0200
    > Miklos Szeredi <miklos@szeredi.hu> wrote:
    >
    > > > throttle_vm_writeout() should be a per-zone thing, I guess. Perhaps fixing
    > > > that would fix your deadlock. That's doubtful, but I don't know anything
    > > > about your deadlock so I cannot say.
    > >
    > > No, doing the throttling per-zone won't in itself fix the deadlock.
    > >
    > > Here's a deadlock example:
    > >
    > > Total memory = 32M
    > > /proc/sys/vm/dirty_ratio = 10
    > > dirty_threshold = 3M
    > > ratelimit_pages = 1M
    > >
    > > Some program dirties 4M (dirty_threshold + ratelimit_pages) of mmap on
    > > a fuse fs. Page balancing is called which turns all these into
    > > writeback pages.
    > >
    > > Then userspace filesystem gets a write request, and tries to allocate
    > > memory needed to complete the writeout.
    > >
    > > That will possibly trigger direct reclaim, and throttle_vm_writeout()
    > > will be called. That will block until nr_writeback goes below 3.3M
    > > (dirty_threshold + 10%). But since all 4M of writeback is from the
    > > fuse fs, that will never happen.
    > >
    > > Does that explain it better?
    > >
    >
    > yup, thanks.
    >
    > This is a somewhat general problem: a userspace process is in the IO path.
    > Userspace block drivers, for example - pretty much anything which involves
    > kernel->userspace upcalls for storage applications.
    >
    > I solved it once in the past by marking the userspace process as
    > PF_MEMALLOC and I beleive that others have implemented the same hack.
    >
    > I suspect that what we need is a general solution, and that the solution
    > will involve explicitly telling the kernel that this process is one which
    > actually cleans memory and needs special treatment.
    >
    > Because I bet there will be other corner-cases where such a process needs
    > kernel help, and there might be optimisation opportunities as well.
    >
    > Problem is, any such mark-me-as-special syscall would need to be
    > privileged, and FUSE servers presently don't require special perms (do
    > they?)

    I think just adding nr_cpus * ratelimit_pages to the dirth_thresh in
    throttle_vm_writeout() will also solve the problem
    [unhandled content-type:application/pgp-signature]
    \
     
     \ /
      Last update: 2007-10-05 09:35    [W:0.023 / U:31.232 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site