lkml.org 
[lkml]   [2011]   [Jun]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFC PATCH 0/3] block: Fix fsync slowness with CFQ cgroups
On Tue, Jun 28, 2011 at 12:47:38PM +1000, Dave Chinner wrote:
>
> Vivek, I'm not sure this is a general solution. If we hand journal
> IO off to a workqueue, then we've got no idea what the "dependent
> task" is.
>
> I bring this up as I have a current patchset that moves all the XFS
> journal IO out of process context into a workqueue to solve
> process-visible operation latency (e.g. 1000 mkdir syscalls run at
> 1ms each, the 1001st triggers a journal checkpoint and takes 500ms)
> and background checkpoint submission races. This effectively means
> that XFS will trigger the same bad CFQ behaviour on fsync, but have
> no means of avoiding it because we don't have a specific task to
> yield to.
>
> And FWIW, we're going to be using workqueues more and more in XFS
> for asynchronous processing of operations. I'm looking to use WQs
> for speculative readahead of inodes, all our delayed metadata
> writeback, log IO submission, free space allocation requests,
> background inode allocation, background inode freeing, background
> EOF truncation, etc to process as much work asynchronously outside
> syscall context as possible (let's use all those CPU cores we
> have!).
>
> All of these things will push potentially dependent IO operations
> outside of the bounds of the process actually doing the operation,
> so some general solution to the "dependent IO in an undefined thread
> context" problem really needs to be solved sooner rather than
> later...
>
> As it is, I don't have any good ideas of how to solve this, but I
> thought it is worth bringing to your attention while you are trying
> to solve a similar issue.

Dave,

Coule of thoughts.

- We can introduce anohter block layer call were dependencies are setup
from worker thread context. So when the process schedules the work, it can
save the task information somewhere and when the worker thread actually
calls the specified funciton, that function can setup the dependency
between worker thread and submitting task.

Probably original process can tear down the dependency connection
when IO is done. I am assuming that IO submitting process is waiting
for all IO to finish.

In current framework one can specify multiple processes being dependent
on one thread but not vice-a-versa. I think we should be able to
handle that by maintaining a linked list of dependent queues instead
of single pointer. So if a process submits a bunch of jobs with help
of bunch of worker threads from multiple cpus, I think that case is
manageable with some extension to current patches.

- Or we can also try to do something more exotic and that when we schedule
a work, one should be able to tell which cgroup the worker should run in.
When the worker actually runs, it can migrate itself to destination
destination cgroup and submit IO. This does not take care of cases like
journalling thread where multiple processes are dependent on single
kernel thread. In that case above dependent queue solution should work
well.

So I think above API can be extended to handle the case of work queues
also or we could look into migrating worker in user specified cgroup if
that turns out to be a better solution.

Thanks
Vivek


\
 
 \ /
  Last update: 2011-06-28 15:53    [W:0.129 / U:0.008 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site