lkml.org 
[lkml]   [2012]   [Dec]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH] fadvise: perform WILLNEED readahead in a workqueue
On Sun, Dec 16, 2012 at 05:23:02AM +0000, Eric Wong wrote:
> Dave Chinner <david@fromorbit.com> wrote:
> > On Sun, Dec 16, 2012 at 03:35:49AM +0000, Eric Wong wrote:
> > > Dave Chinner <david@fromorbit.com> wrote:
> > > > On Sun, Dec 16, 2012 at 12:25:49AM +0000, Eric Wong wrote:
> > > > > Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > > > > > On Sat, 15 Dec 2012 00:54:48 +0000
> > > > > > Eric Wong <normalperson@yhbt.net> wrote:
> > > > > >
> > > > > > > Applications streaming large files may want to reduce disk spinups and
> > > > > > > I/O latency by performing large amounts of readahead up front
> >
> > > This could also be a use case for an audio/video player.
> >
> > Sure, but this can all be handled by a userspace application. If you
> > want to avoid/batch IO to enable longer spindown times, then you
> > have to load the file into RAM somewhere, and you don't need special
> > kernel support for that.
>
> From userspace, I don't know when/if I'm caching too much and possibly
> getting the userspace cache itself swapped out.

Which causes th disk to spin up. Now you start to see the complexity
of what you are trying to acheive...

> > > So no, there's no difference that matters between the approaches.
> > > But I think doing this in the kernel is easier for userspace users.
> >
> > The kernel provides mechanisms for applications to use. You have not
> > mentioned anything new that requires a new kernel mechanism to
> > acheive - you just need to have the knowledge to put the pieces
> > together properly. People have been solving this same problem for
> > the last 20 years without needing to tweak fadvise(). Or even having
> > an fadvise() syscall...
>
> fadvise() is fairly new, and AFAIK few apps use it. Perhaps if it
> were improved, more people would use it and not have to reinvent
> the wheel.

fadvise() is not "fairly new". It's been around for many, many
years - it was there whan the linux kernel moved to git in 2005, and
I haven't bothered to look any further back in history...

> > Nothing about low latency IO or streaming IO is simple or easy, and
> > changing how readahead works doesn't change that fact. All it does
> > is change the behaviour of every other application that uses
> > fadvise() to minimise IO latency....
>
> I don't want to introduce regressions, either.
>
> Perhaps if part of the FADV_WILLNEED read-ahead were handled
> synchronously (maybe 2M?) and humongous large readaheads (like mine)
> went to the background, that would be a good trade off?

Which you can already do in userspace yourself without changing the
kernel. i.e:

main thread background thread:

readahead(0, 2MB)
spawn background thread
read(0, len)
readahead(2MB,1GB);

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com


\
 
 \ /
  Last update: 2012-12-16 22:41    [W:0.069 / U:1.484 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site