lkml.org 
[lkml]   [2010]   [Apr]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Downsides to madvise/fadvise(willneed) for application startup
On 04/06/2010 02:51 AM, Johannes Weiner wrote:
> On Mon, Apr 05, 2010 at 03:43:02PM -0700, Taras Glek wrote:
>
>> Hello,
>> I am working on improving Mozilla startup times. It turns out that page
>> faults(caused by lack of cooperation between user/kernelspace) are the
>> main cause of slow startup. I need some insights from someone who
>> understands linux vm behavior.
>>
>> Current Situation:
>> The dynamic linker mmap()s executable and data sections of our
>> executable but it doesn't call madvise().
>> By default page faults trigger 131072byte reads. To make matters worse,
>> the compile-time linker + gcc lay out code in a manner that does not
>> correspond to how the resulting executable will be executed(ie the
>> layout is basically random). This means that during startup 15-40mb
>> binaries are read in basically random fashion. Even if one orders the
>> binary optimally, throughput is still suboptimal due to the puny readahead.
>>
>> IO Hints:
>> Fortunately when one specifies madvise(WILLNEED) pagefaults trigger 2mb
>> reads and a binary that tends to take 110 page faults(ie program stops
>> execution and waits for disk) can be reduced down to 6. This has the
>> potential to double application startup of large apps without any clear
>> downsides. Suse ships their glibc with a dynamic linker patch to
>> fadvise() dynamic libraries(not sure why they switched from doing
>> madvise before).
>>
>> I filed a glibc bug about this at
>> http://sourceware.org/bugzilla/show_bug.cgi?id=11431 . Uli commented
>> with his concern about wasting memory resources. What is the impact of
>> madvise(WILLNEED) or the fadvise equivalent on systems under memory
>> pressure? Does the kernel simply start ignoring these hints?
>>
> It will throttle based on memory pressure. In idle situations it will
> eat your file cache, however, to satisfy the request.
>
Define idle situations. Do you mean that madv(willneed) will aggresively
readahead, but only while cpu(or disk?) is idle?
I am trying to optimize application startup which means that the cpu is
busy while not blocked on io.
> Now, the file cache should be much bigger than the amount of unneeded
> pages you prefault with the hint over the whole library, so I guess the
> benefit of prefaulting the right pages outweighs the downside of evicting
> some cache for unused library pages.
>
> Still, it's a workaround for deficits in the demand-paging/readahead
> heuristics and thus a bit ugly, I feel. Maybe Wu can help.
>
>
Can't wait to hear the juicy details.
>> Also, once an application is started is it reasonable to keep it
>> madvise(WILLNEED)ed or should the madvise flags be reset?
>>
> It's a one-time operation that starts immediate readahead, no permanent
> changes are done.
>
I may be measuring this wrong, but in my experience the only change
madvise(willneed) does in increase the length parameter to
__do_page_cache_readahead(). My script is at
http://hg.mozilla.org/users/tglek_mozilla.com/startup/file/6453ad2a7906/kernelio.stp
.


Taras


\
 
 \ /
  Last update: 2010-04-06 23:59    [W:0.135 / U:0.072 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site