Messages in this thread | | | Date | Tue, 28 Jun 2011 15:12:33 -0700 | From | Andrew Morton <> | Subject | Re: [PATCH v4 0/2] fadvise: move active pages to inactive list with POSIX_FADV_DONTNEED |
| |
On Mon, 27 Jun 2011 15:29:19 +0200 Andrea Righi <andrea@betterlinux.com> wrote:
> There were some reported problems in the past about trashing page cache when a > backup software (i.e., rsync) touches a huge amount of pages (see for example > [1]). > > This problem has been almost fixed by the Minchan Kim's patch [2] and a proper > use of fadvise() in the backup software. For example this patch set [3] has > been proposed for inclusion in rsync. > > However, there can be still other similar trashing problems: when the backup > software reads all the source files, some of them may be part of the actual > working set of the system. When a POSIX_FADV_DONTNEED is performed _all_ pages > are evicted from pagecache, both the working set and the use-once pages touched > only by the backup software. > > With the following solution when POSIX_FADV_DONTNEED is called for an active > page instead of removing it from the page cache it is added to the tail of the > inactive list. Otherwise, if it's already in the inactive list the page is > removed from the page cache. Pages mapped by other processes or unevictable > pages are not touched at all. > > In this way if the backup was the only user of a page, that page will be > immediately removed from the page cache by calling POSIX_FADV_DONTNEED. If the > page was also touched by other processes it'll be moved to the inactive list, > having another chance of being re-added to the working set, or simply reclaimed > when memory is needed.
So if an application touches a page twice and then runs POSIX_FADV_DONTNEED, that page will now not be freed.
That's a big behaviour change. For many existing users POSIX_FADV_DONTNEED simply doesn't work any more!
I'd have thought that adding a new POSIX_FADV_ANDREA would be safer than this.
The various POSIX_FADV_foo's are so ill-defined that it was a mistake to ever use them. We should have done something overtly linux-specific and given userspace more explicit and direct pagecache control.
| |