Messages in this thread | | | Date | Wed, 8 Jun 2016 17:04:22 +0200 | From | Michal Hocko <> | Subject | Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival |
| |
On Wed 08-06-16 16:35:37, Lukasz Odzioba wrote: > When the application does not exit cleanly (i.e. SIGTERM) we might
I do not see how a SIGTERM would make any difference. But see below.
> end up with some pages in lru_add_pvec, which is ok. With THP > enabled huge pages may also end up on per cpu lru_add_pvecs. > In the systems with a lot of processors we end up with quite a lot > of memory pending for addition to LRU cache - in the worst case > scenario up to CPUS * PAGE_SIZE * PAGEVEC_SIZE, which on machine > with 200+CPUs means GBs in practice.
It is 56kB per CPU for normal pages which is not really that bad. 28MB for THP only cache is a lot though.
> We are able to reproduce this problem with the following program: > > void main() { > { > size_t size = 55 * 1000 * 1000; // smaller than MEM/CPUS > void *p = mmap(NULL, size, PROT_READ | PROT_WRITE, > MAP_PRIVATE | MAP_ANONYMOUS , -1, 0); > if (p != MAP_FAILED) > memset(p, 0, size); > //munmap(p, size); // uncomment to make the problem go away
Is this really true? Both munmap and exit_mmap do the same lru_add_drain() which flushes only the local CPU cache so munmap shouldn't make any difference.
> } > > When we run it it will leave significant amount of memory on pvecs. > This memory will be not reclaimed if we hit OOM, so when we run > above program in a loop: > $ for i in `seq 100`; do ./a.out; done > many processes (95% in my case) will be killed by OOM. > > This patch flushes lru_add_pvecs on compound page arrival making > the problem less severe - kill rate drops to 0%.
I believe this deserves a more explanation. What do you think about the following. " The primary point of the LRU add cache is to save the zone lru_lock contention with a hope that more pages will belong to the same zone and so their addition can be batched. The huge page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like a safer option when compared to a potential excess in the caching which can be quite large and much harder to fix because lru_add_drain_all is way to expensive and it is not really clear what would be a good moment to call it. "
Does this sound better?
> > Suggested-by: Michal Hocko <mhocko@suse.com> > Tested-by: Lukasz Odzioba <lukasz.odzioba@intel.com> > Signed-off-by: Lukasz Odzioba <lukasz.odzioba@intel.com> > --- > mm/swap.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/mm/swap.c b/mm/swap.c > index 9591614..3fe4f18 100644 > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -391,9 +391,8 @@ static void __lru_cache_add(struct page *page) > struct pagevec *pvec = &get_cpu_var(lru_add_pvec); > > get_page(page); > - if (!pagevec_space(pvec)) > + if (!pagevec_add(pvec, page) || PageCompound(page)) > __pagevec_lru_add(pvec); > - pagevec_add(pvec, page); > put_cpu_var(lru_add_pvec); > } > > -- > 1.8.3.1 >
-- Michal Hocko SUSE Labs
| |