Messages in this thread | ![/](/images/icornerl.gif) | | From | Yu Zhao <> | Date | Thu, 14 Dec 2023 23:46:29 -0700 | Subject | Re: [RFC v2] mm: Multi-Gen LRU: fix use mm/page_idle/bitmap |
| |
On Fri, Dec 8, 2023 at 12:12 AM Henry Huang <henry.hj@antgroup.com> wrote: > > Thanks for replying this RFC. > > > 1. page_idle/bitmap isn't a capable interface at all -- yes, Google > > proposed the idea [1], but we don't really use it anymore because of > > its poor scalability. > > In our environment, we use /sys/kernel/mm/page_idle/bitmap to check > pages whether were accessed during a peroid of time.
Is it a production environment? If so, what's your 1. scan interval 2. memory size
I'm trying to understand why scalability isn't a problem for you. On an average server, there are hundreds of millions of PFNs, so it'd be very expensive to use that ABI even for a time interval of minutes.
> We manage all pages > idle time in userspace. Then use a prediction algorithm to select pages > to reclaim. These pages would more likely be idled for a long time.
"There is a system in place now that is based on a user-space process that reads a bitmap stored in sysfs, but it has a high CPU and memory overhead, so a new approach is being tried." https://lwn.net/Articles/787611/
Could you elaborate how you solved this problem?
> We only need kernel to tell use whether a page is accessed, a boolean > value in kernel is enough for our case.
How do you define "accessed"? I.e., through page tables or file descriptors or both?
> > 2. PG_idle/young, being a boolean value, has poor granularity. If > > anyone must use page_idle/bitmap for some specific reason, I'd > > recommend exporting generation numbers instead. > > Yes, at first time, we try using multi-gen LRU proactvie scan and > exporting generation&refs number to do the same thing. > > But there are serveral problems: > > 1. multi-gen LRU only care about self-memcg pages. In our environment, > it's likely to see that different memcg's process share pages.
This is related to my question above: are those pages mapped into different memcgs or not?
> multi-gen LRU only update gen of pages in current memcg. It's hard to > judge a page whether is accessed depends on gen update.
This depends. I'd be glad to elaborate after you clarify the above.
> We still have no ideas how to solve this problem. > > 2. We set swappiness 0, and use proactive scan to select cold pages > & proactive reclaim to swap anon pages. But we can't control passive > scan(can_swap = false), which would make anon pages cold/hot inversion > in inc_min_seq.
There is an option to prevent the inversion, IIUC, the force_scan option is what you are looking for.
| ![\](/images/icornerr.gif) |