Messages in this thread | | | Date | Fri, 30 Sep 2022 11:40:18 +0200 | Subject | Re: [PATCH 0/3] ksm: fix incorrect count of merged pages when enabling use_zero_pages | From | David Hildenbrand <> |
| |
>>> >>>> >>>>> >>>>> second, once a page is merged with a zero page, it's not really handled >>>>> by KSM anymore. if you have a big allocation, of which you only touch a >>>>> few pages, would the rest be considered "merged"? no, it's just zero >>>>> pages, right? >>>> >>>> If you haven't touched memory, there is nothing populated -- no shared >>>> zeropage. >>>> >>>> We only populate shared zeropages in private anonymous mappings on read >>>> access without prior write. >>> >>> that's what I meant. if you read without writing, you get zero pages. >>> you don't consider those to be "shared" from a KSM point of view >>> >>> does it make a difference if some pages that have been written to but >>> now only contain zeroes are discarded and mapped back to the zero pages? >> >> That's a good question. When it comes to unmerging, you'd might expect >> that whatever was deduplicated will get duplicated again -- and your >> memory consumption will adjust accordingly. The stats might give an >> admin an idea regarding how much memory is actually overcommited. See >> below on the important case where we essentially never see the shared >> zeropage. >> >> The motivation behind these patches would be great -- what is the KSM >> user and what does it want to achieve with these numbers? > > anyone who works on big amounts of very sparse data, especially in a VM > (as I explained above, with KSM without use_zero_pages KVM guests lose > the zero page colouring)
I meant the patches in question here :)
> >> >>> >>>> >>>>> this is the same, except that we take present pages with zeroes in it >>>>> and we discard them and map them to zero pages. it's kinda like if we >>>>> had never touched them. >>>> >>>> MADV_UNMERGEABLE >>>> >>>> "Undo the effect of an earlier MADV_MERGEABLE operation on the >>>> specified address range; KSM unmerges whatever pages it had merged in >>>> the address range specified by addr and length." >>>> >>>> Now please explain to me how not undoing a zeropage merging is correct >>>> according to this documentation. >>>> >>> >>> because once it's discarded and replaced with a zero page, the page is >>> not handled by KSM anymore. >>> >>> I understand what you mean, that KSM did an action that now cannot be >>> undone, but how would you differentiate between zero pages that were >>> never written to and pages that had been written to and then discarded >>> and mapped back to a zero page because they only contained zeroes? >> >> An application that always properly initializes (write at least some >> part once) all its memory will never have the shared zeropage mapped. VM >> guest memory comes to mind, probably still the most important KSM use case. >> >> There are currently some remaining issues when taking a GUP R/O longterm >> pin on such a page (e.g., vfio). In contrast to KSM pages, such pins are >> not reliable for the shared zeropage, but I have fixes for them pending. >> However, that is rather a corner case (it didn't work at all correctly a >> while ago) and will be sorted out soon. >> >> So the question is if MADV_UNMERGEABLE etc. (stats) should be adjusted >> to document the behavior with use_zero_pages accordingly. > > we can count how many times a page full of zeroes was merged with a > zero-page, but we can't count how many time one of those pages was then > unmerged. once it's merged it becomes a zero-page, like the others.
Right. We could special case on MADV_MERGEABLE ("how many zero pages do we have mapped into MADV_MERGEABLE VMAs"), but it gets tricky once we enable MADV_MERGEABLE on a region where the shared zeropage was already populated. Probably not worth the effort.
-- Thanks,
David / dhildenb
| |