lkml.org 
[lkml]   [2022]   [Sep]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH 0/3] ksm: fix incorrect count of merged pages when enabling use_zero_pages
From
>>> 
>>>>
>>>>>
>>>>> second, once a page is merged with a zero page, it's not really handled
>>>>> by KSM anymore. if you have a big allocation, of which you only touch a
>>>>> few pages, would the rest be considered "merged"? no, it's just zero
>>>>> pages, right?
>>>>
>>>> If you haven't touched memory, there is nothing populated -- no shared
>>>> zeropage.
>>>>
>>>> We only populate shared zeropages in private anonymous mappings on read
>>>> access without prior write.
>>>
>>> that's what I meant. if you read without writing, you get zero pages.
>>> you don't consider those to be "shared" from a KSM point of view
>>>
>>> does it make a difference if some pages that have been written to but
>>> now only contain zeroes are discarded and mapped back to the zero pages?
>>
>> That's a good question. When it comes to unmerging, you'd might expect
>> that whatever was deduplicated will get duplicated again -- and your
>> memory consumption will adjust accordingly. The stats might give an
>> admin an idea regarding how much memory is actually overcommited. See
>> below on the important case where we essentially never see the shared
>> zeropage.
>>
>> The motivation behind these patches would be great -- what is the KSM
>> user and what does it want to achieve with these numbers?
>
> anyone who works on big amounts of very sparse data, especially in a VM
> (as I explained above, with KSM without use_zero_pages KVM guests lose
> the zero page colouring)

I meant the patches in question here :)

>
>>
>>>
>>>>
>>>>> this is the same, except that we take present pages with zeroes in it
>>>>> and we discard them and map them to zero pages. it's kinda like if we
>>>>> had never touched them.
>>>>
>>>> MADV_UNMERGEABLE
>>>>
>>>> "Undo the effect of an earlier MADV_MERGEABLE operation on the
>>>> specified address range; KSM unmerges whatever pages it had merged in
>>>> the address range specified by addr and length."
>>>>
>>>> Now please explain to me how not undoing a zeropage merging is correct
>>>> according to this documentation.
>>>>
>>>
>>> because once it's discarded and replaced with a zero page, the page is
>>> not handled by KSM anymore.
>>>
>>> I understand what you mean, that KSM did an action that now cannot be
>>> undone, but how would you differentiate between zero pages that were
>>> never written to and pages that had been written to and then discarded
>>> and mapped back to a zero page because they only contained zeroes?
>>
>> An application that always properly initializes (write at least some
>> part once) all its memory will never have the shared zeropage mapped. VM
>> guest memory comes to mind, probably still the most important KSM use case.
>>
>> There are currently some remaining issues when taking a GUP R/O longterm
>> pin on such a page (e.g., vfio). In contrast to KSM pages, such pins are
>> not reliable for the shared zeropage, but I have fixes for them pending.
>> However, that is rather a corner case (it didn't work at all correctly a
>> while ago) and will be sorted out soon.
>>
>> So the question is if MADV_UNMERGEABLE etc. (stats) should be adjusted
>> to document the behavior with use_zero_pages accordingly.
>
> we can count how many times a page full of zeroes was merged with a
> zero-page, but we can't count how many time one of those pages was then
> unmerged. once it's merged it becomes a zero-page, like the others.

Right. We could special case on MADV_MERGEABLE ("how many zero pages do
we have mapped into MADV_MERGEABLE VMAs"), but it gets tricky once we
enable MADV_MERGEABLE on a region where the shared zeropage was already
populated. Probably not worth the effort.


--
Thanks,

David / dhildenb

\
 
 \ /
  Last update: 2022-09-30 11:41    [W:1.397 / U:0.356 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site