lkml.org 
[lkml]   [2017]   [Apr]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [HMM 03/15] mm/unaddressable-memory: new type of ZONE_DEVICE for unaddressable memory
From
Date
On 4/23/17 6:13 AM, Dan Williams wrote:
> On Sat, Apr 22, 2017 at 11:11 AM, Jerome Glisse <jglisse@redhat.com> wrote:
>> On Fri, Apr 21, 2017 at 10:30:01PM -0700, Dan Williams wrote:
>>> On Fri, Apr 21, 2017 at 8:30 PM, Jérôme Glisse <jglisse@redhat.com> wrote:
>>
>> [...]
>>
>>>> +/*
>>>> + * Specialize ZONE_DEVICE memory into multiple types each having differents
>>>> + * usage.
>>>> + *
>>>> + * MEMORY_DEVICE_PERSISTENT:
>>>> + * Persistent device memory (pmem): struct page might be allocated in different
>>>> + * memory and architecture might want to perform special actions. It is similar
>>>> + * to regular memory, in that the CPU can access it transparently. However,
>>>> + * it is likely to have different bandwidth and latency than regular memory.
>>>> + * See Documentation/nvdimm/nvdimm.txt for more information.
>>>> + *
>>>> + * MEMORY_DEVICE_UNADDRESSABLE:
>>>> + * Device memory that is not directly addressable by the CPU: CPU can neither
>>>> + * read nor write _UNADDRESSABLE memory. In this case, we do still have struct
>>>> + * pages backing the device memory. Doing so simplifies the implementation, but
>>>> + * it is important to remember that there are certain points at which the struct
>>>> + * page must be treated as an opaque object, rather than a "normal" struct page.
>>>> + * A more complete discussion of unaddressable memory may be found in
>>>> + * include/linux/hmm.h and Documentation/vm/hmm.txt.
>>>> + */
>>>> +enum memory_type {
>>>> + MEMORY_DEVICE_PERSISTENT = 0,
>>>> + MEMORY_DEVICE_UNADDRESSABLE,
>>>> +};
>>>
>>> Ok, this is a bikeshed, but I think it is important. I think these
>>> should be called MEMORY_DEVICE_PUBLIC and MEMORY_DEVICE_PRIVATE. The
>>> reason is that persistence has nothing to do with the code paths that
>>> deal with the pmem use case of ZONE_DEVICE. The only property the mm
>>> cares about is that the address range behaves the same as host memory
>>> for dma and cpu accesses. The "unaddressable" designation always
>>> confuses me because a memory range isn't memory if it's
>>> "unaddressable". It is addressable, it's just "private" to the device.
>>
>> I can change the name but the memory is truely unaddressable, the CPU
>> can not access it whatsoever (well it can access a small window but
>> even that is not guaranteed).
>>
>
> Understood, but that's still "addressable only by certain agents or
> through a proxy" which seems closer to "private" to me.
>

Actually, MEMORY_DEVICE_PRIVATE / _PUBLIC seems like a good choice to
me, because the memory may not remain CPU-unaddressable in the future.
By that, I mean that I know of at least one company (ours) that is
working on products that will support hardware-based memory coherence
(and access counters to go along with that). If someone were to enable
HMM on such a system, then the device memory would be, in fact, directly
addressable by a CPU--thus exactly contradicting the "unaddressable" name.

Yes, it is true that we would have to modify HMM anyway, in order to
work in that situation, partly because HMM today relies on CPU and
device page faults in order to work. And it's also true that we might
want to take a different approach than HMM, to support that kind of
device: for example, making it a NUMA node has been debated here, recently.

But even so, I think the potential for the "unaddressable" memory
actually becoming "addressable" someday, is a good argument for using a
different name.

thanks,

--
John Hubbard
NVIDIA

\
 
 \ /
  Last update: 2017-04-24 02:40    [W:0.073 / U:1.520 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site