Messages in this thread | | | Date | Wed, 08 Jul 2009 18:57:38 -0500 | From | Anthony Liguori <> | Subject | Re: [Xen-devel] Re: [RFC PATCH 0/4] (Take 2): transcendent memory ("tmem") for Linux |
| |
Dan Magenheimer wrote: > Hi Anthony -- > > Thanks for the comments. > > >> I have trouble mapping this to a VMM capable of overcommit >> without just coming back to CMM2. >> >> In CMM2 parlance, ephemeral tmem pools is just normal kernel memory >> marked in the volatile state, no? >> > > They are similar in concept, but a volatile-marked kernel page > is still a kernel page, can be changed by a kernel (or user) > store instruction, and counts as part of the memory used > by the VM. An ephemeral tmem page cannot be directly written > by a kernel (or user) store,
Why does tmem require a special store?
A VMM can trap write operations pages can be stored on disk transparently by the VMM if necessary. I guess that's the bit I'm missing.
>> It seems to me that an architecture built around hinting >> would be more >> robust than having to use separate memory pools for this type >> of memory >> (especially since you are requiring a copy to/from the pool). >> > > Depends on what you mean by robust, I suppose. Once you > understand the basics of tmem, it is very simple and this > is borne out in the low invasiveness of the Linux patch. > Simplicity is another form of robustness. >
The main disadvantage I see is that you need to explicitly convert portions of the kernel to use a data copying API. That seems like an invasive change to me. Hinting on the other hand can be done in a less-invasive way.
I'm not really arguing against tmem, just the need to have explicit get/put mechanisms for the transcendent memory areas.
> The copy may be expensive on an older machine, but on newer > machines copying a page is relatively inexpensive.
I don't think that's a true statement at all :-) If you had a workload where data never came into the CPU cache (zero-copy) and now you introduce a copy, even with new system, you're going to see a significant performance hit.
> On a reasonable > multi-VM-kernbench-like benchmark I'll be presenting at Linux > Symposium next week, the overhead is on the order of 0.01% > for a fairly significant savings in IOs. > But how would something like specweb do where you should be doing zero-copy IO from the disk to the network? This is the area where I would be concerned. For something like kernbench, you're already bringing the disk data into the CPU cache anyway so I can appreciate that the copy could get lost in the noise.
Regards,
Anthony Liguori
| |