Messages in this thread | | | Subject | Ideas for memory management hackers (2) | From | Hans-Christoph Rohland <> | Date | 16 Nov 1997 13:53:34 +0100 |
| |
Hi,
(I discussed some of these items with David Miller at Linux Kongress, Germany. Dave you remember?)
I think the virtual memory support in linux has the following weaknesses irt. applications which use loads of virtual memory in parallel processes:
- Support for shared pages: - No aging (?) - Bad handling of dirty pages. They have to be written to disk by every process which changed the page before releasing the page.
- VM areas are limited to 128MB. This leads to many swap-files. (I am not quite shure about further limitation. Does it mean that bigger mmaps are not possible? The swapfile limitation is definitely in the kernel.)
- Shared anonymous mappings are not supported.
- SysV-IPC is difficult to tune in high memory situations because you have to decide between many or big segments before compiling the kernel. Also it is handled in the kernel specially.
What I am thinking of is
1) Good support for swapping of shared pages
2) Bigger VM areas
3) A generic mechanism for anonymous pages. On top of this could be build shared anonymous mappings, SysV-SHM and swapfilesystems/named files in swap.
4) implementation of the 'madvise' call, especially the MADV_DONTNEED parameter.
Some comments:
With swapfilesystemes or named files in swap you increase the power of mmap for shared memory. You can open a file in swapspace and by mmapping areas into your address space you have a very flexible mechanism for sharing memory between processes. With normal named files you get performance problems due to syncing the file during unmapping and also because swapping to filesystem is slower than to swap devices.
The need for MADV_DONTNEED is a similar problem: no memory manager can know that I really don't need a part of memory any more. It will always try to save the memory in backing store. But especially transactional programs process some memory and when the transaction finishes there is no need to get this content again. It will be overwritten by the next transaction. So the program can advise the OS to simply discard the page instead of saving the page to the swapfile. This can increase the system performance a lot under low physical memory.
Cheers Christoph
| |