lkml.org 
[lkml]   [2009]   [Nov]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: get_user_pages question
Date
On Monday 09 November 2009 12:32:52 you wrote:
> On Mon, 9 Nov 2009, Andi Kleen wrote:
> > Mark Veltzer <mark.veltzer@gmail.com> writes:
> > > I am testing this kernel module with several buffers from user space
> > > allocated in several different ways. heap, data segment, static
> > > variable in function and stack. All scenarious work EXCEPT the stack
> > > one. When passing the stack buffer the kernel sees one thing while user
> > > space sees another.
> >
> > In theory it should work, stack is no different from any other pages.
> > First thought was that you used some platform with incoherent caches,
> > but that doesn't seem to be the case if it's standard x86.
>
> It may be irrelevant to Mark's stack case, but it is worth mentioning
> the fork problem: how a process does get_user_pages to pin down a buffer
> somewhere in anonymous memory, a thread forks (write protecting anonymous
> memory shared between parent and child), child userspace writes to a
> location in the same page as that buffer, causing copy-on-write which
> breaks the connection between the get_user_pages buffer and what child
> userspace sees there afterwards.
>
> Hugh
>

Thanks Hugh and Andi

Hugh, you actually hit the nail on the head!

I was forking while doing these mappings and the child won the race and got to
keep the pinned pages while the parent got left with a copy which meant
nothing. The thing is that it was hard to spot because I was using a library
function which called a function etc... which eventually did some system(3).
It only happened on in stack testing case bacause the child was not really
doing anything with the pinned memory on purpose and so in all other cases did
not touch the memory except the stack which it, ofcourse, uses. The child won
the race in the stack case and so shared the data with the kernel and the
parent got a copy with the old data.

I understand that madvise(2) can prevent this copy-on-write and race between
child and parent and I also duplicated it in the kernel using the following
code:

[lock the current->mm for writing]
vma=find_vma(current->mm, [user pointer])
vma->vm_flags|=VM_DONTCOPY
[unlock the current->mm for writing]

The above code is actually a kernel version of madvise(2) and MADV_DONTFORK.

The problem with this solution (either madvise in user space or DONTCOPY in
kernel) is that I give up the ability to fork(2) since the child is left
stackless (or with a hold in it's stack - im not sure...)

My question is: is there a way to allow forking while still pinning STACK
memory via get_user_pages? I can actually live with the current solution since
I can make sure that the user space thread that does the work with the driver
never forks but I'm interested to know what other neat vm tricks linux has up
it's sleeve...

BTW: would it not be a good addition to the madvise(2) manpage to state that
you should be careful with doing madvise(DONTFORK) because you may segfault
your children and that doing so on a stack address has even more chance of
crashing children ? Who should I talk about adding this info to the manual
page? The current manpage that I have only talks about scatter-gather uses of
DONTFORK and does not mention the problems of DONTFORK...

Thanks in advance
Mark


\
 
 \ /
  Last update: 2009-11-09 23:17    [W:1.898 / U:0.092 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site