lkml.org 
[lkml]   [2011]   [Jul]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Fw: [PATCH] sunrpc: use better NUMA affinities
On 29/07/11 16:30, Eric Dumazet wrote:
> Le vendredi 29 juillet 2011 à 16:05 +1000, Greg Banks a écrit :
>> On 29/07/11 15:32, NeilBrown wrote:
>>
>> I seem to remember coming to the conclusion that Jeff eventually
>> addressed this problem...am I misremembering or did something regress?
>>
> Currently, all nfsd kthreads use memory for their kernel stack and
> various initial data from a _single_ node, even if you use
> sunrpc.pool_mode=pernode (or percpu)

That's just plain broken and I'm very pleased to see you fix it.

I was just surprised that it was still broken and wondering how that
happened. Looking at ToT I see that because I dropped the ball in 2008,
Jeff's patches didn't address the problem. In ToT
svc_pool_map_set_cpumask() is called *after* kthread_create() and
applies to the child thread, *after* it's stack has been allocated on
the wrong node. In the working SGI code, svc_pool_map_set_cpumask() is
called by the parent node on itself *before* calling kernel_thread() or
doing any of the data structure allocations, thus ensuring that
everything gets allocated using the default memory allocation policy,
which on SGI NFS servers was globally tuned to be "node-local".

> With my patch, we make sure each thread gets its stack from its local
> node.
>
> Check commit 94dcf29a11b3d20a (kthread: use kthread_create_on_node()) to
> see how this strategy already was adopted for ksoftirqd, kworker,
> migration, and pktgend kthreads.

Ah, I see. It's unfortunate that the kthread_create() API ends up being
passed a CPU number but that's only used to format the name and not for
sensible things :(

--
Greg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2011-07-29 08:55    [W:0.080 / U:1.016 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site