lkml.org 
[lkml]   [2023]   [Mar]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: THP backed thread stacks
Hi Mike,

On Mon, Mar 06, 2023 at 03:57:30PM -0800, Mike Kravetz wrote:
> One of our product teams recently experienced 'memory bloat' in their
> environment. The application in this environment is the JVM which
> creates hundreds of threads. Threads are ultimately created via
> pthread_create which also creates the thread stacks. pthread attributes
> are modified so that stacks are 2MB in size. It just so happens that
> due to allocation patterns, all their stacks are at 2MB boundaries. The
> system has THP always set, so a huge page is allocated at the first
> (write) fault when libpthread initializes the stack.
>
> It would seem that this is expected behavior. If you set THP always,
> you may get huge pages anywhere.
>
> However, I can't help but think that backing stacks with huge pages by
> default may not be the right thing to do. Stacks by their very nature
> grow in somewhat unpredictable ways over time. Using a large virtual
> space so that memory is allocated as needed is the desired behavior.
>
> The only way to address their 'memory bloat' via thread stacks today is
> by switching THP to madvise.
>
> Just wondering if there is anything better or more selective that can be
> done? Does it make sense to have THP backed stacks by default? If not,
> who would be best at disabling? A couple thoughts:
> - The kernel could disable huge pages on stacks. libpthread/glibc pass
> the unused flag MAP_STACK. We could key off this and disable huge pages.
> However, I'm sure there is somebody somewhere today that is getting better
> performance because they have huge pages backing their stacks.
> - We could push this to glibc/libpthreads and have them use
> MADV_NOHUGEPAGE on thread stacks. However, this also has the potential
> of regressing performance if somebody somewhere is getting better
> performance due to huge pages.
> - Other thoughts?

Push this to the application? :)

Something like pthread_attr_getstack() + madvice(MADV_NOHUGEPAGE) will do
the job, no?

> Perhaps this is just expected behavior of THP always which is unfortunate
> in this situation.
> --
> Mike Kravetz
>

--
Sincerely yours,
Mike.

\
 
 \ /
  Last update: 2023-03-27 00:47    [W:0.204 / U:0.068 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site