lkml.org 
[lkml]   [2022]   [Jan]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH v3 2/2] kernfs: Reduce contention around global per-fs kernfs_rwsem.
On Thu, Jan 13, 2022 at 09:42:59PM +1100, Imran Khan wrote:
> Right now a global per file system based rwsem (kernfs_rwsem)
> synchronizes multiple kernfs operations. On a large system with
> few hundred CPUs and few hundred applications simultaenously trying
> to access sysfs, this results in multiple sys_open(s) contending on
> kernfs_rwsem via kernfs_iop_permission and kernfs_dop_revalidate.
>
> - 21.42% 21.34% showgids [kernel.kallsyms] [k] up_read
> 21.34% __libc_start_main
> __GI___libc_open
> entry_SYSCALL_64_after_hwframe
> do_syscall_64
> sys_open
> do_sys_open
> do_filp_open
> - path_openat
> - 20.05% link_path_walk
> - 9.76% walk_component
> lookup_fast
> - d_revalidate.part.24
> - 9.75% kernfs_dop_revalidate
> up_read
> - 9.46% inode_permission
> - __inode_permission
> - 9.46% kernfs_iop_permission
> up_read
> - 0.83% kernfs_iop_get_link
> up_read
> - 0.80% lookup_fast
> d_revalidate.part.24
> kernfs_dop_revalidate
> up_read
>
> - 21.31% 21.21% showgids [kernel.kallsyms] [k] down_read
> 21.21% __libc_start_main
> __GI___libc_open
> entry_SYSCALL_64_after_hwframe
> do_syscall_64
> sys_open
> do_sys_open
> do_filp_open
> - path_openat
> - 19.78% link_path_walk
> - 10.62% inode_permission
> - __inode_permission
> - 10.62% kernfs_iop_permission
> down_read
> - 8.45% walk_component
> lookup_fast
> - d_revalidate.part.24
> - 8.45% kernfs_dop_revalidate
> down_read
> - 0.71% kernfs_iop_get_link
> down_read
> - 0.72% lookup_fast
> - d_revalidate.part.24
> - 0.72% kernfs_dop_revalidate
> down_read
> - 0.71% may_open
> inode_permission
> __inode_permission
> kernfs_iop_permission
> down_read
>
> Since permission is specific to a kernfs_node we can use a hashed
> lock to access/modify permission. Also use kernfs reference counting
> to ensure we are accessing/modifying permissions for an existing
> kernfs_node object.
>
> Using this change brings down the above mentioned down_read/up_read
> numbers to ~8%, thus indicating that contention around kernfs_rwsem
> has reduced to about 1/3rd of earlier value.

Ah, nevermind, you do post the results here, I should have kept reading.

Nice work!

I'll look at these after 5.17-rc1 is out, thanks!

greg k-h

\
 
 \ /
  Last update: 2022-01-13 11:58    [W:0.098 / U:0.500 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site