lkml.org 
[lkml]   [2020]   [Jun]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH] fs: Do not check if there is a fsnotify watcher on pseudo inodes
    On Fri, Jun 12, 2020 at 12:52:28PM +0300, Amir Goldstein wrote:
    > On Fri, Jun 12, 2020 at 12:26 PM Mel Gorman <mgorman@techsingularity.net> wrote:
    > >
    > > The kernel uses internal mounts for a number of purposes including pipes.
    > > On every vfs_write regardless of filesystem, fsnotify_modify() is called
    > > to notify of any changes which incurs a small amount of overhead in fsnotify
    > > even when there are no watchers.
    > >
    > > A patch is pending that reduces, but does not eliminte, the overhead
    > > of fsnotify but for the internal mounts, even the small overhead is
    > > unnecessary. The user API is based on the pathname and a dirfd and proc
    > > is the only visible path for inodes on an internal mount. Proc does not
    > > have the same pathname as the internal entry so even if fatrace is used
    > > on /proc, no events trigger for the /proc/X/fd/ files.
    > >
    >
    > This looks like a good direction and I was going to suggest that as well.
    > However, I am confused by the use of terminology "internal mount".
    > The patch does not do anything dealing with "internal mount".

    I was referring to users of kern_mount.

    > If alloc_file_pseudo() is only called for filesystems mounted as
    > internal mounts,

    I believe this is the case and I did not find a counter-example. The
    changelog that introduced the helper is not explicit but it was created
    in the context of converting a number of internal mounts like pipes,
    anon inodes to a common helper. If I'm wrong, Al will likely point it out.

    > please include this analysis in commit message.
    > In any case, not every file of internal mount is allocated with
    > alloc_file_pseudo(),
    > right?

    Correct. It is not required and there is at least one counter example
    in arch/ia64/kernel/perfmon.c but I don't think that is particularly
    important, I don't think anyone is kept awake at night worrying about
    small performance overhead on Itanium.

    > So maybe it would be better to list all users of alloc_file_pseudo()
    > and say that they all should be opted out of fsnotify, without mentioning
    > "internal mount"?
    >

    The users are DMA buffers, CXL, aio, anon inodes, hugetlbfs, anonymous
    pipes, shmem and sockets although not all of them necessary end up using
    a VFS operation that triggers fsnotify. Either way, I don't think it
    makes sense (or even possible) to watch any of those with fanotify so
    setting the flag seems reasonable.

    I updated the changelog and maybe this is clearer.

    ---8<---
    fs: Do not check if there is a fsnotify watcher on pseudo inodes

    The kernel can create invisible internal mounts for a number of purposes
    including pipes via kern_mount. For pipes, every vfs_write regardless of
    filesystem, fsnotify_modify() is called to notify of any changes which
    incurs a small amount of overhead in fsnotify even when there are no
    watchers. It can also trigger for reads and readv and writev, it was
    simply vfs_write() that was noticed first.

    A patch is pending that reduces, but does not eliminte, the overhead
    of fsnotify but for the internal mounts, even the small overhead is
    unnecessary. The user API for fanotify is based on the pathname and a dirfd
    and proc are the only visible representation of an internal mount. Proc
    does not have the same pathname as the internal entry and the proc inode
    is not the same as the internal inode so even if fatrace is used on /proc,
    no events trigger for the /proc/X/fd/ files.

    This patch changes alloc_file_pseudo() automatically opts out of fsnotify
    by setting FMODE_NONOTIFY flag so that no check is made for fsnotify
    watchers on internal mounts. It is not mandated that mounts created
    with kern_mount use alloc_file_pseudo but a number of important ones
    do including aio, anon inodes, hugetlbfs, anonymous pipes, shmem and
    sockets. There does not appear to be any way to register watchers on such
    inodes or a case where it would even make sense so opting out by default
    seems reasonable.

    The test motivating this was "perf bench sched messaging --pipe". On
    a single-socket machine using threads the difference of the patch was
    as follows.

    5.7.0 5.7.0
    vanilla nofsnotify-v1r1
    Amean 1 1.3837 ( 0.00%) 1.3547 ( 2.10%)
    Amean 3 3.7360 ( 0.00%) 3.6543 ( 2.19%)
    Amean 5 5.8130 ( 0.00%) 5.7233 * 1.54%*
    Amean 7 8.1490 ( 0.00%) 7.9730 * 2.16%*
    Amean 12 14.6843 ( 0.00%) 14.1820 ( 3.42%)
    Amean 18 21.8840 ( 0.00%) 21.7460 ( 0.63%)
    Amean 24 28.8697 ( 0.00%) 29.1680 ( -1.03%)
    Amean 30 36.0787 ( 0.00%) 35.2640 * 2.26%*
    Amean 32 38.0527 ( 0.00%) 38.1223 ( -0.18%)

    The difference is small but in some cases it's outside the noise so
    while marginal, there is still some small benefit to ignoring fsnotify
    for internal mounts in some cases.

    Signed-off-by: Mel Gorman <mgorman@techsingularity.net>

    \
     
     \ /
      Last update: 2020-06-12 15:19    [W:2.487 / U:1.100 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site