lkml.org 
[lkml]   [2000]   [Feb]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [patch] Re: 2.3.42 buffer problem/leak ?
    On Feb 09, Andrea Arcangeli wrote:

    > Harald could you give a try to 2.3.42aa3?
    >
    > ftp://ftp.*.kernel.org/pub/linux/kernel/people/andrea/kerenls/v2.3/2.3.42aa3.gz
    >
    > The interesting bit is here:
    >
    > ftp://ftp.*.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.3/2.3.42aa3/icache-dcache-preshrink-1.gz
    >
    > I attach the interesting patch to the email too. It's against 2.3.42.
    >
    > It basically avoids the caching part of the icache and dcache to grow more
    > than 2% of the memory in the machine. You can tune via sysctl the
    > percentage. The percentage should choosed in function of the hashtable
    > size to avoid an insane number of collisions in the hash that could lead
    > the dcache to be potentially slower than rereading the inode and direntry
    > from the buffer cache. Actually the hashtable is not yet dynamic but of
    > course my thoughts will apply to the future.

    thanks for attaching the patch! while in theory I would agree with Linus
    that fixed limits are not the best way to go, I've run some tests
    and (at least for me) interesting results:

    the problem with icache and dcache filling memory is triggered through
    a nightly cron job running `updatedb' and `du' on all file systems
    and showed up when compiling large sources (XFree86 or kernel) where
    the memory isn't reused for buffer cache.
    so I tried and timed the following sequence after reboot (without X11
    or other running tasks) on K6/2-400 with 128MB and two old/slow SCSI disks
    (4GB 7200rpm 6MB/sec and 2GB 5400rpm 4MB/sec), most fs space is e2fs
    with 1k blocks, for some slabdata info see below (times are real/user/system
    in min (m) and secs)

    time updatedb
    time du /
    time updatedb
    time du /
    make clean
    time make bzImage modules (2.3.42 sources with gcc-2.95.2)

    for 2.2.14, 2.3.39, pristine 2.3.42 and 2.3.42 plus your patch:

    2.2.14:
    updatedb(1) 10m25r 51u 29s
    du(1) 3m38r 9u 24s
    updatedb(2) 3m00r 49u 27s
    du(2) 3m17r 9u 23s
    make 17m46r 977u 62s

    2.3.39:
    updatedb(1) 11m49r
    du(1) 11m22r
    updatedb(2)
    du(2)
    make 18m55r 980u 67s

    2.3.42:
    updatedb(1) 11m52r 50u 26s
    du(1) 11m12r 9u 24s
    updatedb(2) 12m30r 30u 30s
    du(2) 11m23r 9u 26s
    make 18m53r 979u 68s

    2.3.42+patch:
    updatedb(1) 10m40r 50u 31s
    du(1) 1m40r 8u 17s
    updatedb(2) 9m16r 50u 31s
    du(2) 3117r 8u 17s
    make 18m29r 980u 66s


    observations:

    - 2.3.39 and 2.3.42 behave the same way (true for slabdata information below, too).
    - 2.2.14 du is _much_ faster than 2.3.42 without patch
    - Andrea's patch made du much faster than even 2.2.14
    - kernel compilation with 2.3.42+patch is better, but 2.2.14 is still ahead
    - 2.2.14 updatedb for 2nd run is _much_ faster == cached, Andrea's patch
    only gives a small step in that direction




    and here the output of free and slabinfo first after updatedb and du,
    then after kernel compile:

    clean 2.3.42:
    ===============================================================================
    total used free shared buffers cached
    Mem: 126596 115124 11472 0 2136 10028
    -/+ buffers/cache: 102960 23636
    Swap: 82364 2464 79900

    inode_cache 214738 227496
    dentry_cache 113672 166036
    buffer_head 9196 9198
    size-32 4297 4347
    -------------------------------------------------------------------------------
    total used free shared buffers cached
    Mem: 126596 121924 4672 0 1120 22476
    -/+ buffers/cache: 98328 28268
    Swap: 82364 2592 79772

    inode_cache 173888 212064
    dentry_cache 82951 164951
    buffer_head 19314 19320
    size-32 3606 4284
    ===============================================================================


    2.3.42+patch
    ===============================================================================
    total used free shared buffers cached
    Mem: 126596 120028 6568 0 76060 13816
    -/+ buffers/cache: 30152 96444
    Swap: 82364 0 82364

    buffer_head 89970 92400
    inode_cache 27942 33204
    dentry_cache 20310 25916
    size-32 1823 1953
    -------------------------------------------------------------------------------
    total used free shared buffers cached
    Mem: 126596 119416 7180 0 75744 13556
    -/+ buffers/cache: 30116 96480
    Swap: 82364 0 82364

    buffer_head 89334 92778
    inode_cache 28737 33192
    dentry_cache 21063 25730
    size-32 1824 1953
    ===============================================================================


    2.3.39:
    ===============================================================================
    total used free shared buffers cached
    Mem: 126624 120804 5820 0 2408 7196
    -/+ buffers/cache: 111200 15424
    Swap: 82364 1088 81276

    inode_cache 248914 248928
    dentry_cache 145545 155806
    buffer_head 8561 12138
    size-32 4258 4284
    -------------------------------------------------------------------------------
    total used free shared buffers cached
    Mem: 126624 122800 3824 0 1204 24956
    -/+ buffers/cache: 96640 29984
    Swap: 82364 1240 81124

    inode_cache 188034 203556
    dentry_cache 90865 155062
    buffer_head 21687 21714
    size-32 3766 4347
    ===============================================================================


    2.2.14:
    ===============================================================================
    total used free shared buffers cached
    Mem: 128216 117596 10620 4044 89500 7488
    -/+ buffers/cache: 20608 107608
    Swap: 82364 3632 78732

    buffer_head 92800 92946
    dentry_cache 13906 16740
    size-32 603 10206
    vm_area_struct 504 630
    -------------------------------------------------------------------------------
    ===============================================================================


    Harald
    --
    All SCSI disks will from now on ___ _____
    be required to send an email notice 0--,| /OOOOOOO\
    24 hours prior to complete hardware failure! <_/ / /OOOOOOOOOOO\
    \ \/OOOOOOOOOOOOOOO\
    \ OOOOOOOOOOOOOOOOO|//
    Harald Koenig, \/\/\/\/\/\/\/\/\/
    Inst.f.Theoret.Astrophysik // / \\ \
    koenig@tat.physik.uni-tuebingen.de ^^^^^ ^^^^^

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 13:56    [W:0.102 / U:0.012 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site