Messages in this thread | | | Date | Thu, 10 Feb 2000 23:13:06 +0100 | From | Harald Koenig <> | Subject | Re: [patch] Re: 2.3.42 buffer problem/leak ? |
| |
On Feb 09, Andrea Arcangeli wrote:
> Harald could you give a try to 2.3.42aa3? > > ftp://ftp.*.kernel.org/pub/linux/kernel/people/andrea/kerenls/v2.3/2.3.42aa3.gz > > The interesting bit is here: > > ftp://ftp.*.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.3/2.3.42aa3/icache-dcache-preshrink-1.gz > > I attach the interesting patch to the email too. It's against 2.3.42. > > It basically avoids the caching part of the icache and dcache to grow more > than 2% of the memory in the machine. You can tune via sysctl the > percentage. The percentage should choosed in function of the hashtable > size to avoid an insane number of collisions in the hash that could lead > the dcache to be potentially slower than rereading the inode and direntry > from the buffer cache. Actually the hashtable is not yet dynamic but of > course my thoughts will apply to the future.
thanks for attaching the patch! while in theory I would agree with Linus that fixed limits are not the best way to go, I've run some tests and (at least for me) interesting results:
the problem with icache and dcache filling memory is triggered through a nightly cron job running `updatedb' and `du' on all file systems and showed up when compiling large sources (XFree86 or kernel) where the memory isn't reused for buffer cache. so I tried and timed the following sequence after reboot (without X11 or other running tasks) on K6/2-400 with 128MB and two old/slow SCSI disks (4GB 7200rpm 6MB/sec and 2GB 5400rpm 4MB/sec), most fs space is e2fs with 1k blocks, for some slabdata info see below (times are real/user/system in min (m) and secs)
time updatedb time du / time updatedb time du / make clean time make bzImage modules (2.3.42 sources with gcc-2.95.2)
for 2.2.14, 2.3.39, pristine 2.3.42 and 2.3.42 plus your patch:
2.2.14: updatedb(1) 10m25r 51u 29s du(1) 3m38r 9u 24s updatedb(2) 3m00r 49u 27s du(2) 3m17r 9u 23s make 17m46r 977u 62s
2.3.39: updatedb(1) 11m49r du(1) 11m22r updatedb(2) du(2) make 18m55r 980u 67s
2.3.42: updatedb(1) 11m52r 50u 26s du(1) 11m12r 9u 24s updatedb(2) 12m30r 30u 30s du(2) 11m23r 9u 26s make 18m53r 979u 68s
2.3.42+patch: updatedb(1) 10m40r 50u 31s du(1) 1m40r 8u 17s updatedb(2) 9m16r 50u 31s du(2) 3117r 8u 17s make 18m29r 980u 66s
observations:
- 2.3.39 and 2.3.42 behave the same way (true for slabdata information below, too). - 2.2.14 du is _much_ faster than 2.3.42 without patch - Andrea's patch made du much faster than even 2.2.14 - kernel compilation with 2.3.42+patch is better, but 2.2.14 is still ahead - 2.2.14 updatedb for 2nd run is _much_ faster == cached, Andrea's patch only gives a small step in that direction
and here the output of free and slabinfo first after updatedb and du, then after kernel compile:
clean 2.3.42: =============================================================================== total used free shared buffers cached Mem: 126596 115124 11472 0 2136 10028 -/+ buffers/cache: 102960 23636 Swap: 82364 2464 79900
inode_cache 214738 227496 dentry_cache 113672 166036 buffer_head 9196 9198 size-32 4297 4347 ------------------------------------------------------------------------------- total used free shared buffers cached Mem: 126596 121924 4672 0 1120 22476 -/+ buffers/cache: 98328 28268 Swap: 82364 2592 79772
inode_cache 173888 212064 dentry_cache 82951 164951 buffer_head 19314 19320 size-32 3606 4284 ===============================================================================
2.3.42+patch =============================================================================== total used free shared buffers cached Mem: 126596 120028 6568 0 76060 13816 -/+ buffers/cache: 30152 96444 Swap: 82364 0 82364
buffer_head 89970 92400 inode_cache 27942 33204 dentry_cache 20310 25916 size-32 1823 1953 ------------------------------------------------------------------------------- total used free shared buffers cached Mem: 126596 119416 7180 0 75744 13556 -/+ buffers/cache: 30116 96480 Swap: 82364 0 82364
buffer_head 89334 92778 inode_cache 28737 33192 dentry_cache 21063 25730 size-32 1824 1953 ===============================================================================
2.3.39: =============================================================================== total used free shared buffers cached Mem: 126624 120804 5820 0 2408 7196 -/+ buffers/cache: 111200 15424 Swap: 82364 1088 81276
inode_cache 248914 248928 dentry_cache 145545 155806 buffer_head 8561 12138 size-32 4258 4284 ------------------------------------------------------------------------------- total used free shared buffers cached Mem: 126624 122800 3824 0 1204 24956 -/+ buffers/cache: 96640 29984 Swap: 82364 1240 81124
inode_cache 188034 203556 dentry_cache 90865 155062 buffer_head 21687 21714 size-32 3766 4347 ===============================================================================
2.2.14: =============================================================================== total used free shared buffers cached Mem: 128216 117596 10620 4044 89500 7488 -/+ buffers/cache: 20608 107608 Swap: 82364 3632 78732
buffer_head 92800 92946 dentry_cache 13906 16740 size-32 603 10206 vm_area_struct 504 630 ------------------------------------------------------------------------------- ===============================================================================
Harald -- All SCSI disks will from now on ___ _____ be required to send an email notice 0--,| /OOOOOOO\ 24 hours prior to complete hardware failure! <_/ / /OOOOOOOOOOO\ \ \/OOOOOOOOOOOOOOO\ \ OOOOOOOOOOOOOOOOO|// Harald Koenig, \/\/\/\/\/\/\/\/\/ Inst.f.Theoret.Astrophysik // / \\ \ koenig@tat.physik.uni-tuebingen.de ^^^^^ ^^^^^
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |