lkml.org 
[lkml]   [2003]   [Jan]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: buffer leakage in kernel?
Andrew Morton wrote:

> > While this scanning is running, number of "buffers" reported by ''free"
> > and in /proc/meminfo
> > is continuously increasing up to ~ 500 MB !!
>
> You did not specify the filesystem type. I shall assume ext2.

right

> > This is not nice at all when I have another applications running
> > with memory consumption > 500 MB: when my "scanner" approaches 50G
> > border on
> > each disk, I've got numerous "Out of memory" murders :(. Even 'ssh' to
> > this machine
> > is killed :(
> >
> > Could anyone explain why it happens? I suppose that it is a memory
> > leakage in file system buffer management.

Thanx for explanation.

> Sounds like a bug.
>
> Are you reading these large files via a single application, or via one
> process per file?

via single application with 8 I/O threads + 1 thread.

> How large is the buffer into which the application is performing the O_DIRECT
> read?

Application allocates 512 MB and never uses more.
2MB buffers are used for each read() and write() calls.
Each file has only one read or write request going any time. There is no othr
memory-
hungry applications running.

> Please perform this test:
>
> 1: Wait until you have 500M "Buffers"
> 2: cat 64_gig_file > /dev/null
> 3: Now see how large "Buffers" is. It should have reduced a lot.

Yes, it worked, they had reduced.
Does this mean, that cached indirect buffers can't be kicked out of memory
automatically
and ONLY non-O_DIRECT access can do it? I suppose, they should be displaced by
newly allocated indirect buffers and user memory allocation.


Roman

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:32    [W:0.607 / U:0.048 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site