Messages in this thread |  | | Subject | Serious allocating/freeing memory or ReiserFS problem in 2.4.10 | From | Veit Wahlich <> | Date | 26 Sep 2001 04:40:26 +0200 |
| |
Hi!
When I upgraded my kernel from 2.4.9 to 2.4.10 I experienced the problem described in this posting. Because I lack in time, I am not able to read this mailing list regularly, so I do not know whether this problem is known or not. Because I am short on time and this caused crashes on my system I can not apply any screenshots/hardcopys but I will try to help fetching information about this wherever I can. For this please contact cru@zodia.de .
I discovered this while using donkey (www.edonkey2000.com), a non-opensource P2P solution applying a Linux client. Contact is info@edonkey2000.com , I will forward this email to them.
As donkey is a MFTP implementation it begins to "hash" all incomplete files in the download queue when run. This means it reads the whole file and builds a kind of checksum (as I suppose) of all segments that are already downloaded. During this process, the amount of memory in use steadily grows, while the memory used by all processes (including donkey) stays nearly unchanged. Because I am running ReiserFS and donkey is reading huge files, this could also be a ReiserFS problem. I have no experience with donkey and kernels <2.4.9.
Using 384Mb RAM on Linux 2.4.9 with CONFIG_NOHIGHMEM:
When memory usage is up to 95%, the system begins to push all other processes to swap. This includes some daemons, XF86 4.0.3, some killer applications (Mozilla, Evolution) and even parts of the donkey itself. Working becomes nearly impossible but the system keeps working. When "hashing" has been completed, memory usage stays at 99.9% but all processes can be used again, pulling them from swap. Opening and closing bigger applications also "frees" some memory, closing the donkey frees not more memory than the few Mb the donkey processes really used. But in this state working with the machine is possible as every day.
Using 384Mb RAM on Linux 2.4.10 with CONFIG_NOHIGHMEM:
The first time I ran the donkey I discovered that it was much slower and took twice the time it usually takes for hashing, but work on the machine after hashing was fine. I decided to buy more RAM.
Using 1023Mb RAM on Linux 2.4.10 with CONFIG_HIGHMEM4G:
As the kernel told me on boot, I enabled CONFIG_HIGHMEM4G to use more than 896Mb. That gave me 896+127Mb. Now running donkey ended with X to hang, when the system began to swap at 99% mem usage. I was able to ssh to the machine, ps told me X was using 1160% of MEM and killing it was impossible, kill gave no msg on STDOUT/-ERR, the process stayed in state R. Shutting it down failed, shutdown became a zombie.
On next run I started watching the donkey hashing and inspected the memory usage with top. When mem usage reached 99% and swap 72Mb, several processes collapsed with a segfault (some Gnome applets and Evolution), top got some "/proc/*: out of memory" messages on the screen, than also top segfaulted. Donkey stopped working without completing the hashing because one of its own processes collapsed. Still running processes looked fine with ps but shutting down failed as described above.
The third run exactly was like run #2.
Using 896Mb RAM on Linux 2.4.10 with CONFIG_NOHIGHMEM:
When donkey was hashing and swap reached about 70Mb, X collapsed (and got restarted by init), donkey was killed by this. Gracefully shutting down the machine was possible.
Using 896Mb RAM on Linux 2.4.9 with CONFIG_NOHIGHMEM:
Run #2: I booted the 2.4.9 kernel without highmem support and donkey runs fine, I am even able to work a bit while it is hashing (and swapping).
I was unable to test 2.4.9 with CONFIG_HIGHMEM4G because I deleted the source tree backup after 2.4.10 seemed to run fine and I had no time to download and build it again.
The kernel is Linux 2.4.9/2.4.10 without third-party patches. I am running nVidia's NVdriver/GLX v1.0-1512 but this module was disabled in one unsuccessful run, gcc is 2.95.2.1. The new RAM has been successfully tested for 8h by memtest86 on this machine.
Please CC any comments, answers, questions, solutions regarding this posting to cru@zodia.de .
Thanks. // Veit Wahlich
[unhandled content-type:application/pgp-signature] |  |