![]() | |||||||||||
Messages in this thread |
We did some i/o profiling about 6 years ago on a big scientific app that had started in fortran and had been rewritten in c++ the fortran code used r/w on files and used temp files the c++ did memmaps and had big data structures - taking advantage of memory management. one thing I thought was interesting is that it was easy to see how a smart algorithm, not even such a smart one, could adapt i/o to the patterns of i/o in the fortran code, but the c++ i/o patters were really complex. when everything goes into the page cache, it seems like you will loose information. On Sun, May 12, 2002 at 09:38:12AM +1000, Lincoln Dale wrote: > as the person who started this whole thread and made the assertion that > copying from A to B is common: > > At 11:35 AM 11/05/2002 -0700, Linus Torvalds wrote: > >And I personally believe that "generate the data yourself" is actually a > >very common case. A pure pipe between two places is not what a computer is > >good at, or what a computer should be used for. > > i think you'd be surprised. if we include "pipe from disk to network" then > a large number of 'server' applications do exactly this. > webservers do. fileservers do. http caches do. streaming-media servers do. > > sure, they may add additional headers on the front and still generate > dynamic content in some cases, but the "common case" is 'pipe from disk to > network' or 'pipe from network to disk'. > 'network' is typically TCP but can be UDP (with rate-limiting) in some cases. > > > its very good to see this being discussed. thats a large step forward from > many people believing the problem was nonexistent. > > i'm skeptical that continuing to use the page-cache is the correct way to > go -- many of these kinds of applications are doing their own form of > memory-management and hot-content 'caching' so are happy to manage a > few-to-several hundred megabytes of "page cache equivalent" data themselves. > at least on many of the 2.3.xx linux releases, that was one of the big > attractions of 'raw' devices -- they didn't get the box into an OOM situation. > if 2.5.xx and recent 2.4.xx has the issues of > page-cache-doesn't-shrink-fast-enough solved, then its forseeable it will fly. > > > cheers, > > lincoln. > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- --------------------------------------------------------- Victor Yodaiken Finite State Machine Labs: The RTLinux Company. www.fsmlabs.com www.rtlinux.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ | ||||||||||
| Last update: 2005-03-22 13:26 [from the cache] ©2003-2008 | |||||||||||