Messages in this thread | | | Date | Sat, 27 Dec 2003 16:58:16 +0100 | From | Andrea Arcangeli <> | Subject | Re: 2.6.0 Huge pages not working as expected |
| |
On Fri, Dec 26, 2003 at 08:01:57PM -0800, Linus Torvalds wrote: > > > On Sat, 27 Dec 2003, Andrea Arcangeli wrote: > > > > well, at least on the alpha the above mode = 1 is reproducibly a lot > > better (we're talking about a wall time 2/3 times shorter IIRC) than > > random placement. The l2 is huge and one way cache associative, > > What kind of strange and misguided hw engineer did that?
;)
> I can understand a one-way L1, simply to keep the cycle time low, but > what's the point of a one-way L2? Braindead external cache controller?
dunno, though if you implement the cache coloring with that algorithm I it run fast compared to other archs. No idea how much hardware gain one can get with an huge one way set associative cache (with some help from the OS to ensure it's all used and not trashed) if compared to a (tiny) 16-way associative cache.
> > > The current patch is for 2.2 with an horrible API (it uses a kernel > > module to set those params instead of a sysctl, despite all the real > > code is linked into the kernel), while developing it I only focused on > > the algorithms and the final behaviour in production. the engine to ask > > the allocator a page of the right color works O(1) with the number of > > free pages and it's from Jason. > > Does it keep fragmentation down? > > That's the problem that Davem had in one of his cache-coloring patches: it > worked well enough if you had lots of memory, but it _totally_ broke down > when memory was low. You couldn't allocate higher-order pages at all after > a while because of the fragmented memory.
what can happen is that you've a leaf at order 0 but you don't take it and you split some order 1-MAX_ORDER page instead to get an order 0 of the right color (same can happen with order 1 vs order 2-MAX_ORDER). So yes, it can fragment the memory more quickly, but the very same fragment patterns can happen w/o page coloring, it's just quicker to get into a fragmented state.
However it defragments down perfectly if two contigous order 0 pages are free, no difference in such case, it's just that you may end with more non mergeable order 0 pages more quickly, but it's all quite random and I never heard problems about that. The kernel must free cache and possibly swap too if it fails order > 0 allocations anyways, or the order 1 could not succeed. the swapping/cache-shrinking basically relocates the right colored pages into non right colored pages until the defragmentation happened.
it should be simple to add a very weak mode (the opposite of the strict mode) that forbids the color aware allocator to split an high order page if there's at least one page already available of the right order. That would provide an even lesser guarantee of right coloring though. that could be the same as non coloring at all in practice, that's why I probably not even considered it till today. It's probably simpler to disable it via sysctl than to implement this "weak" mode. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |