lkml.org 
[lkml]   [2005]   [Jun]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Variation in measure_migration_cost() with scheduler-cache-hot-autodetect.patch in -mm

* Chen, Kenneth W <kenneth.w.chen@intel.com> wrote:

> I'm consistently getting a smaller than expected cache migration cost
> as measured by Ingo's scheduler-cache-hot-autodetect.patch currently
> in -mm tree. In this patch, the memory used to calibrate migration
> cost is obtained by vmalloc call. Would it make sense to use
> __get_free_pages() instead? I did the following experiments on a
> variety of machines I have access to:
>
> migration cost migration cost
> with vmalloc mem with __get_free_pages
> 3.0GHz Xeon, 8MB cache 6.23 ms 6.32 ms
> 3.4GHz Xeon, 2MB cache 1.62 ms 2.00 ms
> 1.6GHz Itanium2, 9MB 9.2 ms 10.2 ms
> 1.4GHz Itanium2, 4MB 4.2 ms 4.4 ms
>
> Why the discrepancy? Possible cache coloring issue?

probably coloring effects, yes. Another reason could be that
touch_cache() touches 6 separate areas of memory, which combined with
the stack give a minimum of 7 hugepage TLBs. How many are there in these
Xeons? If there are say 4 of them then we could be trashing these TLB
entries. There are much more 4K TLBs. To reduce the number of TLBs
utilized, could you change touch_cache() to do something like:

unsigned long size = __size/sizeof(long), chunk1 = size/2;
unsigned long *cache = __cache;
int i;

for (i = 0; i < size/4; i += 8) {
switch (i % 4) {
case 0: cache[i]++;
case 1: cache[size-1-i]++;
case 2: cache[chunk1-i]++;
case 3: cache[chunk1+i]++;
}
}

does this change the migration-cost values? Btw., how did you determine
the value of the 'ideal' migration cost? Was this based on the database
benchmark measurements?

There are a couple of reasons vmalloc() is better than gfp(): 1) it has
no size limit in the measured range, and 2) it more accurately mimics
migration costs of userspace apps, which typically have most of their
cache-footprint in paged memory, not in hugepage memory.

Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-06-22 10:36    [W:0.058 / U:0.320 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site