lkml.org 
[lkml]   [2002]   [Sep]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [BENCHMARK] Corrected gcc3.2 v gcc2.95.3 contest results

On Mon, 23 Sep 2002, Con Kolivas wrote:

> Agreed. There probably is no statistically significant difference in the
> different gcc versions.
>
> Contest is very new and I appreciate any feedback I can get to make it
> as worthwhile a benchmark as possible to those who know.

your measurements are really useful i think, and people like Andrew
started to watch those numbers - this is why at this point a bit more
effort can/should be taken to filter out fluctuations better. Ie. a single
fluctuation could send Andrew out on a wild goose chase while perhaps in
reality his kernel was the fastest. Running every test twice should at
least give a ballpart figure wrt. fluctuations, without increasing the
runtime unrealistically.

i agree that only the IO benchmarks are problematic from this POV - things
like the process load and your other CPU-saturating numbers look perfectly
valid.

obviously another concern to to make testing not take days to accomplish.
This i think is one of the hardest things - making timely measurements
which are still meaningful and provide stable results.

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:29    [from the cache]
©2003-2011 Jasper Spaans