Messages in this thread | | | Date | Mon, 23 Sep 2002 09:49:53 +0200 (CEST) | From | Ingo Molnar <> | Subject | Re: [BENCHMARK] Corrected gcc3.2 v gcc2.95.3 contest results |
| |
On Mon, 23 Sep 2002, Con Kolivas wrote:
> IO Full Load: > 2.5.38 170.21 42% > 2.5.38-gcc32 230.77 30%
> This time only the IO loads showed a statistically significant > difference.
how many times are you running each test? You should run them at least twice (ideally 3 times at least), to establish some sort of statistical noise measure. Especially IO benchmarks tend to fluctuate very heavily depending on various things - they are also very dependent on the initial state - ie. how the pagecache happens to lay out, etc. Ie. a meaningful measurement result would be something like:
IO Full Load: 2.5.38 170.21 +- 55.21 sec 42% 2.5.38-gcc32 230.77 +- 60.22 sec 30%
where the first column is the average of two measurements, the second column is the delta of the two measurements divided by 2. This way we can see the 'spread' of the results.
I simply cannot believe that gcc32 can produce any visible effect in any of the IO benchmarks, the only explanation would be heavy fluctuation of IO results.
Ingo
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |