Messages in this thread | | | Date | Mon, 23 Sep 2002 18:12:34 -0700 | From | jw schultz <> | Subject | Re: [BENCHMARK] Corrected gcc3.2 v gcc2.95.3 contest results |
| |
On Tue, Sep 24, 2002 at 07:47:45AM +1000, Con Kolivas wrote: > Quoting Oliver Xymoron <oxymoron@waste.org>: > > > On Tue, Sep 24, 2002 at 12:24:49AM +1000, Con Kolivas wrote: > > > > > > That is the system I was considering. I just need to run enough > > > benchmarks to make this worthwhile though. That means about 5 for > > > each it seems - which may take me a while. A basic mean will suffice > > > for a measure of central tendency. I also need to quote some measure > > > of variability. Standard deviation? > > > > No, standard deviation is inappropriate here. We have no reason to > > expect the distribution of problem cases to be normal or even smooth. > > What we'd really like is range and mean. Don't throw out the outliers > > either, the pathological cases are of critical interest. > > Yes. Definitely the outliers appear to make the difference to the results. The > mean and range appear to be the most important on examining this data. The only > purpose to quoting other figures would be for inferential statistics to > determine if there is a statistically significant difference to the groups. My > overnight benchmarking has generated a few results and I will publish something > soon.
Happy am i to be wrong in suggesting you would benefit from the help of a statistician. My apologies.
Sounds like we are getting to relative performance and confidence interval (much bettern than +/- x) which would be useful for those doing performance improvements and for us who must tune or are watching the improvments take place.
-- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws
Remember Cernan and Schmitt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |