Messages in this thread | | | From | Rasmus Villemoes <> | Subject | Re: [PATCH 2/3] lib/string_helpers.c: don't lose precision in string_get_size() | Date | Mon, 26 Oct 2015 22:48:55 +0100 |
| |
So I also played with this over the weekend, and also threw together a stupid script to check the output. I see you have more or less the same idea I used, namely to combine size and blk_size earlier.
I put some code on github, https://github.com/Villemoes/get_size. All versions still fail a very simply case, size=1594323, blk_size=1, for which the correct answer is "1.52 MiB", but we get "1.51 MiB". But both your version and my two attempts seem to have the property that they are always at most one ULP (unit in the last place) too low, and never too high. ATM, my version 2 fails 70 of the 13598 test cases, while yours fail 86 (the former being a strict subset of the latter), so they're very similar.
Regardless of which algorithm we go with, I have some cleanups I'd like to do, but they're mostly independent and can wait.
Rasmus
| |