lkml.org 
[lkml]   [2015]   [Oct]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: [PATCH 2/3] lib/string_helpers.c: don't lose precision in string_get_size()
Date
So I also played with this over the weekend, and also threw together a
stupid script to check the output. I see you have more or less the same
idea I used, namely to combine size and blk_size earlier.

I put some code on github, https://github.com/Villemoes/get_size. All
versions still fail a very simply case, size=1594323, blk_size=1, for
which the correct answer is "1.52 MiB", but we get "1.51 MiB". But both
your version and my two attempts seem to have the property that they are
always at most one ULP (unit in the last place) too low, and never too
high. ATM, my version 2 fails 70 of the 13598 test cases, while yours
fail 86 (the former being a strict subset of the latter), so they're
very similar.

Regardless of which algorithm we go with, I have some cleanups I'd like
to do, but they're mostly independent and can wait.

Rasmus



\
 
 \ /
  Last update: 2015-10-26 23:21    [W:0.093 / U:1.200 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site