lkml.org 
[lkml]   [2021]   [Jul]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Subject[PATCH v6 09/13] iio: afe: rescale: fix precision on fractional log scale
Date
From: Liam Beguin <lvb@xiphos.com>

The IIO_VAL_FRACTIONAL_LOG2 scale type doesn't return the expected
scale. Update the case so that the rescaler returns a fractional type
and a more precise scale.

Signed-off-by: Liam Beguin <lvb@xiphos.com>
---
drivers/iio/afe/iio-rescale.c | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
index 35fa3b4e53e0..47cd4a6d9aca 100644
--- a/drivers/iio/afe/iio-rescale.c
+++ b/drivers/iio/afe/iio-rescale.c
@@ -44,12 +44,9 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
*val2 = rescale->denominator;
return IIO_VAL_FRACTIONAL;
case IIO_VAL_FRACTIONAL_LOG2:
- tmp = *val * 1000000000LL;
- do_div(tmp, rescale->denominator);
- tmp *= rescale->numerator;
- do_div(tmp, 1000000000LL);
- *val = tmp;
- return scale_type;
+ *val = rescale->numerator * *val;
+ *val2 = rescale->denominator * (1 << *val2);
+ return IIO_VAL_FRACTIONAL;
case IIO_VAL_INT_PLUS_NANO:
tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
tmp = div_s64(tmp, rescale->denominator);
--
2.30.1.489.g328c10930387
\
 
 \ /
  Last update: 2021-07-21 05:15    [W:0.147 / U:0.672 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site