ODFstudio/odfstudio
2
0
Fork 0
Code Issues Packages Activity

Better accuracy in rtl_math_approxValue(), tdf#138360 related

Browse Source 
Similar to commit 5abb1890ff for
rtl_math_round() use the reciprocal value in an inverse operation
for negative exponents to not use the inexact 1e-1
0.10000000000000001 and so on factors.

Change-Id: I05b852e06f2c31d6e0ce622b07277a81a5690833
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107172
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
This commit is contained in:
Eike Rathke
2020-12-03 19:58:43 +01:00
parent bbb73810cc
commit 520949f17a
1 changed files with 11 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sal/rtl/math.cxx
+11 -3
View File
@@ -1299,16 +1299,24 @@ double SAL_CALL rtl_math_approxValue( double fValue ) SAL_THROW_EXTERN_C()
int nExp = static_cast< int >(floor(log10(fValue)));
nExp = 14 - nExp;
double fExpValue = getN10Exp(nExp);
double fExpValue = getN10Exp(abs(nExp));
if (nExp < 0)
fValue /= fExpValue;
else
fValue *= fExpValue;
fValue *= fExpValue;
// If the original value was near DBL_MIN we got an overflow. Restore and
// bail out.
if (!std::isfinite(fValue))
return fOrigValue;
fValue = std::round(fValue);
fValue /= fExpValue;
if (nExp < 0)
fValue *= fExpValue;
else
fValue /= fExpValue;
// If the original value was near DBL_MAX we got an overflow. Restore and
// bail out.