Commit 5c63bf67 authored by Klaus Aehlig's avatar Klaus Aehlig
Browse files

Improve rounding accuracty in updateStatistics



Rearrange the arithmetic operations in statistics update
by avoiding small differences of large numbers.
Signed-off-by: default avatarKlaus Aehlig <aehlig@google.com>
Reviewed-by: default avatarPetr Pudlak <pudlak@google.com>
parent 0a446fc8
......@@ -197,7 +197,7 @@ updateStatistics (SumStatistics s) (x, y) = SumStatistics $ s + (y - x)
updateStatistics (StdDevStatistics n s var) (x, y) =
let !ds = y - x
!dss = y * y - x * x
!dnnvar = n * dss - (2 * s + ds) * ds
!dnnvar = (n * dss - 2 * s * ds) - ds * ds
!s' = s + ds
!var' = max 0 $ var + dnnvar / (n * n)
in StdDevStatistics n s' var'
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment