Commit 5c63bf67 authored by Klaus Aehlig's avatar Klaus Aehlig

Improve rounding accuracty in updateStatistics

Rearrange the arithmetic operations in statistics update
by avoiding small differences of large numbers.
Signed-off-by: default avatarKlaus Aehlig <aehlig@google.com>
Reviewed-by: default avatarPetr Pudlak <pudlak@google.com>
parent 0a446fc8
......@@ -197,7 +197,7 @@ updateStatistics (SumStatistics s) (x, y) = SumStatistics $ s + (y - x)
updateStatistics (StdDevStatistics n s var) (x, y) =
let !ds = y - x
!dss = y * y - x * x
!dnnvar = n * dss - (2 * s + ds) * ds
!dnnvar = (n * dss - 2 * s * ds) - ds * ds
!s' = s + ds
!var' = max 0 $ var + dnnvar / (n * n)
in StdDevStatistics n s' var'
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment