• Iustin Pop's avatar
    Rework unit parsing · 1cdcf8f3
    Iustin Pop authored
    Due to how conversions were implemented previously, 1TB failed to
    parse on 32-bit, as we were overflowing during computation, even
    though the final result would fit easily.
    This patch moves the parsing of the scaling factor to a separate
    function, and all the conversions are done via the Rational type
    (which has unlimited arbitrary precision), and conversion to the
    desired type only happens at the last step.
    The unit-tests are adjusted too, unfortunately they use the same
    algorithm as the code… suggestions on how to improve things are
    Signed-off-by: default avatarIustin Pop <iustin@google.com>
    Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
QC.hs 61.6 KB