Do not easily round the result of decimal calculation

If you're programming a little, you may have heard that "floating point numbers have an error".

But are you always aware of that? I wasn't aware of it and took the step as expected.

Common examples

One day, I had to pay 0.5% interest on the user's deposit balance, rounded down to the nearest whole number. I wrote this code because I thought it was a simple calculation.

JavaScript


Math.floor(value * 1.005)

…… ** This is a mistake. ** **

Let's try putting 1000 in value. I feel like adding 0.5% to 1000 gives 1005, but in reality it becomes 1004. The above example is JavaScript, but in most major languages the result is the same. Let's try it at hand.

Why is this happening? If you try to output 1000 * 1.005 as it is, it will be 1004.9999999999999. This value has just not reached 1005, so if you truncate it, it will be 1004. Floating point numbers cannot accurately represent 1.005, so it seems that an error has occurred.

Watch out for truncation and rounding up

This error is really easy to occur. All of the following examples do not produce the expected results.

JavaScript


Math.floor(0.6 / 0.2) // 2
0.6 / 0.2             // 2.9999999999999996

JavaScript


//Consumption tax rounded up to the nearest whole number
Math.ceil(900 * 1.08) // 973
900 * 1.08            // 972.0000000000001

JavaScript


let f = 0;
for (let i = 0; i < 10; i++) {
  f += 0.1;
}
Math.floor(f) // 0
f             // 0.9999999999999999

It is better to think that the calculation result of floating point number is not an integer.

Workaround

You can avoid this problem by rounding the calculation result to a sufficiently low digit before rounding it.

JavaScript


//Rounded to 4 decimal places
Math.floor((value * 1.005).toFixed(3))

Java


Math.floor(String.format("%.3f", value * 1.005))

PHP


floor(round($value * 1.005, 3))

It may not be suitable for situations where rigorous calculations are required, but for most applications this should be sufficient. You may want to extend the Math class so that it can be called from the entire application.

17/2/25 postscript

I received a comment. Decimal types allow for error-free calculations without weird implementations. In Java, the BigDecimal class can be used as standard, and JavaScript also has a library called decimal.js. It's better to use something like this than to implement it yourself. (Addition so far)

Summary

Be careful with decimal calculations.

Recommended Posts

Do not easily round the result of decimal calculation
Filter the result of BindingResult [Spring]
Get the result of POST in Java
Easily measure the size of Java Objects
Back calculation of the transition of the internal seed of Random
ActiveSupport underscore is not the inverse of camelize
Do not give jsessionid at the first access
How to display the result of form input