Understanding floating point problems
Posted
by
Maxim Gershkovich
on Stack Overflow
See other posts from Stack Overflow
or by Maxim Gershkovich
Published on 2011-01-12T01:49:05Z
Indexed on
2011/01/12
1:53 UTC
Read the original article
Hit count: 454
Could someone here please help me understand how to determine when floating point limitations will cause errors in your calculations. For example the following code.
CalculateTotalTax = function (TaxRate, TaxFreePrice) {
return ((parseFloat(TaxFreePrice) / 100) * parseFloat(TaxRate)).toFixed(4);
};
I have been unable to input any two values that have caused for me an incorrect result for this method. If I remove the toFixed(4) I can infact see where the calculations start to lose accuracy (somewhere around the 6th decimal place). Having said that though, my understanding of floats is that even small numbers can sometimes fail to be represented or have I misunderstood and can 4 decimal places (for example) always be represented accurately.
MSDN explains floats as such...
This means they cannot hold an exact representation of any quantity that is not a binary fraction (of the form k / (2 ^ n) where k and n are integers)
Now I assume this applies to all floats (inlcuding those used in javascript).
Fundamentally my question boils down to this. How can one determine if any specific method will be vulnerable to errors in floating point operations, at what precision will those errors materialize and what inputs will be required to produce those errors?
Hopefully what I am asking makes sense.
© Stack Overflow or respective owner