Floating point equality and tolerances
Posted
by
doron
on Stack Overflow
See other posts from Stack Overflow
or by doron
Published on 2013-07-01T12:28:00Z
Indexed on
2013/07/01
16:21 UTC
Read the original article
Hit count: 270
Comparing two floating point number by something like a_float == b_float
is looking for trouble since a_float / 3.0 * 3.0
might not be equal to a_float
due to round off error.
What one normally does is something like fabs(a_float - b_float) < tol
.
How does one calculate tol
?
Ideally tolerance should be just larger than the value of one or two of the least significant figures. So if the single precision floating point number is use tol = 10E-6
should be about right. However this does not work well for the general case where a_float
might be very small or might be very large.
How does one calculate tol
correctly for all general cases? I am interested in C or C++ cases specifically.
© Stack Overflow or respective owner