Floating point equality and tolerances

Posted by doron on Stack Overflow See other posts from Stack Overflow or by doron
Published on 2013-07-01T12:28:00Z Indexed on 2013/07/01 16:21 UTC
Read the original article Hit count: 270

Comparing two floating point number by something like a_float == b_float is looking for trouble since a_float / 3.0 * 3.0 might not be equal to a_float due to round off error.

What one normally does is something like fabs(a_float - b_float) < tol.

How does one calculate tol?

Ideally tolerance should be just larger than the value of one or two of the least significant figures. So if the single precision floating point number is use tol = 10E-6 should be about right. However this does not work well for the general case where a_float might be very small or might be very large.

How does one calculate tol correctly for all general cases? I am interested in C or C++ cases specifically.

© Stack Overflow or respective owner

Related posts about c++

Related posts about c