c++ float subtraction rounding error
Posted
by
Volkan Ozyilmaz
on Stack Overflow
See other posts from Stack Overflow
or by Volkan Ozyilmaz
Published on 2014-08-21T19:17:28Z
Indexed on
2014/08/21
22:20 UTC
Read the original article
Hit count: 156
c++
|floating-point
I have a float value between 0 and 1. I need to convert it with -120 to 80. To do this, first I multiply with 200 after 120 subtract. When subtract is made I had rounding error. Let's look my example.
float val = 0.6050f;
val *= 200.f;
Now val is 121.0 as I expected.
val -= 120.0f;
Now val is 0.99999992
I thought maybe I can avoid this problem with multiplication and division.
float val = 0.6050f;
val *= 200.f;
val *= 100.f;
val -= 12000.0f;
val /= 100.f;
But it didn't help. I have still 0.99 on my hand.
Is there a solution for it?
Edit: After with detailed logging, I understand there is no problem with this part of code. Before my log shows me "0.605", after I had detailed log and I saw "0.60499995946884155273437500000000000000000000000000" the problem is in different place.
© Stack Overflow or respective owner