Why is my number being rounded incorrectly?
Posted
by izb
on Stack Overflow
See other posts from Stack Overflow
or by izb
Published on 2010-03-24T16:43:21Z
Indexed on
2010/03/24
16:53 UTC
Read the original article
Hit count: 268
This feels like the kind of code that only fails in-situ, but I will attempt to adapt it into a code snippet that represents what I'm seeing.
float f = myFloat * myConstInt; /* Where myFloat==13.45, and myConstInt==20 */
int i = (int)f;
int i2 = (int)(myFloat * myConstInt);
After stepping through the code, i==269, and i2==268. What's going on here to account for the difference?
© Stack Overflow or respective owner