Numerical precision of double type in Visual C++ 2008 Express debugger
- by damik
I'm using Visual C++ 2008 Express Edition and when i debug code:
double x = 0.2;
I see in debugging tooltip on x 0.20000000000000001
but:
typedef numeric_limits< double > double_limit;
int a = double_limit::digits10
gives me: a = 15
Why results in debugger are longer than maybe ?
What is this strange precision based on ?
My CPU is Intel Core 2 Duo T7100