C precision of double: compiler dependent?

Posted by yCalleecharan on Stack Overflow See other posts from Stack Overflow or by yCalleecharan
Published on 2010-04-05T13:01:40Z Indexed on 2010/04/05 13:03 UTC
Read the original article Hit count: 320

Filed under:

Hi,on my 32-bit machine (with an Intel T7700 duo core), I have 15 precision digits for both double and long double types for the C language. I compared the parameters LDBL_DIG for long doubles and DBL_DIG for doubles and they are both 15. I got these answers using MVS2008. I was wondering if these results can be compiler dependent or do they just depend on my processor?

Thanks a lot...

Related posts about precision

SQL SERVER – Precision of SMALLDATETIME – A 1 Minute Precision

as seen on SQL Authority - Search for 'SQL Authority'
I am myself surprised that I am writing this post today. I am going to present one of the very known facts of SQL Server SMALLDATETIME datatype. Even though this is a very well-known datatype, many a time, I have seen developers getting confused with precision of the SMALLDATETIME datatype. The precision… >>> More
Using Taylor Series to Avoid Loss of Precision

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to use Taylor series to develop a numerically sound algorithm for solving a function. I've been at it for quite a while, but haven't had any luck yet. I'm not sure what I'm doing wrong. The function is f(x)=1 + x - sin(x)/ln(1+x) x~0 Also: why does loss of precision even occur in… >>> More
Delphi - Frac function losing precision.

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a TDateTime variable which is assigned a value at runtime of 40510.416667. When I extract the time to a TTime type variable using the Frac function, it sets it to 0.41666666666. Why has it changed the precision of the value and is there a workround to retain the precision from the original… >>> More
SQL Server cast fails with arithmetic overflow

as seen on Stack Overflow - Search for 'Stack Overflow'
According to the entry for decimal and numeric data types in SQL Server 2008 Books Online, precision is: p (precision) The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum… >>> More
Why don't computers store decimal numbers as a second whole number?

as seen on Programmers - Search for 'Programmers'
Computers have trouble storing fractional numbers where the denominator is something other than a solution to 2^x. This is because the first digit after the decimal is worth 1/2, the second 1/4 (or 1/(2^1) and 1/(2^2)) etc. Why deal with all sorts of rounding errors when the computer could have… >>> More

Developer IT

C precision of double: compiler dependent? - Developer IT

C precision of double: compiler dependent?

c

precision

compiler

Related posts about c

Related posts about precision

SQL SERVER – Precision of SMALLDATETIME – A 1 Minute Precision

Using Taylor Series to Avoid Loss of Precision

Delphi - Frac function losing precision.

SQL Server cast fails with arithmetic overflow

Why don't computers store decimal numbers as a second whole number?

Categories cloud