Significant figures in the decimal module
Posted
by Jason Baker
on Stack Overflow
See other posts from Stack Overflow
or by Jason Baker
Published on 2008-09-27T18:47:54Z
Indexed on
2010/04/15
20:13 UTC
Read the original article
Hit count: 432
So I've decided to try to solve my physics homework by writing some python scripts to solve problems for me. One problem that I'm running into is that significant figures don't always seem to come out properly. For example this handles significant figures properly:
from decimal import Decimal
>>> Decimal('1.0') + Decimal('2.0')
Decimal("3.0")
But this doesn't:
>>> Decimal('1.00') / Decimal('3.00')
Decimal("0.3333333333333333333333333333")
So two questions:
- Am I right that this isn't the expected amount of significant digits, or do I need to brush up on significant digit math?
- Is there any way to do this without having to set the decimal precision manually? Granted, I'm sure I can use numpy to do this, but I just want to know if there's a way to do this with the decimal module out of curiosity.
© Stack Overflow or respective owner