Python - Number of Significant Digits in results of division
Posted
by russ
on Stack Overflow
See other posts from Stack Overflow
or by russ
Published on 2010-04-01T10:56:55Z
Indexed on
2010/04/01
11:03 UTC
Read the original article
Hit count: 332
Newbie here. I have the following code:
myADC = 128
maxVoltage = 5.0
maxADC = 255.0
VoltsPerADC = maxVoltage/maxADC
myVolts = myADC * VoltsPerADC
print "myADC = {0: >3}".format(myADC)
print "VoltsPerADC = {0: >7}".format(VoltsPerADC)
print VoltsPerADC
print "myVolts = {0: >7}".format(myVolts)
print myVolts
This outputs the following:
myADC = 128
VoltsPerADC = 0.0196078
0.0196078431373
myVolts = 2.5098
2.50980392157
I have been searching for an explanation of how the number of significant digits is determined by default, but have had trouble locating an explanation that makes sense to me. This link link text suggests that by default the "print" statement prints numbers to 10 significant figures, but that does not seem to be the case in my results.
How are the number of significant digits/precision determined?
Can someone shed some light on this for me.
Thanks in advance for your time and patience.
© Stack Overflow or respective owner