Problem with truncation of floating point values in DBSlayer.
Posted
by chrisdew
on Stack Overflow
See other posts from Stack Overflow
or by chrisdew
Published on 2010-06-09T07:48:17Z
Indexed on
2010/06/09
7:52 UTC
Read the original article
Hit count: 354
When I run a query through DBslayer http://code.nytimes.com/projects/dbslayer the floating point results are truncated to a total of six digits (plus decimal point and negative sign when needed).
{ ... "lat":52.2228,"lng":-2.19906, ... }
When I run the same query in MySQL, the results are as expected.
| 52.22280884 | -2.19906425 |
Firstly, am I correct in identifying DBSlayer as the cause of this effect? (Or the JSON library it uses, etc.)
Secondly, is this floating point precision configurable within DBSlayer?
Thanks,
Chris.
P.S. Ubuntu 9.10, x86_64
Path: . URL: http://dbslayer.googlecode.com/svn/trunk Repository Root: http://dbslayer.googlecode.com/svn Repository UUID: 5df2be84-4748-0410-afd4-f777a056bd0c Revision: 65 Node Kind: directory Schedule: normal Last Changed Author: dgottfrid Last Changed Rev: 65 Last Changed Date: 2008-03-28 22:52:46 +0000 (Fri, 28 Mar 2008)
© Stack Overflow or respective owner