Rails precision error

Posted by dontangg on Stack Overflow See other posts from Stack Overflow or by dontangg
Published on 2011-01-04T07:26:18Z Indexed on 2011/01/04 7:53 UTC
Read the original article Hit count: 282

Filed under:

ruby-on-rails

|

precision

When I run this in my Rails application:

my_envelope.transactions.sum(:amount)

This SQL is shown in the log files:

SQL (0.3ms)  SELECT SUM("transactions"."amount") AS sum_id FROM "transactions" WHERE (envelope_id = 834498537)

And this value is returned:

<BigDecimal:1011be570,'0.2515999999 9999997E2',27(27)>

As you can see, the value is 25.159999. It should be 25.16. When I run the same SQL on the database myself, the correct value is returned.

I'm a little confused because I know that there are precision problems with Floats, but it is returning a BigDecimal. The SQL column type is decimal. Any ideas?

© Stack Overflow or respective owner

Related posts about ruby-on-rails

Ruby on Rails - How can I start? [closed]

as seen on Programmers - Search for 'Programmers'
I have misconception in understanding the relationship between Ruby language and Ruby on Rails Framework. Because of 'I am an absolute beginner' in web development I have no idea if I have to grasp the fundamentals of Ruby before I go with Ruby on Rails! I also want to ask who is behind both Ruby… >>> More
Ruby on rails: Image downloads with Authentication/Authorization/Time outs

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Guys, I'm having few doubts on implementing file downloads. I'm creating an app where I use attachment_fu with Amazon s3 to upload files. Things are working pretty well so far on uploading side. Now its the time to start the file downloads. Here is what I need, a logged in user search and browse… >>> More
DES3 decryption in Ruby on Rails

as seen on Stack Overflow - Search for 'Stack Overflow'
My RoR server receives a string, that was encrypted in C++ application using des3 with base64 encoding The cipher object is created so: cipher = OpenSSL::Cipher::Cipher::new("des3") cipher.key = key_str cipher.iv = iv_str key_str and iv_str: are string representations of key and initialization… >>> More
Ruby on Rails deployment, on "thin" server with lot of attachments

as seen on Stack Overflow - Search for 'Stack Overflow'
A lot of PDFs are stored inside MySQL as a BLOB field for each PDF file. The average file size is 500K each. The Rails app will stream the :binary data as file downloads, where there is a user click on the download link. Assume there is a maximum of 5 users downloading 5 PDFs concurrently, what… >>> More
Apply Behavior Driven Development to Ruby on Rails with Rspec

as seen on Internet.com - Search for 'Internet.com'
Applying the Behavior Driven Development (BDD) methodology to your Rails development takes you beyond traditional Test Driven Development. Learn how to do it with the Ruby-based BDD framework RSpec. >>> More

Related posts about precision

SQL SERVER – Precision of SMALLDATETIME – A 1 Minute Precision

as seen on SQL Authority - Search for 'SQL Authority'
I am myself surprised that I am writing this post today. I am going to present one of the very known facts of SQL Server SMALLDATETIME datatype. Even though this is a very well-known datatype, many a time, I have seen developers getting confused with precision of the SMALLDATETIME datatype. The precision… >>> More
Using Taylor Series to Avoid Loss of Precision

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to use Taylor series to develop a numerically sound algorithm for solving a function. I've been at it for quite a while, but haven't had any luck yet. I'm not sure what I'm doing wrong. The function is f(x)=1 + x - sin(x)/ln(1+x) x~0 Also: why does loss of precision even occur in… >>> More
Delphi - Frac function losing precision.

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a TDateTime variable which is assigned a value at runtime of 40510.416667. When I extract the time to a TTime type variable using the Frac function, it sets it to 0.41666666666. Why has it changed the precision of the value and is there a workround to retain the precision from the original… >>> More
SQL Server cast fails with arithmetic overflow

as seen on Stack Overflow - Search for 'Stack Overflow'
According to the entry for decimal and numeric data types in SQL Server 2008 Books Online, precision is: p (precision) The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum… >>> More
Why don't computers store decimal numbers as a second whole number?

as seen on Programmers - Search for 'Programmers'
Computers have trouble storing fractional numbers where the denominator is something other than a solution to 2^x. This is because the first digit after the decimal is worth 1/2, the second 1/4 (or 1/(2^1) and 1/(2^2)) etc. Why deal with all sorts of rounding errors when the computer could have… >>> More