Search Results

Search found 22224 results on 889 pages for 'point of sale'.

Page 5/889 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

• Another floating point question

  - by jeffmax329

  I have read most of the posts on here regarding floating point, and I understand the basic underlying issue that using IEEE 754 (and just by the nature of storing numbers in binary) certain fractions cannot be represented. I am trying to figure out the following: If both Python and JavaScript use the IEEE 754 standard, why is it that executing the following in Python .1 + .1 Results in 0.20000000000000001 (which is to be expected) Where as in Javascript (in at least Chrome and Firefox) the answer is .2 However performing .1 + .2 In both languages results in 0.30000000000000004 In addition, executing var a = 0.3; in JavaScript and printing a results in 0.3 Where as doing a = 0.3 in Python results in 0.29999999999999999 I would like to understand the reason for this difference in behavior. In addition, many of the posts on OS link to a JavaScript port of Java's BigDecimal, but the link is dead. Does anyone have a copy?

  Read the article

• Using write to print floating point numbers.

  - by Tom

  Hi, As an exercise to achieve something larger, i'm trying to use write to print a floating point number. I haven't done this in a while. I must be doing something wrong because I cant get it to work. Here is my code #include <unistd.h> int main(){ float f = 4.5; write(1,&f,sizeof float); return 0; } However, when running it im getting ?@ Any thoughts? Thanks in advance.

  Read the article

• Floating point innacuracies

  - by Greg

  While writing a function which will perform some operation with each number in a range I ran into some problems with floating point inaccuracies. The problem can be seen in the code below: #include <iostream> using namespace std; int main() { double start = .99999, end = 1.00001, inc = .000001; int steps = (end - start) / inc; for(int i = 0; i <= steps; ++i) { cout << (start + (inc * i)) << endl; } } The problem is that the numbers the above program outputs look like this: 0.99999 0.999991 0.999992 0.999993 0.999994 0.999995 0.999996 0.999997 0.999998 0.999999 1 1 1 1 1 1.00001 1.00001 1.00001 1.00001 1.00001 1.00001 They only appear to be correct up to the first 1. What is the proper way to solve this problem?

  Read the article

• Find max integer size that a floating point type can handle without loss of precision

  - by Checkers

  Double has range more than a 64-bit integer, but its precision is less dues to its representation (since double is 64-bit as well, it can't fit more actual values). So, when representing larger integers, you start to lose precision in the integer part. #include <boost/cstdint.hpp> #include <limits> template<typename T, typename TFloat> void maxint_to_double() { T i = std::numeric_limits<T>::max(); TFloat d = i; std::cout << std::fixed << i << std::endl << d << std::endl; } int main() { maxint_to_double<int, double>(); maxint_to_double<boost::intmax_t, double>(); maxint_to_double<int, float>(); return 0; } This prints: 2147483647 2147483647.000000 9223372036854775807 9223372036854775800.000000 2147483647 2147483648.000000 Note how max int can fit into a double without loss of precision and boost::intmax_t (64-bit in this case) cannot. float can't even hold an int. Now, the question: is there a way in C++ to check if the entire range of a given integer type can fit into a loating point type without loss of precision? Preferably, it would be a compile-time check that can be used in a static assertion, and would not involve enumerating the constants the compiler should know or can compute.

  Read the article

• floating point equality in Python and in general

  - by eric.frederich

  I have a piece of code that behaves differently depending on whether I go through a dictionary to get conversion factors or whether I use them directly. The following piece of code will print 1.0 == 1.0 -> False But if you replace factors[units_from] with 10.0 and factors[units_to ] with 1.0 / 2.54 it will print 1.0 == 1.0 -> True #!/usr/bin/env python base = 'cm' factors = { 'cm' : 1.0, 'mm' : 10.0, 'm' : 0.01, 'km' : 1.0e-5, 'in' : 1.0 / 2.54, 'ft' : 1.0 / 2.54 / 12.0, 'yd' : 1.0 / 2.54 / 12.0 / 3.0, 'mile' : 1.0 / 2.54 / 12.0 / 5280, 'lightyear' : 1.0 / 2.54 / 12.0 / 5280 / 5.87849981e12, } # convert 25.4 mm to inches val = 25.4 units_from = 'mm' units_to = 'in' base_value = val / factors[units_from] ret = base_value * factors[units_to ] print ret, '==', 1.0, '->', ret == 1.0 Let me first say that I am pretty sure what is going on here. I have seen it before in C, just never in Python but since Python in implemented in C we're seeing it. I know that floating point numbers will change values going from a CPU register to cache and back. I know that comparing what should be two equal variables will return false if one of them was paged out while the other stayed resident in a register. Questions What is the best way to avoid problems like this?... In Python or in general. Am I doing something completely wrong? Side Note This is obviously part of a stripped down example but what I'm trying to do is come with with classes of length, volume, etc that can compare against other objects of the same class but with different units. Rhetorical Questions If this is a potentially dangerous problem since it makes programs behave in an undetermanistic matter, should compilers warn or error when they detect that you're checking equality of floats Should compilers support an option to replace all float equality checks with a 'close enough' function? Do compilers already do this and I just can't find the information.

  Read the article

• How to correctly pass a float from C# to C++ (dll)

  - by RavelT

  I'm getting huge differences when I pass a float from C# to C++. I'm passing a dynamic float wich changes over time. With a debuger I get this: c++ lonVel -0.036019072 float c# lonVel -0.029392920 float I did set my MSVC++2010 floating point model to /fp:fast wich should be the standard in .NET if I'm not mistaken, but this didnt help. Now I cant give out the code but I can show a fraction of it. From C# side it looks like this: namespace Example { public class Wheel { public bool loging = true; #region Members public IntPtr nativeWheelObject; #endregion Members public Wheel() { this.nativeWheelObject = Sim.Dll_Wheel_Add(); return; } #region Wrapper methods public void SetVelocity(float lonRoadVelocity,float latRoadVelocity){Sim.Dll_Wheel_SetVelocity(this.nativeWheelObject,lonRoadVelocity,latRoadVelocity);} #endregion Wrapper methods } internal class Sim { #region PInvokes [DllImport(pluginName, CallingConvention=CallingConvention.Cdecl)] public static extern void Dll_Wheel_SetVelocity(IntPtr wheel,float lonRoadVelocity,float latRoadVelocity); #endregion PInvokes } } And in C++ side @ exportFunctions.cpp: EXPORT_API void Dll_Wheel_SetVelocity(CarWheel* wheel,float lonRoadVelocity,float latRoadVelocity){ wheel->SetVelocity(lonRoadVelocity,latRoadVelocity);} So any sugestions on what I should do in order to get 1:1 results or atleast 99% correct results.

  Read the article

• Given a start and end point, how can I constrain the end point so the resulting line segment is horizontal, vertical, or 45 degrees?

  - by GloryFish

  I have a grid of letters. The player clicks on a letter and drags out a selection. Using Bresenham's Algorithm I can create a line of highlighted letters representing the player's selection. However, what I really want is to have the line segment be constrained to 45 degree angles (as is common for crossword-style games). So, given a start point and an end point, how can I find the line that passes through the start point and is closest to the end point? Bonus: To make things super sweet I'd like to get a list of points in the grid that the line passes through, and for super MEGA bonus points, I'd like to get them in order of selection (i.e. from start point to end point).

  Read the article

• About floating point precision and why do we still use it

  - by system_is_b0rken

  Floating point has always been troublesome for precision on large worlds. This article explains behind-the-scenes and offers the obvious alternative - fixed point numbers. Some facts are really impressive, like: "Well 64 bits of precision gets you to the furthest distance of Pluto from the Sun (7.4 billion km) with sub-micrometer precision. " Well sub-micrometer precision is more than any fps needs (for positions and even velocities), and it would enable you to build really big worlds. My question is, why do we still use floating point if fixed point has such advantages? Most rendering APIs and physics libraries use floating point (and suffer it's disadvantages, so developers need to get around them). Are they so much slower? Additionally, how do you think scalable planetary engines like outerra or infinity handle the large scale? Do they use fixed point for positions or do they have some space dividing algorithm?

  Read the article

• understanding floating point variables

  - by Syom

  There is some problem, i can't understand anyway. look at this code please <script type="text/javascript"> function math(x) { var y; y = x*10; alert(y); } </script> <input type="button" onclick="math(0.011)"> What must be alerted after i click on button? i think 0.11, but no, it alerts 0.10999999999999999 explain please this behavior. thanks in advance

  Read the article

• Decimal point issue on cocoa app

  - by Manuel Rocha

  I there, I'm trying making my first cocoa app, but I'm having problems with float numbers because of the regional settings. If I write on the TextBox the float number 1.2 I only can get the number 1, but If I write on the same TextBox the same float number but this time with the ',' sign instead (1,2) I can get the right float value. How can I bypass the regional settings? Kind Regards, Manuel Rocha

  Read the article

• C++ floating point precision

  - by Davinel

  double a = 0.3; std::cout.precision(20); std::cout << a << std::endl; result: 0.2999999999999999889 double a, b; a = 0.3; b = 0; for (char i = 1; i <= 50; i++) { b = b + a; }; std::cout.precision(20); std::cout << b << std::endl; result: 15.000000000000014211 So.. 'a' is smaller than it should be. But if we take 'a' 50 times - result will be bigger than it should be. Why is this? And how to get correct result in this case?

  Read the article

• how IEEE-754 floating point numbers work

  - by hatorade

  Let's say I have this: float i = 1.5 in binary, this float is represented as: 0 01111111 10000000000000000000000 I broke up the binary to represent the 'signed', 'exponent' and 'fraction' chunks. What I don't understand is how this represents 1.5. The exponent is 0 once you subtract the bias (127 - 127), and the fraction part with the implicit leading one is 1.1. How does 1.1 scaled by nothing = 1.5???

  Read the article

• floating point precision in ruby on rails model validations

  - by Chris Allison

  Hello I am trying to validate a dollar amount using a regex: ^[0-9]+\.[0-9]{2}$ This works fine, but whenever a user submits the form and the dollar amount ends in 0(zero), ruby(or rails?) chops the 0 off. So 500.00 turns into 500.0 thus failing the regex validation. Is there any way to make ruby/rails keep the format entered by the user, regardless of trailing zeros?

  Read the article

• how floating point numbers work in C

  - by hatorade

  Let's say I have this: float i = 1.5 in binary, this float is represented as: 0 01111111 10000000000000000000000 I broke up the binary to represent the 'signed', 'exponent' and 'fraction' chunks. What I don't understand is how this represents 1.5. The exponent is 0 once you subtract the bias (127 - 127), and the fraction part with the implicit leading one is 1.1. How does 1.1 scaled by nothing = 1.5???

  Read the article

• Floating Point Arithmetic - Modulo Operator on Double Type

  - by CrimsonX

  So I'm trying to figure out why the modulo operator is returning such a large unusual value. If I have the code: double result = 1.0d % 0.1d; it will give a result of 0.09999999999999995. I would expect a value of 0 Note this problem doesn't exist using the dividing operator - double result = 1.0d / 0.1d; will give a result of 10.0, meaning that the remainder should be 0. Let me be clear: I'm not surprised that an error exists, I'm surprised that the error is so darn large compared to the numbers at play. 0.0999 ~= 0.1 and 0.1 is on the same order of magnitude as 0.1d and only one order of magnitude away from 1.0d. Its not like you can compare it to a double.epsilon, or say "its equal if its < 0.00001 difference". I've read up on this topic on StackOverflow, in the following posts one two three, amongst others. Can anyone suggest explain why this error is so large? Any any suggestions to avoid running into the problems in the future (I know I could use decimal instead but I'm concerned about the performance of that).

  Read the article

• Rounding floating-point numbers to 4 decimal points

  - by Himadri

  I have two decimal numbers. I want those number to be same upto 4 decimal points without rounding. If numbers are different I want 2nd number to be replaced by 1st. What if condition should I write? Eg, 1. num1 = 0.94618976 num2 = 0.94620239 If we round these numbers upto 4 decimal then we get 0.9462 same number, but I don't want to round these numbers. 2. num1 = 0.94620239 num2 = 0.94639125 The one way I found is take absolute difference of both numbers say diff and then check the value. My problem is of checking the range of diff. Thank You.

  Read the article

• Floating point precision and physics calculations

  - by Vee

  The gravity Vector2 in my physics world is (0; 0.1). The number 0.1 is known to be problematic, since "it cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4". Having this value for the gravity gives me problems with collisions and creates quite nasty bugs. Changing the value to 0.11 solves these problems. Is there a more elegant solution that doesn't require changing the value at all?

  Read the article

• Rejigging a floating point equation ...

  - by Jamie

  I'd like to know if there is a way to improve the accuracy of calculating a slope. (This came up a few months back here). It seems by changing: float get_slope(float dXa, float dXb, float dYa, float dYb) { return (dXa - dXb)/(dYa - dYb); } to float get_slope(float dXa, float dXb, float dYa, float dYb) { return dXa/(dYa - dYb) - dXb/(dYa - dYb); } might be an improvement. Suggestions? Edit: It's precision I'm after, not efficiency.

  Read the article

• Floating point comparison in STL, BOOST

  - by Paul

  Is there in the STL or in Boost a set of generic simple comparison functions? The one I found are always requiring template parameters, and/or instantiation of a struct template. I'm looking for something with a syntax like : if ( is_greater(x,y) ) { ... } Which could be implemented as : template <typename T> bool is_greater(const T& x, const T& y) { return x > y + Precision<T>::eps; }

  Read the article

• Floating point precision nuances.

  - by user247077

  Hi, I found this code in NVIDIA's cuda SDK samples. void computeGold( float* reference, float* idata, const unsigned int len) { reference[0] = 0; double total_sum = 0; unsigned int i; for( i = 1; i < len; ++i) { total_sum += idata[i-1]; reference[i] = idata[i-1] + reference[i-1]; } // Here it should be okay to use != because we have integer values // in a range where float can be exactly represented if (total_sum != reference[i-1]) printf("Warning: exceeding single-precision accuracy. Scan will be inaccurate.\n"); } (C) Nvidia Corp Can somebody please tell me a case where the warning would be printed, and most importantly, why. Thank you very much.

  Read the article

• Weird floating point results

  - by pocoa

  >>> float(1) 1.0 >>> float(1) / 10 0.10000000000000001 >>> 4 + (float(1) / 10) 4.0999999999999996 4 + (float(1) / 10) is expected to be 4.10000000000000001 Is this a kind of bug?

  Read the article

• Time difference in seconds (as a floating point)

  - by pocoa

  >>> from datetime import datetime >>> t1 = datetime.now() >>> t2 = datetime.now() >>> delta = t2 - t1 >>> delta.seconds 7 >>> delta.microseconds 631000 Is there any way to get that as 7.631000 ? I can use time module, but I also need that t1 and t2 variables as DateTime objects. So if there is a way to do it with datettime, that would be great.

  Read the article

• Why Does My Floating Point Number 0.7 Evaluate At Less Than 0.7

  - by swamy

  void main() { float a=0.5,b=0.7; if(b<0.7) { if(a<0.5) printf("hello"); else printf("world"); } else printf("jsdf"); } Output: world Please give the explanation.

  Read the article

• Common Properties: Consolidating Loan, Purchase, Inventory and Sale tables into one Transaction tabl

  - by Frank Computer

  Pawnshop Application: I have separate tables for Loan, Purchase, Inventory & Sales transactions. Each tables rows are joined to their respective customer rows by: customer.pk [serial] = loan.fk [integer]; = purchase.fk [integer]; = inventory.fk [integer]; = sale.fk [integer]; Since there are so many common properties within the four tables, I consolidated the four tables into one table called "transaction", where a column: transaction.trx_type char(1) {L=Loan, P=Purchase, I=Inventory, S=Sale} Scenario: A customer initially pawns merchandise, makes a couple of interest payments, then decides he wants to sell the merchandise to the pawnshop, who then places merchandise in Inventory and eventually sells it to another customer. I designed a generic transaction table where for example: transaction.main_amount DECIMAL(7,2) in a loan transaction holds the pawn amount, in a purchase holds the purchase price, in inventory and sale holds sale price. This is clearly a denormalized design, but has made programming alot easier and improved performance. Any type of transaction can now be performed from within one screen, without the need to change to different tables.

  Read the article

• What is a Bias Value of Floating Point Numbers?

  - by mudge

  In learning how floating point numbers are represented in computers I have come across the term "bias value" that I do not quite understand. The bias value in floating point numbers has to do with the negative and positiveness of the exponent part of a floating point number. The bias value of a floating point number is 127 which means that 127 is always added to the exponent part of a floating point number. How does doing this help determine if the exponent is negative or positive or not?

  Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >