Search Results

Search found 1348 results on 54 pages for 'floating accuracy'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 14/54 | < Previous Page | 10 11 12 13 14 15 16 17 18 19 20 21  | Next Page >

  • MySQL float values jumping around on insert?

    - by dubayou
    So i have a SQL table setup as such CREATE TABLE IF NOT EXISTS `points` ( `id` int(11) NOT NULL auto_increment, `lat` float(10,6) NOT NULL, `lng` float(10,6) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM; And im inserting stuff like INSERT INTO `points` (`lat`, `lng`) VALUES ('89.123456','-12.123456'); Gives me a row with lat and lng being 89.123459 and -12.123455 Whats up?

    Read the article

  • Why does ghci say that 1.1 + 1.1 + 1.1 > 3.3 is True?

    - by Frerich Raabe
    I've been going through a Haskell tutorial recently and noticed this behaviour when trying some simple Haskell expressions in the interactive ghci shell: Prelude> 1.1 + 1.1 == 2.2 True Prelude> 1.1 + 1.1 + 1.1 == 3.3 False Prelude> 1.1 + 1.1 + 1.1 > 3.3 True Prelude> 1.1 + 1.1 + 1.1 3.3000000000000003 Does anybody know why that is?

    Read the article

  • Can anyone explain this strange behaviour?

    - by partizan
    Hi, guys. Here is the example with comments: class Program { // first version of structure public struct D1 { public double d; public int f; } // during some changes in code then we got D2 from D1 // Field f type became double while it was int before public struct D2 { public double d; public double f; } static void Main(string[] args) { // Scenario with the first version D1 a = new D1(); D1 b = new D1(); a.f = b.f = 1; a.d = 0.0; b.d = -0.0; bool r1 = a.Equals(b); // gives true, all is ok // The same scenario with the new one D2 c = new D2(); D2 d = new D2(); c.f = d.f = 1; c.d = 0.0; d.d = -0.0; bool r2 = c.Equals(d); // false! this is not the expected result } } So, what do you think about this?

    Read the article

  • Why does GLSL's arithmetic functions yield so different results on the iPad than on the simulator?

    - by cheeesus
    I'm currently chasing some bugs in my OpenGL ES 2.0 fragment shader code which is running on iOS devices. The code runs fine in the simulator, but on the iPad it has huge problems and some of the calculations yield vastly different results, I had for example 0.0 on the iPad and 4013.17 on the simulator, so I'm not talking about small differences which could be the result of some rounding errors. One of the things I noticed is that, on the iPad, float1 = pow(float2, 2.0); can yield results which are very different from the results of float1 = float2 * float2; Specifically, when using pow(x, 2.0) on a variable containing a larger negative number like -8, it seemed to return a value which satified the condition if (powResult <= 0.0). Also, the result of both operations (pow(x, 2.0) as well as x*x) yields different results in the simulator than on the iPad. Used floats are mediump, but I get the same stuff with highp. Is there a simple explanation for those differences? I'm narrowing the problem down, but it takes so much time, so maybe someone can help me here with a simple explanation.

    Read the article

  • How to make your website works on a projector?

    - by iyad al aqel
    I have a presentation tomorrow, and while I was trying to project my website using projector all the elements appeared mixed up. The problem is I used a software called "AXURE" to do the website. The HTML files produced by this software are crap. There's no one unified CSS file. The style is embedded for each element in the <div> tag and every element is positined by absolute pixels. How can make my website works on larger screens?

    Read the article

  • In .net, how do I choose between a Decimal and a Double

    - by Ian Ringrose
    We were discussing this the other day at work and I wish there was a Stackoverflow question I would point people at so here goes.) What is the difference between a Double and a Decimal? When (in what cases) should you always use a Double? When (in what cases) should you always use a Decimal? What’s the diver factors to consider in cases that don’t fall into one of the two camps above? (There a lot of questions that overlap this question, but they tend to be asking what someone should do in a given case, not how to decide in the general case)

    Read the article

  • Double multiplied by 100 and then cast to long is giving wrong value

    - by xyz
    I have the following code: Double i=17.31; long j=(long) (i*100); System.out.println(j); O/P : 1730 //Expected:1731 Double i=17.33; long j=(long) (i*100); System.out.println(j); O/P : 1732 //Expected:1733 Double i=17.32; long j=(long) (i*100); System.out.println(j); O/P : 1732 //Expected:1732{As expected} Double i=15.33; long j=(long) (i*100); System.out.println(j); O/P : 1533 //Expected:1533{as Expected} I have tried to Google but unable to find reason.I am sorry if the question is trivial.

    Read the article

  • Accordion "growing out" from its container - in IE7/8

    - by Richard Knop
    I think this problem is best explained by images. This is how my accordion looks: When you click on the small plus/minus icons the slides under each chapter will expand/collapse. However when the content in the accordion grows too tall, it grows out from its container. So if I click on more plus icons the accordion will look like this (not pretty): As you can see, the container is not growing taller together with the accordion and it does not look good. This problem only occurs in IE7 and IE8. It works in Firefox and Chrome. The HTML looks like this (simplified): <div id="content"> <div class="box2 rounded-corners"> <div class="chapters"> <h3><a href="/clientarea/view/archived-course/teid/133">Obsah</a></h3> <div id="accordion"> <ul> ... // accordion content - too long ... // accordion content - too long </ul> <div class="clear">&nbsp;</div> </div> <div class="clear">&nbsp;</div> </div> <div class="training-body"> ... // content to the right of the accordion </div> </div> </div> The CSS, again siplified: html, body { height: 100%; width: 100%; overflow: auto; } #content { background: white url('/images/background_middle.png') left top repeat-x; padding: 13px; min-height: 40em; height: auto !important; height: 40em; } /* this is the div with rounded corners */ #content .box2 { background: white; padding: 0 15px 15px; border: 1px solid #C5E3F8; position: relative; } /* left sidebar 98 #content div.chapters { float: left; width: 224px; } /* orange heading "OBSAH" */ #content div.chapters h3 { color: #ff6e19; text-transform: uppercase; font-size: .9em; text-align: center; padding-bottom: .5em; margin-top: 1em; margin-bottom: 0; } #content div.chapters h3 a { color: #ff6e19; } /* accordion */ #accordion { width: 226px; border-top: 1px solid #c5e3f8; } #accordion ul { padding-left: 0; margin-top: 0; margin-bottom: 0; margin-left: 0; } /* area to the right of the accordion */ #content div.training-body { float: left; padding-left: 0px; width: 748px; line-height: 1.3em; }

    Read the article

  • How to move value from the stack to ST(0)?

    - by George Edison
    I am having trouble believing the following code is the most efficient way to move a value from the stack to ST(0): .data var dd 4.2 tmp dd ? .code mov EAX, var push EAX ; top of stack now contains a value ; move it to ST(0) pop EAX mov tmp, EAX fld tmp Is the temporary variable really necessary? Further, is there an easier way to get a value from the stack to ST(0)?

    Read the article

  • Function Returning Negative Value

    - by Geowil
    I still have not run it through enough tests however for some reason, using certain non-negative values, this function will sometimes pass back a negative value. I have done a lot of manual testing in calculator with different values but I have yet to have it display this same behavior. I was wondering if someone would take a look at see if I am missing something. float calcPop(int popRand1, int popRand2, int popRand3, float pERand, float pSRand) { return ((((((23000 * popRand1) * popRand2) * pERand) * pSRand) * popRand3) / 8); } The variables are all contain randomly generated values: popRand1: between 1 and 30 popRand2: between 10 and 30 popRand3: between 50 and 100 pSRand: between 1 and 1000 pERand: between 1.0f and 5500.0f which is then multiplied by 0.001f before being passed to the function above Edit: Alright so after following the execution a bit more closely it is not the fault of this function directly. It produces an infinitely positive float which then flips negative when I use this code later on: pPMax = (int)pPStore; pPStore is a float that holds popCalc's return. So the question now is, how do I stop the formula from doing this? Testing even with very high values in Calculator has never displayed this behavior. Is there something in how the compiler processes the order of operations that is causing this or are my values simply just going too high? If the later I could just increase the division to 16 I think.

    Read the article

  • C# ? Can anyone explain the strange behaviour?

    - by partizan
    Hi, guys. Here is the example with comments: class Program { // first version of structure public struct D1 { public double d; public int f; } // during some changes in code then we got D2 from D1 // Field f type became double while it was int before public struct D2 { public double d; public double f; } static void Main(string[] args) { // Scenario with the first version D1 a = new D1(); D1 b = new D1(); a.f = b.f = 1; a.d = 0.0; b.d = -0.0; bool r1 = a.Equals(b); // gives true, all is ok // The same scenario with the new one D2 c = new D2(); D2 d = new D2(); c.f = d.f = 1; c.d = 0.0; d.d = -0.0; bool r2 = c.Equals(d); // false, oops! this is not the result i've expected for } } So, what do you think about this?

    Read the article

  • Pi/Infinite Numbers

    - by Ben Shelock
    I'm curious about infinite numbers in computing, in particular pi. For a computer to render a circle it would have to understand pi. But how can it if it is infinite? Am I looking too much into this? Would it just use a rounded value?

    Read the article

  • Rounding issue when adding floats in python, is this normal?

    - by thepearson
    I just wanted to know if this behavior is expected. If so, can someone explain to me why. This has probably been answered elsewhere I can't seem to find it using Google. Probably not searching with the right terms. I am running Ubuntu 10.04. Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> var = 10.0 >>> var 10.0 >>> var + 5 15.0 >>> var + 5.1 15.1 >>> var + 5.2 15.199999999999999 >>>

    Read the article

  • doing arithmetic upto two significant figures in Python?

    - by user248237
    I have two floats in Python that I'd like to subtract, i.e. v1 = float(value1) v2 = float(value2) diff = v1 - v2 I want "diff" to be computed upto two significant figures, that is compute it using %.2f of v1 and %.2f of v2. How can I do this? I know how to print v1 and v2 up to two decimals, but not how to do arithmetic like that. thanks.

    Read the article

  • C# WTF? Can anyone explain the strange behaviour?

    - by partizan
    Hi, guys. Here is the example with comments: class Program { // first version of structure public struct D1 { public double d; public int f; } // during some changes in code then we got D2 from D1 // Field f type became double while it was int before public struct D2 { public double d; public double f; } static void Main(string[] args) { // Scenario with the first version D1 a = new D1(); D1 b = new D1(); a.f = b.f = 1; a.d = 0.0; b.d = -0.0; bool r1 = a.Equals(b); // gives true, all is ok // The same scenario with the new one D2 c = new D2(); D2 d = new D2(); c.f = d.f = 1; c.d = 0.0; d.d = -0.0; bool r2 = c.Equals(d); // false, oops! this is not the result i've expected for } } So, what do you think about this?

    Read the article

  • int and float increment condition in for loop [on hold]

    - by Mazaya Jamil
    I am doing a program that involves integer and float number.Let say I want to calculate atx={1,1/2,2,3,4} and want to use for-loop. But I know the condition of increment for(x=1;x<=4;x++) as x++=x+1. I want to find the iteration at x={1,2,3,4} and at x={1/2}. But I do not have idea how to modify the for-loop statement; either to make the increment of 0.5 or 1. But if I set 0.5, I will get the answers for 5/2 and 7/2 instead.

    Read the article

  • Javascript function returning number with two decimals...?

    - by muzzledBYbrass
    I have a function to return points along a line and my return comes back with two decimal points...? For example, a return of my variable px will be something like -88.4029.032940598. vx is the x vector and mult is the distance of the line plus distance to calculate the point. Here is the operation that is returning these values: var mult = parseFloat(mag + theUnit); var px = coord_one.x_point + (vx * mult); console.log(px); Never have seen this before- I appreciate any and all help!

    Read the article

  • how to make javascript rearrangeable windows?

    - by ranganaMIT
    hi guys, I want to make a web page with several type of forms,each one on a different area that allows those areas to be rearranged and user will be able to change their position according to their preference.(X form to the left top, Y form to the right bottom etc.) i don't know what that type of windows are called, but think you can get a idea what I'm mentioning here.only thing i know is it can be achieved with javascript, i searched for it but i can't find the relevant tutorial or guide to self study. some one please tell me what they are, and how to make them and include in my web page? is there any thing that ease of making those thing except javascript?please provide me a source of studying and i really appreciate the instructions of the knowledged people.thanks in advance! regards, rangana.

    Read the article

  • optimization math computation (multiplication and summing)

    - by wiso
    Suppose you want to compute the sum of the square of the differences of items: $\sum_{i=1}^{N-1} (x_i - x_{i+1})^2$, the simplest code (the input is std::vector<double> xs, the ouput sum2) is: double sum2 = 0.; double prev = xs[0]; for (vector::const_iterator i = xs.begin() + 1; i != xs.end(); ++i) { sum2 += (prev - (*i)) * (prev - (*i)); // only 1 - with compiler optimization prev = (*i); } I hope that the compiler do the optimization in the comment above. If N is the length of xs you have N-1 multiplications and 2N-3 sums (sums means + or -). Now suppose you know this variable: sum = $x_1^2 + x_N^2 + 2 sum_{i=2}^{N-1} x_i^2$ Expanding the binomial square: $sum_i^{N-1} (x_i-x_{i+1})^2 = sum - 2\sum_{i=1}^{N-1} x_i x_{i+1}$ so the code becomes: double sum2 = 0.; double prev = xs[0]; for (vector::const_iterator i = xs.begin() + 1; i != xs.end(); ++i) { sum2 += (*i) * prev; prev = (*i); } sum2 = -sum2 * 2. + sum; Here I have N multiplications and N-1 additions. In my case N is about 100. Well, compiling with g++ -O2 I got no speed up (I try calling the inlined function 2M times), why?

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 10 11 12 13 14 15 16 17 18 19 20 21  | Next Page >