Search Results

Search found 950 results on 38 pages for 'floating'.

Page 13/38 | < Previous Page | 9 10 11 12 13 14 15 16 17 18 19 20  | Next Page >

  • Function Returning Negative Value

    - by Geowil
    I still have not run it through enough tests however for some reason, using certain non-negative values, this function will sometimes pass back a negative value. I have done a lot of manual testing in calculator with different values but I have yet to have it display this same behavior. I was wondering if someone would take a look at see if I am missing something. float calcPop(int popRand1, int popRand2, int popRand3, float pERand, float pSRand) { return ((((((23000 * popRand1) * popRand2) * pERand) * pSRand) * popRand3) / 8); } The variables are all contain randomly generated values: popRand1: between 1 and 30 popRand2: between 10 and 30 popRand3: between 50 and 100 pSRand: between 1 and 1000 pERand: between 1.0f and 5500.0f which is then multiplied by 0.001f before being passed to the function above Edit: Alright so after following the execution a bit more closely it is not the fault of this function directly. It produces an infinitely positive float which then flips negative when I use this code later on: pPMax = (int)pPStore; pPStore is a float that holds popCalc's return. So the question now is, how do I stop the formula from doing this? Testing even with very high values in Calculator has never displayed this behavior. Is there something in how the compiler processes the order of operations that is causing this or are my values simply just going too high? If the later I could just increase the division to 16 I think.

    Read the article

  • C# ? Can anyone explain the strange behaviour?

    - by partizan
    Hi, guys. Here is the example with comments: class Program { // first version of structure public struct D1 { public double d; public int f; } // during some changes in code then we got D2 from D1 // Field f type became double while it was int before public struct D2 { public double d; public double f; } static void Main(string[] args) { // Scenario with the first version D1 a = new D1(); D1 b = new D1(); a.f = b.f = 1; a.d = 0.0; b.d = -0.0; bool r1 = a.Equals(b); // gives true, all is ok // The same scenario with the new one D2 c = new D2(); D2 d = new D2(); c.f = d.f = 1; c.d = 0.0; d.d = -0.0; bool r2 = c.Equals(d); // false, oops! this is not the result i've expected for } } So, what do you think about this?

    Read the article

  • Pi/Infinite Numbers

    - by Ben Shelock
    I'm curious about infinite numbers in computing, in particular pi. For a computer to render a circle it would have to understand pi. But how can it if it is infinite? Am I looking too much into this? Would it just use a rounded value?

    Read the article

  • Rounding issue when adding floats in python, is this normal?

    - by thepearson
    I just wanted to know if this behavior is expected. If so, can someone explain to me why. This has probably been answered elsewhere I can't seem to find it using Google. Probably not searching with the right terms. I am running Ubuntu 10.04. Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> var = 10.0 >>> var 10.0 >>> var + 5 15.0 >>> var + 5.1 15.1 >>> var + 5.2 15.199999999999999 >>>

    Read the article

  • doing arithmetic upto two significant figures in Python?

    - by user248237
    I have two floats in Python that I'd like to subtract, i.e. v1 = float(value1) v2 = float(value2) diff = v1 - v2 I want "diff" to be computed upto two significant figures, that is compute it using %.2f of v1 and %.2f of v2. How can I do this? I know how to print v1 and v2 up to two decimals, but not how to do arithmetic like that. thanks.

    Read the article

  • C# WTF? Can anyone explain the strange behaviour?

    - by partizan
    Hi, guys. Here is the example with comments: class Program { // first version of structure public struct D1 { public double d; public int f; } // during some changes in code then we got D2 from D1 // Field f type became double while it was int before public struct D2 { public double d; public double f; } static void Main(string[] args) { // Scenario with the first version D1 a = new D1(); D1 b = new D1(); a.f = b.f = 1; a.d = 0.0; b.d = -0.0; bool r1 = a.Equals(b); // gives true, all is ok // The same scenario with the new one D2 c = new D2(); D2 d = new D2(); c.f = d.f = 1; c.d = 0.0; d.d = -0.0; bool r2 = c.Equals(d); // false, oops! this is not the result i've expected for } } So, what do you think about this?

    Read the article

  • int and float increment condition in for loop [on hold]

    - by Mazaya Jamil
    I am doing a program that involves integer and float number.Let say I want to calculate atx={1,1/2,2,3,4} and want to use for-loop. But I know the condition of increment for(x=1;x<=4;x++) as x++=x+1. I want to find the iteration at x={1,2,3,4} and at x={1/2}. But I do not have idea how to modify the for-loop statement; either to make the increment of 0.5 or 1. But if I set 0.5, I will get the answers for 5/2 and 7/2 instead.

    Read the article

  • Javascript function returning number with two decimals...?

    - by muzzledBYbrass
    I have a function to return points along a line and my return comes back with two decimal points...? For example, a return of my variable px will be something like -88.4029.032940598. vx is the x vector and mult is the distance of the line plus distance to calculate the point. Here is the operation that is returning these values: var mult = parseFloat(mag + theUnit); var px = coord_one.x_point + (vx * mult); console.log(px); Never have seen this before- I appreciate any and all help!

    Read the article

  • how to make javascript rearrangeable windows?

    - by ranganaMIT
    hi guys, I want to make a web page with several type of forms,each one on a different area that allows those areas to be rearranged and user will be able to change their position according to their preference.(X form to the left top, Y form to the right bottom etc.) i don't know what that type of windows are called, but think you can get a idea what I'm mentioning here.only thing i know is it can be achieved with javascript, i searched for it but i can't find the relevant tutorial or guide to self study. some one please tell me what they are, and how to make them and include in my web page? is there any thing that ease of making those thing except javascript?please provide me a source of studying and i really appreciate the instructions of the knowledged people.thanks in advance! regards, rangana.

    Read the article

  • optimization math computation (multiplication and summing)

    - by wiso
    Suppose you want to compute the sum of the square of the differences of items: $\sum_{i=1}^{N-1} (x_i - x_{i+1})^2$, the simplest code (the input is std::vector<double> xs, the ouput sum2) is: double sum2 = 0.; double prev = xs[0]; for (vector::const_iterator i = xs.begin() + 1; i != xs.end(); ++i) { sum2 += (prev - (*i)) * (prev - (*i)); // only 1 - with compiler optimization prev = (*i); } I hope that the compiler do the optimization in the comment above. If N is the length of xs you have N-1 multiplications and 2N-3 sums (sums means + or -). Now suppose you know this variable: sum = $x_1^2 + x_N^2 + 2 sum_{i=2}^{N-1} x_i^2$ Expanding the binomial square: $sum_i^{N-1} (x_i-x_{i+1})^2 = sum - 2\sum_{i=1}^{N-1} x_i x_{i+1}$ so the code becomes: double sum2 = 0.; double prev = xs[0]; for (vector::const_iterator i = xs.begin() + 1; i != xs.end(); ++i) { sum2 += (*i) * prev; prev = (*i); } sum2 = -sum2 * 2. + sum; Here I have N multiplications and N-1 additions. In my case N is about 100. Well, compiling with g++ -O2 I got no speed up (I try calling the inlined function 2M times), why?

    Read the article

  • c++ float subtraction rounding error

    - by Volkan Ozyilmaz
    I have a float value between 0 and 1. I need to convert it with -120 to 80. To do this, first I multiply with 200 after 120 subtract. When subtract is made I had rounding error. Let's look my example. float val = 0.6050f; val *= 200.f; Now val is 121.0 as I expected. val -= 120.0f; Now val is 0.99999992 I thought maybe I can avoid this problem with multiplication and division. float val = 0.6050f; val *= 200.f; val *= 100.f; val -= 12000.0f; val /= 100.f; But it didn't help. I have still 0.99 on my hand. Is there a solution for it? Edit: After with detailed logging, I understand there is no problem with this part of code. Before my log shows me "0.605", after I had detailed log and I saw "0.60499995946884155273437500000000000000000000000000" the problem is in different place.

    Read the article

  • C Different answers for a variable when running 'Debug' and 'Start without debug'

    - by Craz
    I keep getting this weird output from my code everytime I use the 'start without degugging' (ctrl-F5) as opposed to normal 'debug' (F5). When I try to find the following value of norm_differnece in debug (pressing F5) mode, it gives me the correct answer for norm_difference normdifference = 1.000000 but in 'start without debugging' (pressing ctrl-f5) the wrong output normdifference = 1456816083547664100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.000000 The following is a segment of code which is gives the output Note: X[] = is a array of stored DOUBLE values for(i=0;i<n;i++){ sum_difference += (pow((X[i*n]-X[i]),2)); } norm_difference = sqrt(norm_difference); for(i=0;i<n;i++){ sum_norm_1 += pow(X[i],2); } norm_1 = sqrt(norm_1); //Take square root of the sum of squares for the row printf("normdifference = %f \n norm_1 = %f \n",norm_difference,norm_1);

    Read the article

  • Is it possible in AvalonDock to programatically change DockableContent state to floating?

    - by Marqus
    In my application I'm loading windows from 'plugins', so my app doesn't know them until runtime. I'm creating DockableContent for each plugin and sets it's Content to Control returned by plugin. Each plugin tells, if it's window should be initially docked or floating, user can change it later. So I have an instance of DockableContent and I want to change it to FloatingWindow programatically. Changing content.DockableStyle to DockableStyle.Floating isn't enough, what else I have to do?

    Read the article

  • Problem with truncation of floating point values in DBSlayer.

    - by chrisdew
    When I run a query through DBslayer http://code.nytimes.com/projects/dbslayer the floating point results are truncated to a total of six digits (plus decimal point and negative sign when needed). { ... "lat":52.2228,"lng":-2.19906, ... } When I run the same query in MySQL, the results are as expected. | 52.22280884 | -2.19906425 | Firstly, am I correct in identifying DBSlayer as the cause of this effect? (Or the JSON library it uses, etc.) Secondly, is this floating point precision configurable within DBSlayer? Thanks, Chris. P.S. Ubuntu 9.10, x86_64 Path: . URL: http://dbslayer.googlecode.com/svn/trunk Repository Root: http://dbslayer.googlecode.com/svn Repository UUID: 5df2be84-4748-0410-afd4-f777a056bd0c Revision: 65 Node Kind: directory Schedule: normal Last Changed Author: dgottfrid Last Changed Rev: 65 Last Changed Date: 2008-03-28 22:52:46 +0000 (Fri, 28 Mar 2008)

    Read the article

  • Can someone confirm how Microsoft Excel 2007 internally represents numbers?

    - by Jon
    I know the IEEE 754 floating point standard by heart as I had to learn it for an exam. I know exactly how floating point numbers are used and the problems that they can have. I can manually do any operation on the binary representation of floating point numbers. However, I have not found a single source which unambiguously states that excel uses 64 bit floating point numbers to internally represent every single cell "type" in excel except for text. I have no idea whether some of the types use signed or unsigned integers and some use 64 bit floating point. I have found literally trillions of articles which 1) describe floating point numbers and then 2) talk about being careful with excel because of floating point numbers. I have not found a single statement saying "all types are 64 bit floating point numbers except text". I have not found a single statement which says "changing the type of a cell only changes its visual representation and not its internal representation, unless you change the type from text to some other type which is not text or you change some other type which is not text to text". This is literally all I want to know, and it's so simple and axiomatic that I am amazed that I can find trillions of articles and pages which talk around these statements but do not state them directly.

    Read the article

  • How can I simulate objects floating on water without a physics engine?

    - by user1075940
    In my game the water movement is done in a shader using Gerstner equations. The water movement looks realistic enough for a school project but I encounter serious problem when I wanted to do sailing on waves (similar to this). I managed to do collision with land by calculating quad's vertices and normals beneath ship, however same method can not be applied to water because XZ are displaced and Y is calculated in a shader :( How to approach this problem ? Is it possible to retrieve transformed grid from shader? Unfortunately no external physics libraries can be used.

    Read the article

  • How to simulate objects floating on water without a physics engine?

    - by user1075940
    In my game the water movement is done in a shader using Gerstner equations. The water movement looks realistic enough for a school project but I encounter serious problem when I wanted to do sailing on waves (similar to this). I managed to do collision with land by calculating quad's vertices and normals beneath ship, however same method can not be applied to water because XZ are displaced and Y is calculated in a shader :( How to approach this problem ? Is it possible to retrieve transformed grid from shader? Unfortunately no external physics libraries can be used.

    Read the article

  • How can I stop pixel seams appearing in adjacent mesh boundaries due to floating point imprecision?

    - by ufomorace
    Graphics cards are mathematically imprecise. So when some meshes are joined by their borders, the graphics card often makes mistakes and decides that some pixels at the seam represent neither object, and unwanted pixels appear. It's a natural behaviour on all graphics cards. How are such worries avoided in Pro Games? Batching? Shaders? Different tangent vectors? Merging? Overlaping seams? Dark backgrounds? Extra vertices at borders? Z precision? Camera distance tweaks? Screencap of a fix that ended up not working:

    Read the article

< Previous Page | 9 10 11 12 13 14 15 16 17 18 19 20  | Next Page >