Search Results

Search found 5384 results on 216 pages for 'integer division'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 2/216 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

  • Java - simple division in Java ---> bug/feature?!

    - by msr
    Hello, Im astonished. Im trying this simple calculation in a Java application: System.out.println("b=" + (1 - 7/10)); Obviously Im wainting for "b=0.3" in the output but here's what I get: b=1 What?! Why this happens? If I make: System.out.println("b=" + (1-0.7)); I get the right result which is "b=0.3". What's going wrong here? Thanks!

    Read the article

  • Python - Number of Significant Digits in results of division

    - by russ
    Newbie here. I have the following code: myADC = 128 maxVoltage = 5.0 maxADC = 255.0 VoltsPerADC = maxVoltage/maxADC myVolts = myADC * VoltsPerADC print "myADC = {0: >3}".format(myADC) print "VoltsPerADC = {0: >7}".format(VoltsPerADC) print VoltsPerADC print "myVolts = {0: >7}".format(myVolts) print myVolts This outputs the following: myADC = 128 VoltsPerADC = 0.0196078 0.0196078431373 myVolts = 2.5098 2.50980392157 I have been searching for an explanation of how the number of significant digits is determined by default, but have had trouble locating an explanation that makes sense to me. This link link text suggests that by default the "print" statement prints numbers to 10 significant figures, but that does not seem to be the case in my results. How are the number of significant digits/precision determined? Can someone shed some light on this for me. Thanks in advance for your time and patience.

    Read the article

  • Java - simple division in Java ---> bug?!

    - by msr
    Hello, Im astonished. Im trying this simple calculation in a Java application: System.out.println("b=" + (1 - 7/10)); Obviously Im wainting for "b=0.3" in the output but here's what I get: b=1 What?! Why this happens? If I make: System.out.println("b=" + (1-0.7)); I get the right result which is "b=0.3". What's going wrong here? Thanks!

    Read the article

  • Zero division does not throw exception in nunit

    - by Boris
    Running the following C# code through NUnit yields Test.ControllerTest.TestSanity: Expected: <System.DivideByZeroException> But was: null So either no DivideByZeroException is thrown, or NUnit does not catch it. Similar to this question, but the answers he got, do not seem to work for me. This is using NUnit 2.5.5.10112, and .NET 4.0.30319. [Test] public void TestSanity() { Assert.Throws<DivideByZeroException>(new TestDelegate(() => DivideByZero())); } private void DivideByZero() { // Parse "0" to make sure to get an error at run time, not compile time. var a = (1 / Double.Parse("0")); } Any ideas?

    Read the article

  • division with wrong result

    - by PeterK
    Hi, I am trying to divide integers but get 0 as result. I just do not understand what i am doing wrong. I am using only int's in this example but get the same result testing with float or double. The code i use is: int wrongAnswers = askedQuestions - playerResult; int percentCorrect = (playerResult / askedQuestions) * 100; int percentWrong = (wrongAnswers / askedQuestions) * 100; NSLog(@"askedQuestions: %i", askedQuestions); NSLog(@"playerResult: %i", playerResult); NSLog(@"wrongAnswers: %i", wrongAnswers); NSLog(@"percentCorrect: %i", percentCorrect); NSLog(@"percentWrong: %i", percentWrong); NSLog(@"calc: %i", (wrongAnswers + playerResult)); NSLog(@"wrong answers %: %i %%", ((wrongAnswers / askedQuestions) * 100)); The result i get is: 2011-01-09 16:45:53.411 XX[8296:207] askedQuestions: 5 2011-01-09 16:45:53.412 XX[8296:207] playerResult: 2 2011-01-09 16:45:53.412 XX[8296:207] wrongAnswers: 3 2011-01-09 16:45:53.413 XX[8296:207] percentCorrect: 0 % 2011-01-09 16:45:53.414 XX[8296:207] percentWrong: 0 % 2011-01-09 16:45:53.414 XX[8296:207] calc: 5 2011-01-09 16:45:53.415 XX[8296:207] wrong answers : 0 % I would very much appreciate help :-)

    Read the article

  • String to Integer Smalltalk

    - by Anton
    Pretty simple question I need to get an integer from the user and I only know how to get a string from them. So if there is a way to get an integer from the user or to convert the string to an integer please let me know.

    Read the article

  • PostgreSQL - can't save items - "type integer but expression is of type character"

    - by user984621
    I am getting still over and over again this error, the column age has the type integer, I am saving into this column integer-value, I also tried to don't save nothing into this column, but still getting this error... Could anyone help me, how to fix that? PG::Error: ERROR: column "age" is of type integer but expression is of type character varying at character 102 HINT: You will need to rewrite or cast the expression. : INSERT INTO "user_details" ("created_at", "age", "updated_at", "user_id") VALUES ($1, $2, $3, $4) RETURNING "id"

    Read the article

  • Converting a string to an integer (Android)

    - by James Rattray
    I'm not 100% on how to do this so i'm asking... May seem stupid but... How do I convert a string into an integer? Background info: I have a textbox I have the user enter a number into... it then: EditText et = (EditText) findViewById(R.id.entry1); String hello = et.getText().toString(); gets the string 'hello' Now I want to convert it to a integer so I can get the number they typed in -will be used later on in code... Is there a way to get the edittext to a integer? -That would skip the middle man... If not, string to integer will be just fine... Thanks alot, James

    Read the article

  • Properly removing an Integer from a List<Integer>

    - by Yuval A
    Here's a nice pitfall I just encountered. Consider a list of integers: List<Integer> list = new ArrayList<Integer>(); list.add(5); list.add(6); list.add(7); list.add(1); Any educated guess on what happens when you execute list.remove(1)? What about list.remove(new Integer(1))? This can cause some nasty bugs. What is the proper way to differentiate between remove(int index), which removes an element from given index and remove(Object o), which removes an element by reference, when dealing with lists of integers? The main point to consider here is the one @Nikita mentioned - exact parameter matching takes precedence over auto-boxing.

    Read the article

  • SQL SERVER – Storing 64-bit Unsigned Integer Value in Database

    - by Pinal Dave
    Here is a very interesting question I received in an email just another day. Some questions just are so good that it makes me wonder how come I have not faced it first hand. Anyway here is the question - “Pinal, I am migrating my database from MySQL to SQL Server and I have faced unique situation. I have been using Unsigned 64-bit integer in MySQL but when I try to migrate that column to SQL Server, I am facing an issue as there is no datatype which I find appropriate for my column. It is now too late to change the datatype and I need immediate solution. One chain of thought was to change the data type of the column from Unsigned 64-bit (BIGINT) to VARCHAR(n) but that will just change the data type for me such that I will face quite a lot of performance related issues in future. In SQL Server we also have the BIGINT data type but that is Signed 64-bit datatype. BIGINT datatype in SQL Server have range of -2^63 (-9,223,372,036,854,775,808) to 2^63-1 (9,223,372,036,854,775,807). However, my digit is much larger than this number. Is there anyway, I can store my big 64-bit Unsigned Integer without loosing much of the performance of by converting it to VARCHAR.” Very interesting question, for the sake of the argument, we can ask user that there should be no need of such a big number or if you are taking about identity column I really doubt that if your table will grow beyond this table. Here the real question which I found interesting was how to store 64-bit unsigned integer value in SQL Server without converting it to String data type. After thinking a bit, I found a fairly simple answer. I can use NUMERIC data type. I can use NUMERIC(20) datatype for 64-bit unsigned integer value, NUMERIC(10) datatype for 32-bit unsigned integer value and NUMERIC(5) datatype for 16-bit unsigned integer value. Numeric datatype supports 38 maximum of 38 precision. Now here is another thing to keep in mind. Using NUMERIC datatype will indeed accept the 64-bit unsigned integer but in future if you try to enter negative value, it will also allow the same. Hence, you will need to put any additional constraint over column to only accept positive integer there. Here is another big concern, SQL Server will store the number as numeric and will treat that as a positive integer for all the practical purpose. You will have to write in your application logic to interpret that as a 64-bit Unsigned Integer. On another side if you are using unsigned integers in your application, there are good chance that you already have logic taking care of the same. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology Tagged: SQL Datatype

    Read the article

  • error of integer overflow

    - by user308565
    This the part of my OpenGL code, I am getting an error for : struct Ball { float x; float y; float rot; float dir; bool rmv; Ball* next; }; Ball* curBall; void addBall() { if (balls==NULL) { balls=new Ball; balls->next=NULL; curBall=balls; } else { curBall->next=new Ball; curBall=curBall->next; curBall->next=NULL; } curBall->x=((float)rand()/(float)(RAND_MAX+1))*(ww-1) +1; curBall->y=((float)rand()/(float)(RAND_MAX+1))*(wh-1) +1; curBall->dir=((float)rand()/(float)(RAND_MAX+1))*(2*PI-1) +1; curBall->rot=((float)rand()/(float)(RAND_MAX+1))*(359) +1; curBall->rmv=false; } error : In function ‘void addBall()’: file.cpp:120: warning: integer overflow in expression file.cpp:121: warning: integer overflow in expression file.cpp:122: warning: integer overflow in expression file.cpp:123: warning: integer overflow in expression

    Read the article

  • SQlite: Column format for unix timestamp; Integer types

    - by SF.
    Original problem: What is the right column format for a unix timestamp? The net is full of confusion: some posts claim SQLite has no unsigned types - either whatsoever, or with exception of the 64bit int type (but there are (counter-)examples that invoke UNSIGNED INTEGER). The data types page mentions it only in a bigint example. It also claims there is a 6-byte integer but doesn't give a name for it. Of course standard INTEGER being 4-byte signed signed stores unix timestamps as negative numbers. I've heard that some systems return 64-bit timestamps too. OTOH I'm not too fond of wasting 4 bytes to store 1 extra bit (top bit of timestamp), and even if I have to pick a bigger data format, I'd rather go for the 6-byte one. I've even seen a post that claims SQLite unix timestamp is of type REAL... Complete problem: Could someone please clarify that mess?

    Read the article

  • 8 byte Integer with Doctrine and PHP

    - by Rufinus
    Hi, the players: 64bit linux with php 5 (ZendFramework 1.10.2) PostgreSQL 7.3 Doctrine 1.2 Via a Flash/Flex client i get an 8byte integer value. the field in the database is an BIGINT (8 byte) PHP_INT_SIZE show that system supports 8byte integer. printing out the value in the code as it is and as intval() leads to this: Plain: 1269452776100 intval: 1269452776099 float rounding failure ? but what really driving me nuts is ERROR: invalid input syntax for integer: "1269452776099.000000"' when i try to use it in a query. like: Doctrine_Core::getTable('table')->findBy('external_id',$external_id); or Doctrine_Core::getTable('table')->findBy('external_id',intval($external_id)); How i am supposed to handle this ? or how can i give doctrine a floating point number which it should use on a bigint field ? Any help is much appreciated! TIA

    Read the article

  • Implicit casting Integer calculation to float in C++

    - by Ziddiri
    Is there any compiler that has a directive or a parameter to cast integer calculation to float implicitly. For example: float f = (1/3)*5; cout << f; the "f" is "0", because calculation's constants(1, 3, 10) are integer. I want to convert integer calculation with a compiler directive or parameter. I mean, I won't use explicit casting or ".f" prefix like that: float f = ((float)1/3)*5; or float f = (1.0f/3.0f)*5.0f; Do you know any c/c++ compiler which has any parameter to do this process without explicit casting or ".f" thing?

    Read the article

  • Integer variables at WIX

    - by Hila
    I would like to install a feature according to the brand. So in my brand.wxi I defined: <?define brand.FeatureLevel = 1 ?> And in my wxs I wrote: <Feature Id="FF" Title="FF" Level="$(var.brand.FeatureLevel)"> <ComponentRef Id="..." /> <ComponentRef Id="..." /> </Feature> This definition works fine (wheather I've placed 0 or 1 as FeatureLevel). My only problem is a warning I get at compilation time: The 'Level' attribute is invalid - The value '$(var.brand.FeatureLevel)' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:integer' - The string '$(var.brand.FeatureLevel)' is not a valid Integer value. Is there a way to fix this warning? Can I define integer variable? I couldn't find a way...

    Read the article

  • Fastest way to calculate a 128-bit integer modulo a 64-bit integer

    - by Paul Baker
    I have a 128-bit unsigned integer A and a 64-bit unsigned integer B. What's the fastest way to calculate A % B - that is the (64-bit) remainder from dividing A by B? I'm looking to do this in either C or assembly language, but I need to target the 32-bit x86 platform. This unfortunately means that I cannot take advantage of compiler support for 128-bit integers, nor of the x64 architecture's ability to perform the required operation in a single instruction.

    Read the article

  • Java Integer: what is faster comparison or subtraction?

    - by Vladimir
    I've found that java.lang.Integer implementation of compareTo method looks as follows: public int compareTo(Integer anotherInteger) { int thisVal = this.value; int anotherVal = anotherInteger.value; return (thisVal<anotherVal ? -1 : (thisVal==anotherVal ? 0 : 1)); } The question is why use comparison instead of subtraction: return thisVal - anotherVal;

    Read the article

  • Python, a smarter way of string to integer conversion

    - by Hellnar
    Hello I have written this code to convert string in such format "0(532) 222 22 22" to integer such as 05322222222 . class Phone(): def __init__(self,input): self.phone = input def __str__(self): return self.phone #convert to integer. def to_int(self): return int((self.phone).replace(" ","").replace("(","").replace(")","")) test = Phone("0(532) 222 22 22") print test.to_int() It feels very clumsy to use 3 replace methods to solve this. I am curious if there is a better solution?

    Read the article

  • define integer array fortran

    - by Praveen
    Hello friends, I am a newbie in Fortran. Can any1 tell me how to define an integer array in prior. E.g. I want to define an array with no.of days in 12 months. like... integer,allocatable(12,1) :: days days=[31,28,31,30,31,30,31,31,30,31,30,31] Is this syntax correct? If not, please let me know the correct one. Thanks Praveen

    Read the article

  • Beautify Integer values for comparing as strings

    - by Zahra
    Hi. For some reason I need to enter my integer values to database as string, then I want to run a query on them and compare those integers as strings. Is there any way to beautify integer numbers (between 1 and 1 US billion as an example) so I can compare them as strings? Thanks in advance.

    Read the article

  • Correctly assigning value to a Core Data attribute with an integer data-type

    - by Gordon Fontenot
    I'm missing something here, and feeling like an idiot about it. I'm using a UIPickerView in my app, and I need to assign the row number to a 32-bit integer attribute for a Core Data object. To do this, I am using this method: -(void)pickerView:(UIPickerView *)pickerView didSelectRow:(NSInteger)row inComponent:(NSInteger)component { object.integerValue = row; } This is giving me a warning: warning: passing argument 1 of 'setIntegerValue:' makes pointer from integer without a cast What am I mixing up here? --Edit 1-- Ok, so I can get rid of the warning by changing the method to do the following: NSNumber *number = [NSNumber numberWithInteger:row]; object.integerValue = rating; However, I still get a value of 0 for object.integerValue if I use NSLog to print it out. object.integerValue has a max value of 5, so I print out number instead, and then I'm getting a number above 62,000,000. Which doesn't seem right to me, since there are 5 rows. If I NSLog the row variable, I get a number between 0 and 5. So why do I end up with a completely different number after casting the number to NSNumber? --Edit 2-- Ok, so I'm realizing that there is some fundamental idea that I don't understand. I now understand that the 60 million + number can be cast back to the correct 0-5 number by using integerValue. So, it seems my question is how can I save an integer between 0-5 to the attribute if the NSNumber that is returned is over 60 million? Do I need to be using a different data type?

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >