Search Results

Search found 1806 results on 73 pages for 'numeric precision'.

Page 10/73 | < Previous Page | 6 7 8 9 10 11 12 13 14 15 16 17  | Next Page >

• Compare double till precision 2

  - by Ram

  Hi, I want to compare two double values till the second digit from decimal point. I am using following statement but I think it is not working properly. dbl1.ToString("g2", CultureInfo.InvariantCulture).Equals( dbl2.ToString("g2", CultureInfo.InvariantCulture)) Am I missing anything here?

  Read the article

• matlab precision determinant problem

  - by ldigas

  I have the following program format compact; format short g; clear; clc; L = 140; J = 77; Jm = 10540; G = 0.8*10^8; d = L/3; for i=1:500000 omegan=1.+0.0001*i; a(1,1) = ((omegan^2)*(Jm/(G*J))*d^2)-2; a(1,2) = 2; a(1,3) = 0; a(1,4) = 0; a(2,1) = 1; a(2,2) = ((omegan^2)*(Jm/(G*J))*d^2)-2; a(2,3) = 1; a(2,4) = 0; a(3,1) = 0; a(3,2) = 1; a(3,3) = ((omegan^2)*(Jm/(G*J))*d^2)-2; a(3,4) = 1; a(4,1) = 0; a(4,2) = 0; a(4,3) = 2; a(4,4) = ((omegan^2)*(Jm/(G*J))*d^2)-2; if(abs(det(a))<1E-10) sprintf('omegan= %8.3f det= %8.3f',omegan,det(a)) end end Analytical solution of the above system, and the same program written in fortran gives out values of omegan equal to 16.3818 and 32.7636 (fortran values; analytical differ a little, but they're there somewhere). So, now I'm wondering ... where am I going wrong with this ? Why is matlab not giving the expected results ? (this is probably something terribly simple, but it's giving me headaches)

  Read the article

• how to increase the precision in Locate Me app

  - by thndrkiss

  Hi, I just installed the locate me app and started to understand the CLLocation stuff. I need to know how to increase the number of digits after the lat and long numbers displayed. Can any one help me. currently it shows 17.5968,82.8486. I need more number of digits after decimal(6 digits after decimal.

  Read the article

• floating point precision in ruby on rails model validations

  - by Chris Allison

  Hello I am trying to validate a dollar amount using a regex: ^[0-9]+\.[0-9]{2}$ This works fine, but whenever a user submits the form and the dollar amount ends in 0(zero), ruby(or rails?) chops the 0 off. So 500.00 turns into 500.0 thus failing the regex validation. Is there any way to make ruby/rails keep the format entered by the user, regardless of trailing zeros?

  Read the article

• Reading numeric Excel data as text using xlrd in Python

  - by Brian

  Hi guys, I am trying to read in an Excel file using xlrd, and I am wondering if there is a way to ignore the cell formatting used in Excel file, and just import all data as text? Here is the code I am using for far: import xlrd xls_file = 'xltest.xls' xls_workbook = xlrd.open_workbook(xls_file) xls_sheet = xls_workbook.sheet_by_index(0) raw_data = [['']*xls_sheet.ncols for _ in range(xls_sheet.nrows)] raw_str = '' feild_delim = ',' text_delim = '"' for rnum in range(xls_sheet.nrows): for cnum in range(xls_sheet.ncols): raw_data[rnum][cnum] = str(xls_sheet.cell(rnum,cnum).value) for rnum in range(len(raw_data)): for cnum in range(len(raw_data[rnum])): if (cnum == len(raw_data[rnum]) - 1): feild_delim = '\n' else: feild_delim = ',' raw_str += text_delim + raw_data[rnum][cnum] + text_delim + feild_delim final_csv = open('FINAL.csv', 'w') final_csv.write(raw_str) final_csv.close() This code is functional, but there are certain fields, such as a zip code, that are imported as numbers, so they have the decimal zero suffix. For example, is there is a zip code of '79854' in the Excel file, it will be imported as '79854.0'. I have tried finding a solution in this xlrd spec, but was unsuccessful.

  Read the article

• Arbitrary precision type

  - by Andrew S.

  Is there such a type\implementation in C#? It is needed to calculate figures up to 1010,000 magnitude.

  Read the article

• PHP Curl and Loop based on a numeric value

  - by danit

  Im using the Twitter API to collect the number of tweets I've favorited, well to be accurate the total pages of favorited tweets. I use this URL: http://api.twitter.com/1/users/show/username.xml I grab the XML element 'favorites_count' For this example lets assume favorites_count=5 The Twitter API uses this URL to get the favorties: http://twitter.com/favorites.xml (Must be authenticated) You can only get the last 20 favorties using this URL, however you can alter the URL to include a 'page' option by adding: ?page=3 to the end of the favorites URL e.g. http://twitter.com/favorites.xml?page=2 So what I need to do is use CURL (I think) to collect the favorite tweets, but using the URL: http://twitter.com/favorites.xml?page=1 http://twitter.com/favorites.xml?page=2 http://twitter.com/favorites.xml?page=3 http://twitter.com/favorites.xml?page=4 etc... Some kind of loop to visit each URL, and collect the Tweets and then output the cotents. Can anyone help with this: - Need to use CURL to authenticate - Collect the number of pages of tweets (Already scripted this) - Then use a loop to go through each page URL based on the pages value?

  Read the article

• Numeric comparison difficulty in R

  - by Matt Parker

  I'm trying to compare two numbers in R as a part of a if-statement condition: (a-b) >= 0.5 In this particular instance, a = 0.58 and b = 0.08... and yet (a-b) >= 0.5 is false. I'm aware of the dangers of using == for exact number comparisons, and this seems related: (a - b) == 0.5) is false, while all.equal((a - b), 0.5) is true. The only solution I can think of is to have two conditions: (a-b) > 0.5 | all.equal((a-b), 0.5). This works, but is that really the only solution? Should I just swear off of the = family of comparison operators forever?

  Read the article

• Qwt setAxisScale() seems to lock up application when given numbers with greater precision than 2e-07

  - by Dane Larsen

  I'm using Qwt for some scientific graphing, and I'm working with some fairly small numbers, the smallest being around 1.0e-22. I'm trying to call setAxisScale(xaxis, xmin, xmax) //xmin = 0, xmax = 2.0e-10 But when I do, the application locks up. I haven't found anything in the documentation that refers to a minimum value. Xmin and xmax are both doubles, so that shouldn't be a problem. Is this a bug in Qwt, or am I doing something wrong? Thanks in advance

  Read the article

• Binning a numeric variable in R

  - by McPeterson

  I have a vector X that contains positive numbers that I want to bin/discretize. For this vector, I want the numbers [0, 10) to show up just as they exist in the vector, but numbers [10,∞) to be 10+. I'm using: x <- c(0,1,3,4,2,4,2,5,43,432,34,2,34,2,342,3,4,2) binned.x <- as.factor(ifelse(x > 10,"10+",x)) but this feels klugey to me. Does anyone know a better solution or a different approach? mcpeterson

  Read the article

• A database of questions with unambiguous numeric answers.

  - by dreeves

  I (and co-hackers) are building a sort of trivia game inspired by this blog post: http://messymatters.com/calibration. The idea is to give confidence intervals and learn how to be calibrated (when you're "90% sure" you should be right 90% of the time). We're thus looking for, ideally, thousands of questions with unambiguous numerical answers. Also, they shouldn't be too boring. There are a lot of random statistics out there -- eg, enclosed water area in different countries -- that would make the game mind-numbing. Things like release dates of classic movies are more interesting (to most people). Other interesting ones we've found include Olympic records, median incomes for different professions, dates of famous inventions, and celebrity ages. Scraping things like above, by the way, was my reason for asking this question: http://stackoverflow.com/questions/2611418/scrape-html-tables So, if you know of other sources of interesting numerical facts (in a parsable form) I'm eager for pointers to them. Thanks!

  Read the article

• Parsing specific numeric data from csv file using python

  - by KJ Lim

  Good morning. I have series of data in cvs file like below, 1,,, 1,137.1,1198,1.6 2,159,300,0.4 3,176,253,0.3 4,197,231,0.3 5,198,525,0.7 6,199,326,0.4 7,215,183,0.2 8,217.1,178,0.2 9,244.2,416,0.5 10,245.1,316,0.4 I want to extract specific data from second column for example 217.1 and 245.1 and have them concatenated into a new file like, 8,217.1,178,0.2 10,245.1,316,0.4 I use cvs module to read my cvs file, but, I can't extract specific data as I desire. Could anyone kindly please help me. Thank you.

  Read the article

• PHP: Format a numeric string by inserting comma

  - by gameover

  I need a way to convert a number into formatted way by inserting comma at suitable places. Can it be done using regex? Example: 12345 => 12,345 1234567 =>1,234,567

  Read the article

• Numeric operations over SHA-1 generated keys in C#

  - by webdreamer

  I'm trying to implement a Chord distributed hash table. I want to use SHA-1 as the hash function to generate node ids and map values to the DHT. However, I'll need to use numerical operations on the SHA-1 generated key, such as a modulo, for example. I wonder in which type of variable should I put the array of bytes I get, and how can I convert from one to another.

  Read the article

• Remove numeric prefix from string - PHP regex

  - by Pekka

  Would somebody care to help me out with a regex to reliably recognize and remove any number, followed by a dot, in the beginning of a string? So that 1. Introduction becomes Introduction and 1290394958595. Appendix A becomes Appendix A

  Read the article

• Reading numeric Date value from CSV file to data.frame in "R"

  - by Dick Eshelman

  D <- read.csv("sample1.csv", header = FALSE, sep = ",") D V1 V2 V3 V4 1 20100316 109825 352120 239065 2 20100317 108625 352020 239000 3 20100318 109125 352324 241065 D[,1] [1] 20100316 20100317 20100318 In the above example how do I get the data in D[,1] to be read, and stored as date values: 2010-03-16, 2010-03-17, 2010-03-18 ? I have lots of data files in this format. TIA,

  Read the article

• Sorting a vector of (double precision) reals and obtain their order

  - by Philipp

  Hello everyone, in C++ would like to sort a lengthy (2^20) vector of reals, obviously sort() does the trick. Having used R before I was used to the nice order() function which yields the permutation that leads to the sorted vector. Probably someone has done this in C++, maybe it's just my weak google-Fu that prevents me from finding it. And yeah, obivously my C++ newbness could stop me from spotting something straightforward. Example: x = {24, 55, 22, 1} then the permutation perm = {3, 2, 0, 1} maps the original x to the sorted x in ascending order. I can probably implement some bubble sort which does not only sort x but performs the same transpositions on the vector {0,1,2,...} and outputs both, but I believe someone must have thought about it and especially have done it efficiently. Thank you very much, Philipp

  Read the article

• Calculate date from numeric value

  - by elias

  The number 71867806 represents the present day, with the smallest unit of days. How can I calculate the currente date from it? (or) convert it into an Unix timestamp? Solution shouldn't use language depending features. Thanks!

  Read the article

• Database: Storing Dates as Numeric Values

  - by Chin

  I'm considering storing some date values as ints. i.e 201003150900 Excepting the fact that I lose any timezone information, is there anything else I should be concerned about with his solution? Any queries using this column would be simple 'where after or before' type lookups. i.e Where datefield is less than 201103000000 (before March next year). currently the app is using MSSQL2005. Any pointers to pitfalls appreciated.

  Read the article

• Floating point precision and physics calculations

  - by Vee

  The gravity Vector2 in my physics world is (0; 0.1). The number 0.1 is known to be problematic, since "it cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4". Having this value for the gravity gives me problems with collisions and creates quite nasty bugs. Changing the value to 0.11 solves these problems. Is there a more elegant solution that doesn't require changing the value at all?

  Read the article

• DateTime Comparison Precision

  - by mnh

  I'm doing DateTime comparison but I don't want to do comparison at second, millisecond and ticks level. What's the most elegant way? If I simply compare the DateTime, then they are seldom equal due to ticks differences.

  Read the article

• update columns when value is numeric in tsql

  - by knittl

  i want to normalize date fields from an old badly designed db dump. i now need to update every row, where the datefield only contains the year. update table set date = '01.01.' + date where date like '____' and isnumeric(date) = 1 and date >= 1950 but this will not work, because sql does not do short circuit evaluation of boolean expressions. thus i get an error "error converting nvarchar '01.07.1989' to int" is there a way to work around this? the column also contains strings with a length of 4, which are not numbers (????, 5/96, 70/8, etc.) the table only has 60000 rows

  Read the article

• Comparing numeric strings

  - by Kiren Siva

  From the question PHP Type-Juggling and (strict) Greater/Lesser Than Comparisons I know PHP interpret strings as numbers whenever it can. "10" < "1a" => 10 less than 1 expecting false "1a" < "2" => 1 less than 2 expecting true "10" > "2" => 10 greater than 2 expecting true But in the case of "10" < "1a" php returns true. I am not understanding the concept please help me to clarify it. Edit: But when I add "10" + "1a" it returns 11 that means php interprets "10" as 10 and "1a" as 1. Is that correct?

  Read the article

• Extend precision of MySQL's double datatype?

  - by tim82

  I am trying to save the value "6.714285714285714" into a DOUBLE datatype field. Unfortunately it does not fit at all and is cutted by one char. Storing larger numbers becomes less precise. Already searched in the mysql manual and it seems to be that double is the most precise data type available. Anyone knows a practicable workaround? Sorry for my bad english and thx a lot!

  Read the article

• Infinite precision type

  - by Andrew S.

  Is there such a type\implementation in C#? It is needed to calculate figures up to 10e10000 magnitude.

  Read the article

< Previous Page | 6 7 8 9 10 11 12 13 14 15 16 17  | Next Page >