Search Results

Search found 228 results on 10 pages for 'bankers rounding'.

Page 5/10 | < Previous Page | 1 2 3 4 5 6 7 8 9 10  | Next Page >

  • Putting BigDecimal data into HSQLDB test database using DbUnit

    - by Denise
    Hi everyone, I'm using Hibernate JPA in my backend. I am writing a unit test using JUnit and DBUnit to insert a set of data into an in-memory HSQL database. My dataset contains: <order_line order_line_id="1" quantity="2" discount_price="0.3"/> Which maps to an OrderLine Java object where the discount_price column is defined as: @Column(name = "discount_price", precision = 12, scale = 2) private BigDecimal discountPrice; However, when I run my test case and assert that the discount price returned equals 0.3, the assertion fails and says that the stored value is 0. If I change the discount_price in the dataset to be 0.9, it rounds up to 1. I've checked to make sure HSQLDB isn't doing the rounding and it definitely isn't because I can insert an order line object using Java code with a value like 5.3 and it works fine. To me, it seems like DBUtils is for some reason rounding the number I've defined. Is there a way I can force this to not happen? Can anyone explain why it might be doing this? Thanks!

    Read the article

  • How do calculators work with precision?

    - by zoul
    Hello! I wonder how calculators work with precision. For example the value of sin(M_PI) is not exactly zero when computed in double precision: #include <math.h> #include <stdio.h> int main() { double x = sin(M_PI); printf("%.20f\n", x); // 0.00000000000000012246 return 0; } Now I would certainly want to print zero when user enters sin(p). I can easily round somewhere on 1e–15 to make this particular case work, but that’s a hack, not a solution. When I start to round like this and the user enters something like 1e–20, they get a zero back (because of the rounding). The same thing happens when the user enters 1/10 and hits the = key repeatedly — when he reaches the rounding treshold, he gets zero. And yet some calculators return plain zero for sin(p) and at the same time they can work with expressions such as (1e–20)/10 comfortably. Where’s the trick?

    Read the article

  • How can a single threaded application like Excel 2003 take more than 50% of a hyper-threaded or dual

    - by Lunatik
    I'm waiting for Excel to finish a recalculation and I notice that the CPU usage as reported by Task Manager occasionally spikes to 51% or 52% on a Pentium 4 with hyper-threading. How is a single-threaded application like Excel 2003 doing this? Is it just a rounding/estimation error on the part of Task Manager? Or is it something to do with HT allocation i.e. I wouldn't see this happening on a genuine dual-core or dual-CPU machine?

    Read the article

  • Why don't computers store decimal numbers as a second whole number?

    - by SomeKittens
    Computers have trouble storing fractional numbers where the denominator is something other than a solution to 2^x. This is because the first digit after the decimal is worth 1/2, the second 1/4 (or 1/(2^1) and 1/(2^2)) etc. Why deal with all sorts of rounding errors when the computer could have just stored the decimal part of the number as another whole number (which is therefore accurate?) The only thing I can think of is dealing with repeating decimals (in base 10), but there could have been an edge solution to that (like we currently have with infinity).

    Read the article

  • OpenGL 2D Rasterization Sub-Pixel Translations

    - by Armin Ronacher
    I have a tile based 2D engine where the projection matrix is an orthographic view of the world without any scaling applied. Thus: one pixel texture is drawn on the screen in the same size. That all works well and looks nice but if the camera makes a sub-pixel movement small lines appear between the tiles. I can tell you in advance what does not fix the problem: GL_NEAREST texture interpolation GL_CLAMP_TO_EDGE What does “fix” the problem is anchoring the camera to the nearest pixel instead of doing a sub-pixel translation. I can live with that, but the camera movement becomes jerky. Any ideas how to fix that problem without resorting to the rounding trick I do currently?

    Read the article

  • How to document and teach others "optimized beyond recognition" computationally intensive code?

    - by rwong
    Occasionally there is the 1% of code that is computationally intensive enough that needs the heaviest kind of low-level optimization. Examples are video processing, image processing, and all kinds of signal processing, in general. The goals are to document, and to teach the optimization techniques, so that the code does not become unmaintainable and prone to removal by newer developers. (*) (*) Notwithstanding the possibility that the particular optimization is completely useless in some unforeseeable future CPUs, such that the code will be deleted anyway. Considering that software offerings (commercial or open-source) retain their competitive advantage by having the fastest code and making use of the newest CPU architecture, software writers often need to tweak their code to make it run faster while getting the same output for a certain task, whlist tolerating a small amount of rounding errors. Typically, a software writer can keep many versions of a function as a documentation of each optimization / algorithm rewrite that takes place. How does one make these versions available for others to study their optimization techniques?

    Read the article

  • New Development Snapshot

    More bug fixing and rounding out IKVM.Reflection. Changes: Fixed resolution of generic type instance names with "fake" type as type argument. Fixed bug in Finalize/finalize special casing for .NET types that caused exception when reflecting on them. Fixed Finalize/finalize special casing for .NET types to also work for java.lang.Throwable derived types. Removed fake core class library types from ikvmstub and added a -bootstrap...Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

    Read the article

  • Moving two objects proportionally

    - by SSL
    I'm trying to move two objects away from each other at a proportional distance, but on different scales. I'm not quite sure how to do this. Object A can go from position 0.1 to 1. Object B has no limits. If object B is decreasing, then Object A should be decreasing at rate R. Likewise, if Object B is increasing, then Object A increases at rate R. How can I tie these two Object positions together so that in an update loop, they automatically update their positions? I tried using: ObjA.Pos += 0.001f * ObjB.VelocityY; //0.001f is the rate This works but there's an error each time it runs. ObjA starts off at its max position 1 but then the next time it will stop at 0.97, 0.94, 0.91 etc.. This is due to the 0.001f rate I put in. Is there a way to control the rate, yet not end up with the rounding error?

    Read the article

  • sprintf() to truncate and not round a float to x decimal places?

    - by jerrygarciuh
    Hi folks, When calculating a golf handicap differential you are supposed to truncate the answer to 1 decimal place without rounding. No idea why but... I know how to do this using TRUNCATE() in mySQL SELECT TRUNCATE( 2.365, 1 ); // outputs 2.3 but I was wondering if sprintf() could do this? The only way I know to work with decimal places in a float is ... echo sprintf("%.1f", 2.365); // outputs 2.4 TIA JG

    Read the article

  • Code-Golf: Friendly Number Abbreviator

    - by David Murdoch
    Based on this question: Is there a way to round numbers into a friendly format? THE CHALLENGE - UPDATED! (removed hundreds abbreviation from spec) The shortest code by character count that will abbreviate an integer (no decimals). Code should include the full program. Relevant range is from 0 - 9,223,372,036,854,775,807 (the upper limit for signed 64 bit integer). The number of decimal places for abbreviation will be positive. You will not need to calculate the following: 920535 abbreviated -1 place (which would be something like 0.920535M). Numbers in the tens and hundreds place (0-999) should never be abbreviated (the abbreviation for the number 57 to 1+ decimal places is 5.7dk - it is unneccessary and not friendly). Remember to round half away from zero (23.5 gets rounded to 24). Banker's rounding is verboten. Here are the relevant number abbreviations: h = hundred (102) k = thousand (103) M = million (106) G = billion (109) T = trillion (1012) P = quadrillion (1015) E = quintillion (1018) SAMPLE INPUTS/OUTPUTS (inputs can be passed as separate arguments): First argument will be the integer to abbreviate. The second is the number of decimal places. 12 1 => 12 // tens and hundreds places are never rounded 1500 2 => 1.5k 1500 0 => 2k // look, ma! I round UP at .5 0 2 => 0 1234 0 => 1k 34567 2 => 34.57k 918395 1 => 918.4k 2134124 2 => 2.13M 47475782130 2 => 47.48G 9223372036854775807 3 => 9.223E // ect... . . . Original answer from related question (javascript, does not follow spec): function abbrNum(number, decPlaces) { // 2 decimal places => 100, 3 => 1000, etc decPlaces = Math.pow(10,decPlaces); // Enumerate number abbreviations var abbrev = [ "k", "m", "b", "t" ]; // Go through the array backwards, so we do the largest first for (var i=abbrev.length-1; i>=0; i--) { // Convert array index to "1000", "1000000", etc var size = Math.pow(10,(i+1)*3); // If the number is bigger or equal do the abbreviation if(size <= number) { // Here, we multiply by decPlaces, round, and then divide by decPlaces. // This gives us nice rounding to a particular decimal place. number = Math.round(number*decPlaces/size)/decPlaces; // Add the letter for the abbreviation number += abbrev[i]; // We are done... stop break; } } return number; }

    Read the article

  • Can Java's random function be zero?

    - by ThirdD3gree
    Just out of curiosity, can Math.random() ever be zero? For example, if I were to have: while (true){ if (Math.random() == 0) return 1; } Would I ever actually get a return of one? There's also rounding error to consider because Math.random() returns a double. I ask because my CS professor stated that random() goes from 0 to 1 inclusive, and I always thought it was exclusive.

    Read the article

  • Convert.ToInt or String.Split : which is more efficient?

    - by zsharp
    Which way is better for removing float decimals places or is there a more efficient way? Rounding is not important, i am interested in displaying the number Float number; 1- int wholeNumber= Convert.ToInt(number); string display=wholeNumber.ToString(); 2- string [] splitNumber= number.ToString().Split('.'); string display=splitNumber[0];

    Read the article

  • RMagick Rounded Corners

    - by mikeycgto
    Need a simple a way of rounding off an Image. I need the corners to be transparent. This link shows how to do it via command line: http://www.imagemagick.org/Usage/thumbnails/#rounded What I need is the corresponding RMagick\Ruby code... Thanks!

    Read the article

  • easy hex/float conversion

    - by yeus
    I am doing some input/output between a c++ and a python program (only floating point values) python has a nice feature of converting floating point values to hex-numbers and back as you can see in this link: http://docs.python.org/library/stdtypes.html#additional-methods-on-float Is there an easy way in C++ to to something similar? and convert the python output back to C++ double/float? This way I would not have the problem of rounding errors when exchanging data between the two processes... thx for the answers!

    Read the article

  • Real life example fo Floating Point error

    - by Rob
    Is there any examples of a company that was burned by floating point data that caused a rounding issue? We're implementing a new system and all the monetary values are stored in floats. I think if i can show actual examples of why this has failed it'll have more weight than the theory of why the values can't be stored properly.

    Read the article

  • MySQL: Digits (1.75) to Friendly Time (1 hour and 45 minutes)

    - by Nick
    In my MySQL database, I have a float field named "HoursSpent" with values like "0.25", "1.75", "2.5", etc. Is there a way that I can have my SELECT statement format those values in a friendly format like this?: 0.25 = 15 minutes 1.75 = 1 hour and 45 minutes 2.5 = 2 hours and 30 minutes The "HoursSpent" field is supposed to only have values in 0.25 increments, but if somebody were to put something random like 0.16, it would be nice if the SELECT statement handled that by rounding it up to the nearest 0.25 (so in this case 0.16 would become 0.25, or 15 minutes).

    Read the article

  • SQL SERVER – Weekly Series – Memory Lane – #031

    - by Pinal Dave
    Here is the list of selected articles of SQLAuthority.com across all these years. Instead of just listing all the articles I have selected a few of my most favorite articles and have listed them here with additional notes below it. Let me know which one of the following is your favorite article from memory lane. 2007 Find Table without Clustered Index – Find Table with no Primary Key Clustered index is very important concept for any table. They impact the performance very heavily. Here is a quick script to find tables without a clustered index. Replace TEXT with VARCHAR(MAX) – Stop using TEXT, NTEXT, IMAGE Data Types Question: “Is VARCHAR (MAX) big enough to store the TEXT field?” Answer: “Yes, VARCHAR(MAX) is big enough to accommodate TEXT field. TEXT, NTEXT and IMAGE data types of SQL Server 2000 will be deprecated in a future version of SQL Server, SQL Server 2005 provides backward compatibility to data types but it is recommended to use new data types which are VARHCAR (MAX), NVARCHAR (MAX) and VARBINARY (MAX).” Limiting Result Sets by Using TABLESAMPLE – Examples Introduced in SQL Server 2005, TABLESAMPLE allows you to extract a sampling of rows from a table in the FROM clause. The rows retrieved are random and they are are not in any order. This sampling can be based on a percentage of number of rows. You can use TABLESAMPLE when only a sampling of rows is necessary for the application instead of a full result set. User Defined Functions (UDF) Limitations UDF have its own advantage and usage but in this article we will see the limitation of UDF. Things UDF can not do and why Stored Procedure are considered as more flexible then UDFs. Stored Procedure are more flexibility then User Defined Functions(UDF). However, this blog post is a good read to know what are the limitations of UDF. Change Database Compatible Level – Backward Compatibility For a long time SQL Server stayed on the compatibility level of 80 which is of SQL Server 2000. However, as soon as SQL Server 2005 introduced the issue of compatibility was quite a major issue. Since that time MS has been releasing the versions at every 2-3 years, changing compatibility is a ever popular topic. In this blog post, we learn how we can do the same using T-SQL. We can also do the same using SSMS and here is the blog post for the same: Change Database Compatible Level – Backward Compatibility – Part 2 – Management Studio. Constraint on VARCHAR(MAX) Field To Limit It Certain Length How can I limit the VARCHAR(MAX) field with maximum length of 12500 characters only. His Question was valid as our application was allowed 12500 characters. First of all – this requirement is bit strange but if someone wants to do the same, they can do it as described in this blog post. 2008 UNPIVOT Table Example Understanding UNPIVOT can be very complicated at times. In this blog post, I have attempted to explain the same concept in very simple words. Create Default Constraint Over Table Column A simple straight to script blog post – I still use this blog quite many times for my own reference. UDF – Get the Day of the Week Function It took me 4 iteration to find this very simple function which can immediately get the day of the week in a single line. 2009 Find Hostname and Current Logged In User Name There are two tricks listed in this blog post where users can find out the hostname and current logged user name immediately and very easily. Interesting Observation of Logon Trigger On All Servers When I was doing a project, I made an interesting observation of executing a logon trigger multiple times. It was absolutely unexpected for me! As I was logging only once, naturally, I was expecting the entry only once. However, it did it multiple times on different threads – indeed an eccentric phenomenon at first sight! Difference Between Candidate Keys and Primary Key One needs to be very careful in selecting the Primary Key as an incorrect selection can adversely impact the database architect and future normalization. For a Candidate Key to qualify as a Primary Key, it should be Non-NULL and unique in any domain. I have observed quite often that Primary Keys are seldom changed. I would like to have your feedback on not changing a Primary Key. Create Multiple Filegroup For Single Database Why should one create multiple file group for any database and what are the advantages of the same. In this blog post, I explain the same in detail. List All Objects Created on All Filegroups in Database In this blog post we discuss the essential question – “How can I find which object belongs to which filegroup. Is there any way to know this?” 2010 DATE and TIME in SQL Server 2008 When DATE is converted to DATETIME it adds the of midnight. When TIME is converted to DATETIME it adds the date of 1900 and it is something one wants to consider if you are going to run scripts from SQL Server 2008 to earlier version with CONVERT. Disabled Index and Update Statistics If you do not need a nonclustered index, I suggest you to drop it as keeping them disabled is an overhead on your system. This is because every time the statistics are updated for system all the statistics for disabled indexes are also updated. Precision of SMALLDATETIME – A 1 Minute Precision The precision of the datatype SMALLDATETIME is 1 minute. It discards the seconds by rounding up or rounding down any seconds greater than zero. 2011 Getting Columns Headers without Result Data – SET FMTONLY ON SET FMTONLY ON returns only metadata to the client. It can be used to test the format of the response without actually running the query. When this setting is ON the resultset only have headers of the results but no data. Copy Database from Instance to Another Instance – Copy Paste in SQL Server SQL Server has a feature which copy database from one database to another database and it can be automated as well using SSIS. Make sure you have SQL Server Agent Turned on as this feature will create a job. Puzzle – SELECT * vs SELECT COUNT(*) If you have ever wondered SELECT * gives error when executed alone but SELECT COUNT(*) does not. Why? in that case, you should read this blog post. Creating All New Database with Full Recovery Model This blog post is very based on very interesting story where the user wants to do something by default for every single new database created. Model database is a secret weapon which should be used very carefully and with proper evalution. If used carefully this can be a very much beneficiary when we need a newly created database behave in certain fashion. 2012 In year 2012 I had two interesting series ran on the blog. If there is no fun in learning, the learning becomes a burden. For the same reason, I had decided to build a three part quiz around SEQUENCE. The quiz was to identify the next value of the sequence. I encourage all of you to take part in this fun quiz. Guess the Next Value – Puzzle 1 Guess the Next Value – Puzzle 2 Guess the Next Value – Puzzle 3 Can anyone remember their final day of schooling?  This is probably a silly question because – of course you can!  Many people mark this as the most exciting, happiest day of their life.  It marks the end of testing, the end of following rules set by teachers, and the beginning of finally being able to earn money and work in your chosen field. Read five part series on developer training subject Developer Training - Importance and Significance - Part 1 Developer Training – Employee Morals and Ethics – Part 2 Developer Training – Difficult Questions and Alternative Perspective - Part 3 Developer Training – Various Options for Developer Training – Part 4 Developer Training – A Conclusive Summary- Part 5 Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Memory Lane, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology

    Read the article

  • Automatically select last row in a set in Excel

    - by Luke
    In Excel 2003, I am trying to keep track of some petty cash, and have it set up with the denomination along the top row, along with a sub total and difference column. I want a small section that shows how many rolls of coins I should have, by taking the total amount, and dividing it by however many should be in a roll, and rounding to the lowest whole number. That part is fine. What I want done is for that ONE section (how many rolls) I should have, based on the last row that has information in it. For example, if the last row is row 13, it should read the data from B13, C13, D13, etc. I don't mind learning Macros, if that's what the solution requires. I don't want to be manually selecting the last row each time though, I just want the worksheet to know automatically.

    Read the article

  • SQL SERVER – Difference Between DATETIME and DATETIME2 – WITH GETDATE

    - by pinaldave
    Earlier I wrote blog post SQL SERVER – Difference Between GETDATE and SYSDATETIME which inspired me to write SQL SERVER – Difference Between DATETIME and DATETIME2. Now earlier two blog post inspired me to write this blog post (and 4 emails and 3 reads from readers). I previously populated DATETIME and DATETIME2 field with SYSDATETIME, which gave me very different behavior as SYSDATETIME was rounded up/down for the DATETIME datatype. I just ran the same experiment but instead of populating SYSDATETIME in this script I will be using GETDATE function. DECLARE @Intveral INT SET @Intveral = 10000 CREATE TABLE #TimeTable (FirstDate DATETIME, LastDate DATETIME2) WHILE (@Intveral > 0) BEGIN INSERT #TimeTable (FirstDate, LastDate) VALUES (GETDATE(), GETDATE()) SET @Intveral = @Intveral - 1 END GO SELECT COUNT(DISTINCT FirstDate) D_FirstDate, COUNT(DISTINCT LastDate) D_LastDate FROM #TimeTable GO SELECT DISTINCT a.FirstDate, b.LastDate FROM #TimeTable a INNER JOIN #TimeTable b ON a.FirstDate = b.LastDate GO SELECT * FROM #TimeTable GO DROP TABLE #TimeTable GO Let us run above script and observe the results. You will find that the values of GETDATE which is populated in both the columns FirstDate and LastDate are very much same. This is because GETDATE is of datatype DATETIME and the precision of the GETDATE is smaller than DATETIME2 there is no rounding happening. In other word, this experiment is pointless. I have included this as I got 4 emails and 3 twitter questions on this subject. If your datatype of variable is smaller than column datatype there is no manipulation of data, if data type of variable is larger than column datatype the data is rounded. Reference: Pinal Dave (http://www.SQLAuthority.com) Filed under: Pinal Dave, SQL, SQL Authority, SQL DateTime, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, T SQL, Technology

    Read the article

  • RAID5 over LVM on Ubuntu Server 12.04.3

    - by April Ethereal
    I'm trying to create a RAID5 software array using LVM. I use VirtualBox as I'm only learning how LVM works. So I've created 4 virtual SCSI drives and then did the following: pvcreate /dev/sd[b-e] vgcreate /dev/sd[b-e] raid5_vg lvcreate --type raid5 -i 3 -L 1G -n raid_lv raid5_vg However, I get an error after the last command: WARNING: Unrecognised segment type raid5 Using default stripesize 64.00 KiB Rounding size (256 extents) up to stripe boundary size (258 extents) Cannot update volume group raid5_vg with unknown segments in it! So it looks like raid5 is not a valid segment type. "lvm segtypes" also doesn't contain 'raid5' entry: root@ubuntu-lvm:~# lvm segtypes striped zero error free snapshot mirror So my question is - how could I create RAID5 logical volume using LVM only? It seems that it is possible, I saw a few references (not for Ubuntu, unfortunately) for RedHat and Gentoo systems. I don't want to use mdadm for now, until I find out that it is mandatory. Some info about my system is below: root@ubuntu-lvm:~# uname -a Linux ubuntu-lvm 3.8.0I use Ubuntu Server 12.04.3 (i686)-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux root@ubuntu-lvm:~# dpkg -l | grep lvm ii lvm2 2.02.66-4ubuntu7.3 The Linux Logical Volume Manager Thanks.

    Read the article

  • Silverlight Cream for April 23, 2010 -- #845

    - by Dave Campbell
    In this Issue: Jason Allor, Bill Reiss, Mike Snow, Tim Heuer, John Papa, Jeremy Likness, and Dave Campbell. Shoutouts: You saw it at MIX10 and DevConnections... now you can give it a dance, John Papa announced eBay Simple Lister Beta Now Available Mike Snow posted some info about and a link to his new Flickr/Bing/Google High End Image Viewer and he's looking for feedback From SilverlightCream.com: Hierarchical Data Trees With A Custom DataSource Jason Allor is rounding out a series here in his new blog (bookmark it), and he's created his own custom HierarchicalDataSource class for use with the TreeView. Space Rocks game step 11: Start level logic Bill Reiss has Episode 11 up in his Space Rocks game ... working on NewGame and start level logic Silverlight Tip of the Day #3 – Mouse Right Clicks Mike Snow has Tip 3 up ... about handling right-mouse clicks in Silverlight 4 -- oh yeah, we got right mouse now ... grab Mike's project to check it out. Silverlight 4 enables Authorization header modification Tim Heuer talks about the ability to modify the Authorization header in network calls with Silverlight 4. He gives not only the quick-and-dirty of how to use it, but has some good examples, code, and code results for show and tell. WCF RIA Services - Hands On Lab John Papa built a bookstore app in roughly 10 minutes in the keynote at DevConnections. He now has a tutorial on doing just that plus all the code up. Transactions with MVVM Not strictly Silverlight (or WPF), but Jeremy Likness has an interesting article up on MVVM and transaction processing. Read the post then grab his helper class. Your First Windows Phone 7 Application As with the First Silverlight App a couple weeks ago, if you've got any WP7 experience at all, just keep going... this is for folks that have not looked at it yet, have not downloaded anything... oh, and it's by Dave Campbell Stay in the 'Light! Twitter SilverlightNews | Twitter WynApse | WynApse.com | Tagged Posts | SilverlightCream Join me @ SilverlightCream | Phoenix Silverlight User Group Technorati Tags: Silverlight    Silverlight 3    Silverlight 4    Windows Phone MIX10

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10  | Next Page >