Search Results

Search found 4580 results on 184 pages for 'faster'.

Page 160/184 | < Previous Page | 156 157 158 159 160 161 162 163 164 165 166 167  | Next Page >

• Hibernate Relationship Mapping/Speed up batch inserts

  - by manyxcxi

  I have 5 MySQL InnoDB tables: Test,InputInvoice,InputLine,OutputInvoice,OutputLine and each is mapped and functioning in Hibernate. I have played with using StatelessSession/Session, and JDBC batch size. I have removed any generator classes to let MySQL handle the id generation- but it is still performing quite slow. Each of those tables is represented in a java class, and mapped in hibernate accordingly. Currently when it comes time to write the data out, I loop through the objects and do a session.save(Object) or session.insert(Object) if I'm using StatelessSession. I also do a flush and clear (when using Session) when my line count reaches the max jdbc batch size (50). Would it be faster if I had these in a 'parent' class that held the objects and did a session.save(master) instead of each one? If I had them in a master/container class, how would I map that in hibernate to reflect the relationship? The container class wouldn't actually be a table of it's own, but a relationship all based on two indexes run_id (int) and line (int). Another direction would be: How do I get Hibernate to do a multi-row insert?

  Read the article

• Speeding up a group by date query on a big table in postgres

  - by zaius

  I've got a table with around 20 million rows. For arguments sake, lets say there are two columns in the table - an id and a timestamp. I'm trying to get a count of the number of items per day. Here's what I have at the moment. SELECT DATE(timestamp) AS day, COUNT(*) FROM actions WHERE DATE(timestamp) >= '20100101' AND DATE(timestamp) < '20110101' GROUP BY day; Without any indices, this takes about a 30s to run on my machine. Here's the explain analyze output: GroupAggregate (cost=675462.78..676813.42 rows=46532 width=8) (actual time=24467.404..32417.643 rows=346 loops=1) -> Sort (cost=675462.78..675680.34 rows=87021 width=8) (actual time=24466.730..29071.438 rows=17321121 loops=1) Sort Key: (date("timestamp")) Sort Method: external merge Disk: 372496kB -> Seq Scan on actions (cost=0.00..667133.11 rows=87021 width=8) (actual time=1.981..12368.186 rows=17321121 loops=1) Filter: ((date("timestamp") >= '2010-01-01'::date) AND (date("timestamp") < '2011-01-01'::date)) Total runtime: 32447.762 ms Since I'm seeing a sequential scan, I tried to index on the date aggregate CREATE INDEX ON actions (DATE(timestamp)); Which cuts the speed by about 50%. HashAggregate (cost=796710.64..796716.19 rows=370 width=8) (actual time=17038.503..17038.590 rows=346 loops=1) -> Seq Scan on actions (cost=0.00..710202.27 rows=17301674 width=8) (actual time=1.745..12080.877 rows=17321121 loops=1) Filter: ((date("timestamp") >= '2010-01-01'::date) AND (date("timestamp") < '2011-01-01'::date)) Total runtime: 17038.663 ms I'm new to this whole query-optimization business, and I have no idea what to do next. Any clues how I could get this query running faster?

  Read the article

• How do I select every 6th element from a list (using Linq)

  - by iDog

  Hi, I've got a list of 'double' values. I need to select every 6th record. It's a list of coordinates, where I need to get the minimum and maximum value of every 6th value. List of coordinates (sample): [2.1, 4.3, 1.0, 7.1, 10.6, 39.23, 0.5, ... ] with hundrets of coordinates. Result should look like: [x_min, y_min, z_min, x_max, y_max, z_max] with exactly 6 coordinates. Following code works, but it takes to long to iterate over all coordinates. I'd like to use Linq instead (maybe faster?) for (int i = 0; i < 6; i++) { List<double> coordinateRange = new List<double>(); for (int j = i; j < allCoordinates.Count(); j = j + 6) coordinateRange.Add(allCoordinates[j]); if (i < 3) boundingBox.Add(coordinateRange.Min()); else boundingBox.Add(coordinateRange.Max()); } Any suggestions? Many thanks! Greets!

  Read the article

• Update table with index is too slow

  - by pauloya

  Hi, I was watching the Profiler on a live system of our application and I saw that there was an update instruction that we run periodically (every second) that was quite slow. It took around 400ms every time. The query includes this update (which is the slow part) UPDATE BufferTable SET LrbCount = LrbCount + 1, LrbUpdated = getdate() WHERE LrbId = @LrbId This is the table CREATE TABLE BufferTable( LrbId [bigint] IDENTITY(1,1) NOT NULL, ... LrbInserted [datetime] NOT NULL, LrbProcessed [bit] NOT NULL, LrbUpdated [datetime] NOT NULL, LrbCount [tinyint] NOT NULL, ) The table has 2 indexes (non unique and non clustered) with the fields by this order: * Index1 - (LrbProcessed, LrbCount) * Index2 - (LrbInserted, LrbCount, LrbProcessed) When I looked at this I thought that the problem would come from Index1 since LrbCount is changing a lot and it changes the order of the data in the index. But after desactivating index1 I saw the query was taking the same time as initially. Then I rebuilt index1 and desactivated index2, this time the query was very fast. It seems to me that Index2 should be faster to update, the order of the data shouldn't change since the LrbInserted time is not changed. Can someone explain why index2 is much heavier to update then index1? Thank you!

  Read the article

• Threaded Python port scanner

  - by Amnite

  I am having issues with a port scanner I'm editing to use threads. This is the basics for the original code: for i in range(0, 2000): s = socket(AF_INET, SOCK_STREAM) result = s.connect_ex((TargetIP, i)) if(result == 0) : c = "Port %d: OPEN\n" % (i,) s.close() This takes approx 33 minutes to complete. So I thought I'd thread it to make it run a little faster. This is my first threading project so it's nothing too extreme, but I've ran the following code for about an hour and get no exceptions yet no output. Am I just doing the threading wrong or what? import threading from socket import * import time a = 0 b = 0 c = "" d = "" def ScanLow(): global a global c for i in range(0, 1000): s = socket(AF_INET, SOCK_STREAM) result = s.connect_ex((TargetIP, i)) if(result == 0) : c = "Port %d: OPEN\n" % (i,) s.close() a += 1 def ScanHigh(): global b global d for i in range(1001, 2000): s = socket(AF_INET, SOCK_STREAM) result = s.connect_ex((TargetIP, i)) if(result == 0) : d = "Port %d: OPEN\n" % (i,) s.close() b += 1 Target = raw_input("Enter Host To Scan:") TargetIP = gethostbyname(Target) print "Start Scan On Host ", TargetIP Start = time.time() threading.Thread(target = ScanLow).start() threading.Thread(target = ScanHigh).start() e = a + b while e < 2000: f = raw_input() End = time.time() - Start print c print d print End g = raw_input()

  Read the article

• What is the last entry in an unfilled array (C++)?

  - by jwaffe

  I put C++ because I'm just starting in C# and I'm not sure if there's a difference. if you declare an array char arr[10] and fill in values for arr[0] through arr[8], what value will be put in arr[9]? a space ' '? An endline '\n'? '\0'? Or is it nothing at all? I'm asking this because I've always used tactics like this char word[20]; for(count = 0 ; count < 20 ; count++) { cout << word[count]; } to print the entire contents of an array, and I was wondering if I could simplify it (e.g., if the last entry was '\0') by using something like this char word[20]; while(word[count] != '\0') { cout << word[count]; } that way, I wouldn't have to remember how many pieces of data were entered into an array if all the spaces weren't filled up. If you know an even faster way, let me know. I tend to make a bunch of mistakes on arrays.

  Read the article

• Pi help with Php (mass looping)

  - by Pieman

  My primary question is: Is this alot of loops? while ($decimals < 50000 and $remainder != "0") { $number = floor($remainder/$currentdivider); //Always round down! 10/3 =3, 10/7 = 1 $remainder = $remainder%$currentdivider; // 10%3 =1, 10%1 $thisnumber = $thisnumber . $number; $remainder = $remainder . 0; //10 $decimals += 1; } Or could I fit more into it? -without the server crashing/lagging. I'm just wondering, Also is there a more effiecent way of doing the above? (e.g. finidng out that 1/3 = 0.3 to 50,000 decimals.) Finally: I'm doing this for a pi formulae the (1 - 1/3 + 1/5 - 1/7 etc.) one, And i'm wondering if there is a better one. (In php) I have found one that finds pi to 2000 in 4 seconds. But thats not what I want. I want an infinite series that converges closer to Pi so every refresh, users can view it getting closer... see: http://zombiewrath.com/pi.php (Old one) and 'zombiewrath.com/superpi.php' (Newer one) But obv. converging using the above formulae takes ALONG time. Is there any other 'loop' like Pi formulaes (workable in php) that converge faster? Thanks alot...

  Read the article

• How does a java compiler resolve a non-imported name

  - by gexicide

  Consider I use a type X in my java compilation unit from package foo.bar and X is not defined in the compilation unit itself nor is it directly imported. How does a java compiler resolve X now efficiently? There are a few possibilities where X could reside: X might be imported via a star import a.b.* X might reside in the same package as the compilation unit X might be a language type, i.e. reside in java.lang The problem I see is especially (2.). Since X might be a package-private type, it is not even required that X resides in a compilation unit that is named X.java. Thus, the compiler must look into all entries of the class path and search for any classes in a package foo.bar, it then must read every class that is in package foo.bar to check whether X is included. That sounds very expensive. Especially when I compile only a single file, the compiler has to read dozens of class files only to find a type X. If I use a lot of star imports, this procedure has to be repeated for a lot of types (although class files won't be read twice, of course). So is it advisable to import also types from the same package to speed up the compilation process? Or is there a faster method for resolving an unimported type X which I was not able to find?

  Read the article

• JAVA : How to get the positions of all matches in a String?

  - by user692704

  I have a text document and a query (the query could be more than one word). I want to find the position of all occurrences of the query in the document. I thought of the documentText.indexOf(query) and using regular expression but I could not make it work. I end up with the following method: First, I have create a dataType called QueryOccurrence public class QueryOccurrence implements Serializable{ public QueryOccurrence(){} private int start; private int end; public QueryOccurrence(int nameStart,int nameEnd,String nameText){ start=nameStart; end=nameEnd; } public int getStart(){ return start; } public int getEnd(){ return end; } public void SetStart(int i){ start=i; } public void SetEnd(int i){ end=i; } } Then, I have used this datatype in the following method: public static List<QueryOccurrence>FindQueryPositions(String documentText, String query){ // Normalize do the following: lower case, trim, and remove punctuation String normalizedQuery = Normalize.Normalize(query); String normalizedDocument = Normalize.Normalize(documentText); String[] documentWords = normalizedDocument.split(" ");; String[] queryArray = normalizedQuery.split(" "); List<QueryOccurrence> foundQueries = new ArrayList(); QueryOccurrence foundQuery = new QueryOccurrence(); int index = 0; for (String word : documentWords) { if (word.equals(queryArray[0])){ foundQuery.SetStart(index); } if (word.equals(queryArray[queryArray.length-1])){ foundQuery.SetEnd(index); if((foundQuery.End()-foundQuery.Start())+1==queryArray.length){ //add the found query to the list foundQueries.add(foundQuery); //flush the foundQuery variable to use it again foundQuery= new QueryOccurrence(); } } index++; } return foundQueries; } This method return a list of all occurrence of the query in the document each one with its position. Could you suggest any easer and faster way to accomplish this task. Thanks

  Read the article

• Why doesn't my processor have built-in BigInt support?

  - by ol

  As far as I understood it, BigInts are usually implemented in most programming languages as strings containing numbers, where, eg.: when adding two of them, each digit is added one after another like we know it from school, e.g.: 246 816 * * ---- 1062 Where * marks that there was an overflow. I learned it this way at school and all BigInt adding functions I've implemented work similar to the example above. So we all know that our processors can only natively manage ints from 0 to 2^32 / 2^64. That means that most scripting languages in order to be high-level and offer arithmetics with big integers, have to implement/use BigInt libraries that work with integers as strings like above. But of course this means that they'll be far slower than the processor. So what I've asked myself is: Why doesn't my processor have a built-in BigInt function? It would work like any other BigInt library, only (a lot) faster and at a lower level: Processor fetches one digit from the cache/RAM, adds it, and writes the result back again. Seems like a fine idea to me, so why isn't there something like that?

  Read the article

• Storing large json strings to database + hash

  - by Guy

  I need to store quiete large JSON data strings to the database. I am using gzip to compress the string and therefore BLOB MySQL data type to store it. However, only 5% of all the requests contain unique data and only unique data ought to be stored to the database. My approach is as follows. array_multisort data (array [a, b, c] is virtually the same as [a, c, b]). json_encode data (json_encode is faster than serialize; we need string array representation for the step 3). sha1 data (slower than md5, though less possible the collisions). Check if the hash exists in the database. 5.1 yes – do not insert the data. 5.2. no – gzip the data and store it along the hash. Is there anything about this (apart from storing JSON data to the database in the first place) that sounds fishy or should be done a different way? p.s. We are talking about a database with roughly 1kk unique records being created every month.

  Read the article

• Which way of declaring a variable is fastest?

  - by ADB

  For a variable used in a function that is called very often and for implementation in J2ME on a blackberry (if that changed something, can you explain)? class X { int i; public void someFunc(int j) { i = 0; while( i < j ){ [...] i++; } } } or class X { static int i; public void someFunc(int j) { i = 0; while( i < j ){ [...] i++; } } } or class X { public void someFunc(int j) { int i = 0; while( i < j ){ [...] i++; } } } I know there is a difference how a static versus non-static class variable is accessed, but I don't know it would affect the speed. I also remember reading somewhere that in-function variables may be accessed faster, but I don't know why and where I read that. Background on the question: some painting function in games are called excessively often and even small difference in access time can affect the overall performance when a variable is used in a largish loop.

  Read the article

• Why is the operation address incremented by two?

  - by Gavin Jones

  I am looking at a Javascript emulator of a NES to try and understand how it works. On this line: addr = this.load(opaddr+2); The opcode is incremented by two. However, the documentation (see appendix E) I'm reading says: Zero page addressing uses a single operand which serves as a pointer to an address in zero page ($0000-$00FF) where the data to be operated on can be found. By using zero page addressing, only one byte is needed for the operand, so the instruction is shorter and, therefore, faster to execute than with addressing modes which take two operands. An example of a zero page instruction is AND $12. So if the operand's argument is only one byte, shouldn't it appear directly after it, and be + 1 instead of + 2? Why +2? This is how I think it works, which may be incorrect. Suppose our memory looks like: ------------------------- | 0 | 1 | 2 | 3 | 4 | 5 | <- index ------------------------- | a | b | c | d | e | f | <- memory ------------------------- ^ \ PC and our PC is 0, pointing to a. For this cycle, we say that the opcode: var pc= 0; //for example's sake var opcode= memory[pc]; //a So shouldn't the first operand be the next slot, i.e. b? var first_operand = memory[pc + 1]; //b

  Read the article

• Page specific CSS or a single css file when developing a mobile (webkit) based site?

  - by Mike

  I am working on a mobile site for webkit browsers. I have been trying to find information on using multiple style sheets versus a single css file. There is a lot of information on this topic, but it not a lot of information pertaining to mobile browsers. My site will have a bunch of pages that while have page specific css. For a non-mobile site, it seems like generally people say that a single file will be faster, but that multiple files are easier to develop. However, on a mobile site is that still the case? If you put everything in one file, that will get cached after load, but that will make the first load slower. If you had page specific files, the first page would get loaded quicker, but every other page would then take a hit while making the page specific css http request. Does anyone have any thoughts on this? It sounds like they are saying one file is better as long as its under 1 MB (which my files def will)? http://www.yuiblog.com/blog/2010/07/12/mobile-browser-cache-limits-revisited/

  Read the article

• iphone xcode sqlite3_open remote host

  - by cicana

  Hi, i try to get a connection my server, with the sqlite3_open command! my question...is it possible to that? i got the following code... // Get the path to the documents directory and append the databaseName databaseName = @"AnimalDatabase.sql"; NSString *serverpath = @"http://localhost/app/"; databasePath = [serverpath stringByAppendingPathComponent:databaseName]; and then this here -(void) readAnimalsFromDatabase { // Setup the database object sqlite3 *database; // Init the animals Array animals = [[NSMutableArray alloc] init]; // Open the database from the users filessytem if(sqlite3_open([databasePath UTF8String], &database) == SQLITE_OK) { // Setup the SQL Statement and compile it for faster access const char *sqlStatement = "select * from animals"; sqlite3_stmt *compiledStatement; if(sqlite3_prepare_v2(database, sqlStatement, -1, &compiledStatement, NULL) == SQLITE_OK) { // Loop through the results and add them to the feeds array while(sqlite3_step(compiledStatement) == SQLITE_ROW) { // Read the data from the result row NSString *aName = [NSString stringWithUTF8String:(char *)sqlite3_column_text(compiledStatement, 1)]; NSString *aDescription = [NSString stringWithUTF8String:(char *)sqlite3_column_text(compiledStatement, 2)]; NSString *aImageUrl = [NSString stringWithUTF8String:(char *)sqlite3_column_text(compiledStatement, 3)]; // Create a new animal object with the data from the database Animal *animal = [[Animal alloc] initWithName:aName description:aDescription url:aImageUrl]; // Add the animal object to the animals Array [animals addObject:animal]; [animal release]; } } // Release the compiled statement from memory sqlite3_finalize(compiledStatement); } sqlite3_close(database); } any suggestion??

  Read the article

• What's best performance way to constantly change image on WP7?

  - by AlRodriguez

  I'm trying to make my own type of remote desktop for WP7. I have a WCF service that returns an image on what's on the target machine's screen. Here's the WCF Server Code: // Method to load desktop image Bitmap image = new Bitmap( ViewSize.Width, ViewSize.Height ); Graphics g = Graphics.FromImage( image ); g.CopyFromScreen( Position.X, Position.Y, 0, 0, ViewSize ); g.Dispose( ); return image; // Convert image to byte[] which is returned to client using ( MemoryStream ms = new MemoryStream( ) ) { Bitmap image = screenGrabber.LoadScreenImage( ); image.Save( ms, ImageFormat.Jpeg ); imageArray = ms.ToArray( ); } Here's the code for the WP7 client: MemoryStream stream = new MemoryStream( data ); BitmapImage image = new BitmapImage( ); image.SetSource( stream ); BackgroundImage.Source = image; The BackgroundImage variable is an Image control. I'm noticing this freeze on the emulator after a short while, and will eventually crash from an OutOfMemoryException. This is already pretty slow ( images show up a good half second later than what's on the screen ), and I'm wondering if there's a better/faster way of doing this? Any help would be great. Thanks in advance.

  Read the article

• delay loop output in C++

  - by itachisxeyes

  I have a while loop that runs in a do while loop. I need the while loop to run exactly every second no faster no slower. but i'm not sure how i would do that. this is the loop, off in its own function. I have heard of the sleep() function but I also have heard that it is not very accurate. int min5() { int second = 00; int minute = 0; const int ZERO = 00; do{ while (second <= 59){ if(minute == 5) break; second += 1; if(second == 60) minute += 1; if(second == 60) second = ZERO; if(second < 60) cout << "Current Time> "<< minute <<" : "<< second <<" \n"; } } while (minute <= 5); }

  Read the article

• Xenserver 5.6 SR_BACKEND_FAILURE_47 no such volume group, but it is there

  - by Juan Carlos

  I've looked everywhere (Google, here, a bunch of other sites), and while I have found people with similar problems, I couldn't find a single one with a solution to this. Last night our xenserver 5.6 box corrupted the /var/xapi/state.db, and I couldn't fix the xml, no matter what I did. After a good hour fiddling with the file, I figured it would be faster to just reinstall. The server had one 2tb hard drive running Xen and its VMs, and since Xen's install said it would erase the hard drive it was installed on, I plugged a new harddrive and installed Xen on it, without selecting any hard drives for storage. I Figured I could make it happen after install, using the partition on the old harddrive with all my VMs on it. After instalation finished and the system booted I did: #fdisk -l found the old partition at /dev/sda3 #ll /dev/disk/by-id found the partition at /dev/disk/by-id/scsi-3600188b04c02f100181ab3a48417e490-part3 #xe host-list uuid ( RO) : a019d93e-4d84-4a4b-91e3-23572b5bd8a4 name-label ( RW): xenserver-scribfourteen name-description ( RW): Default install of XenServer #pvscan PV /dev/sda3 VG VG_XenStorage-405a2ece-d10e-d6c5-ede2-e1ad2c29c68d lvm2 [1.81 TB / 204.85 GB free] Total: 1 [1.81 TB] / in use: 1 [1.81 TB] / in no VG: 0 [0 ] #vgscan Reading all physical volumes. This may take a while... Found volume group "VG_XenStorage-405a2ece-d10e-d6c5-ede2-e1ad2c29c68d" using metadata type lvm2 # pvdisplay --- Physical volume --- PV Name /dev/sda3 VG Name VG_XenStorage-405a2ece-d10e-d6c5-ede2-e1ad2c29c68d PV Size 1.81 TB / not usable 6.97 MB Allocatable yes PE Size (KByte) 4096 Total PE 474747 Free PE 52441 Allocated PE 422306 PV UUID U03Gt9-WtHi-8Nnu-QB2Q-c7BV-CO9A-cFpYWW # xe sr-introduce name-label="VMs" type=lvm uuid=U03Gt9-WtHi-8Nnu-QB2Q-c7BV-CO9A-cFpYWW name-description="VMs Local HD Storage" content-type=user shared=false device-config=:device=/dev/disk/by-id/scsi-3600188b04c02f100181ab3a483f9f0ae-part3 U03Gt9-WtHi-8Nnu-QB2Q-c7BV-CO9A-cFpYWW # xe pbd-create host-uuid=a019d93e-4d84-4a4b-91e3-23572b5bd8a4 sr-uuid=U03Gt9-WtHi-8Nnu-QB2Q-c7BV-CO9A-cFpYWW device-config:device=/dev/disk/by-id/scsi-3600188b04c02f100181ab3a483f9f0ae-part3 adf92b7f-ad40-828f-0728-caf94d2a0ba1 # xe pbd-plug uuid=adf92b7f-ad40-828f-0728-caf94d2a0ba1 Error code: SR_BACKEND_FAILURE_47 Error parameters: , The SR is not available [opterr=no such volume group: VG_XenStorage-U03Gt9-WtHi-8Nnu-QB2Q-c7BV-CO9A-cFpYWW] At this point I did a # vgrename VG_XenStorage-405a2ece-d10e-d6c5-ede2-e1ad2c29c68d VG_XenStorage-U03Gt9-WtHi-8Nnu-QB2Q-c7BV-CO9A-cFpYWW cause the VG name was different, but pdb-plug still gives me the same error. So, now I'm kinda lost about what to do, I'm not used to Xen and most sites I've been finding are really unhelpful. I hope someone can guide me in the right way to fix this. I cant lose those VMs (got backups, but from inside the guests, not the VMs themselves).

  Read the article

• Disk performance below expectations

  - by paulH

  this is a follow-up to a previous question that I asked (Two servers with inconsistent disk speed). I have a PowerEdge R510 server with a PERC H700 integrated RAID controller (call this Server B) that was built using eight disks with 3Gb/s bandwidth that I was comparing with an almost identical server (call this Server A) that was built using four disks with 6Gb/s bandwidth. Server A had much better I/O rates than Server B. Once I discovered the difference with the disks, I had Server A rebuilt with faster 6Gbps disks. Unfortunately this resulted in no increase in the performance of the disks. Expecting that there must be some other configuration difference between the servers, we took the 6Gbps disks out of Server A and put them in Server B. This also resulted in no increase in the performance of the disks. We now have two identical servers built, with the exception that one is built with six 6Gbps disks and the other with eight 3Gbps disks, and the I/O rates of the disks is pretty much identical. This suggests that there is some bottleneck other than the disks, but I cannot understand how Server B originally had better I/O that has subsequently been 'lost'. Comparative I/O information below, as measured by SQLIO. The same parameters were used for each test. It's not the actual numbers that are significant but rather the variations between systems. In each case D: is a 2 disk RAID 1 volume, and E: is a 4 disk RAID 10 volume (apart from the original Server A, where E: was a 2 disk RAID 0 volume). Server A (original setup with 6Gpbs disks) D: Read (MB/s) 63 MB/s D: Write (MB/s) 170 MB/s E: Read (MB/s) 68 MB/s E: Write (MB/s) 320 MB/s Server B (original setup with 3Gpbs disks) D: Read (MB/s) 52 MB/s D: Write (MB/s) 88 MB/s E: Read (MB/s) 112 MB/s E: Write (MB/s) 130 MB/s Server A (new setup with 3Gpbs disks) D: Read (MB/s) 55 MB/s D: Write (MB/s) 85 MB/s E: Read (MB/s) 67 MB/s E: Write (MB/s) 180 MB/s Server B (new setup with 6Gpbs disks) D: Read (MB/s) 61 MB/s D: Write (MB/s) 95 MB/s E: Read (MB/s) 69 MB/s E: Write (MB/s) 180 MB/s Can anybody suggest any ideas what is going on here? The drives in use are as follows: Dell Seagate F617N ST3300657SS 300GB 15K RPM SAS Dell Hitachi HUS156030VLS600 300GB 3.5 inch 15000rpm 6GB SAS Hitachi Hus153030vls300 300GB Server SAS Dell ST3146855SS Seagate 3.5 inch 146GB 15K SAS

  Read the article

• Tuning Linux IP routing parameters -- secret_interval and tcp_mem

  - by Jeff Atwood

  We had a little failover problem with one of our HAProxy VMs today. When we dug into it, we found this: Jan 26 07:41:45 haproxy2 kernel: [226818.070059] __ratelimit: 10 callbacks suppressed Jan 26 07:41:45 haproxy2 kernel: [226818.070064] Out of socket memory Jan 26 07:41:47 haproxy2 kernel: [226819.560048] Out of socket memory Jan 26 07:41:49 haproxy2 kernel: [226822.030044] Out of socket memory Which, per this link, apparently has to do with low default settings for net.ipv4.tcp_mem. So we increased them by 4x from their defaults (this is Ubuntu Server, not sure if the Linux flavor matters): current values are: 45984 61312 91968 new values are: 183936 245248 367872 After that, we started seeing a bizarre error message: Jan 26 08:18:49 haproxy1 kernel: [ 2291.579726] Route hash chain too long! Jan 26 08:18:49 haproxy1 kernel: [ 2291.579732] Adjust your secret_interval! Shh.. it's a secret!! This apparently has to do with /proc/sys/net/ipv4/route/secret_interval which defaults to 600 and controls periodic flushing of the route cache The secret_interval instructs the kernel how often to blow away ALL route hash entries regardless of how new/old they are. In our environment this is generally bad. The CPU will be busy rebuilding thousands of entries per second every time the cache is cleared. However we set this to run once a day to keep memory leaks at bay (though we've never had one). While we are happy to reduce this, it seems odd to recommend dropping the entire route cache at regular intervals, rather than simply pushing old values out of the route cache faster. After some investigation, we found /proc/sys/net/ipv4/route/gc_elasticity which seems to be a better option for keeping the route table size in check: gc_elasticity can best be described as the average bucket depth the kernel will accept before it starts expiring route hash entries. This will help maintain the upper limit of active routes. We adjusted elasticity from 8 to 4, in the hopes of the route cache pruning itself more aggressively. The secret_interval does not feel correct to us. But there are a bunch of settings and it's unclear which are really the right way to go here. /proc/sys/net/ipv4/route/gc_elasticity (8) /proc/sys/net/ipv4/route/gc_interval (60) /proc/sys/net/ipv4/route/gc_min_interval (0) /proc/sys/net/ipv4/route/gc_timeout (300) /proc/sys/net/ipv4/route/secret_interval (600) /proc/sys/net/ipv4/route/gc_thresh (?) rhash_entries (kernel parameter, default unknown?) We don't want to make the Linux routing worse, so we're kind of afraid to mess with some of these settings. Can anyone advise which routing parameters are best to tune, for a high traffic HAProxy instance?

  Read the article

• How to correctly partition usb flash drive and which filesystem to choose considering wear leveling?

  - by random1

  Two problems. First one: how to partition the flash drive? I shouldn't need to do this, but I'm no longer sure if my partition is properly aligned since I was forced to delete and create a new partition table after gparted complained when I tried to format the drive from FAT to ext4. The naive answer would be to say "just use default and everything is going to be alright". However if you read the following links you'll know things are not that simple: https://lwn.net/Articles/428584/ and http://linux-howto-guide.blogspot.com/2009/10/increase-usb-flash-drive-write-speed.html Then there is also the issue of cylinders, heads and sectors. Currently I get this: $sfdisk -l -uM /dev/sdd Disk /dev/sdd: 30147 cylinders, 64 heads, 32 sectors/track Warning: The partition table looks like it was made for C/H/S=*/255/63 (instead of 30147/64/32). For this listing I'll assume that geometry. Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End MiB #blocks Id System /dev/sdd1 1 30146 30146 30869504 83 Linux $fdisk -l /dev/sdd Disk /dev/sdd: 31.6 GB, 31611420672 bytes 255 heads, 63 sectors/track, 3843 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00010c28 So from my current understanding I should align partitions at 4 MiB (currently it's at 1 MiB). But I still don't know how to set the heads and sectors properly for my device. Second problem: file system. From the benchmarks I saw ext4 provides the best performance, however there is the issue of wear leveling. How can I know that my Transcend JetFlash 700's microcontroller provides for wear leveling? Or will I just be killing my drive faster? I've seen a lot of posts on the web saying don't worry the newer drives already take care of that. But I've never seen a single piece of backed evidence of that and at some point people start mixing SSD with USB flash drives technology. The safe option would be to go for ext2, however a serious of tests that I performed showed horrible performance!!! These values are from a real scenario and not some synthetic test: 42 files: 3,429,415,284 bytes copied to flash drive original fat32: 15.1 MiB/s ext4 after new partition table: 10.2 MiB/s ext2 after new partition table: 1.9 MiB/s Please read the links that I posted above before answering. I would also be interested in answers backed up with some references because a lot is said and re-said but then it lacks facts. Thank you for the help.

  Read the article

• DB2 Integrity Checks and Exception Tables

  - by imthefirestartr

  I am working on planning a migration of a DB2 8.1 database from a horrible IBM encoding to UTF-8 to support further languages etc. I am encountering an issue that I am stuck on. A few notes on this migration: We are using db2move to export and load the data and db2look to get the details fo the database (tablespaces, tables, keys etc). We found the loading process worked nicely with db2move import, however, the data takes 7 hours to load and this was unacceptable downtime when we actually complete the conversion on the main database. We are now using db2move load, which is much faster as it seems to simply throw the data in without integrity checks. Which leads to my current issue. After completing the db2move load process, several tables are in a check pending state and require integrity checks. Integrity checks are done via the following: set integrity for . immediate checked This works for most tables, however, some tables give an error: DB21034E The command was processed as an SQL statement because it was not a valid Command Line Processor command. During SQL processing it returned: SQL3603N Check data processing through the SET INTEGRITY statement has found integrity violation involving a constraint with name "blah.SQL120124110232400". SQLSTATE=23514 The internets tell me that the solution to this issue is to create an exception table based on the actual table and tell the SET INTEGRITY command to send any exceptions to that table (as below): db2 create table blah_EXCEPTION like blah db2 SET INTEGRITY FOR blah IMMEDIATE CHECKED FOR EXCEPTION IN blah USE blah_EXCEPTION NOW, here is the specific issue I am having! The above forces all the rows with issues to the specified exception table. Well that's just super, buuuuuut I can not lose data in this conversion, its simply unacceptable. The internets and IBM has a vague description of sending the violations to the exception tables and then "dealing with the data" that is in the exception table. Unfortunately, I am not clear what this means and I was hoping that some wise individual knows and could help me out and let me know how I can retrieve this data from these tables and place the data in the original/proper table rather than these exception tables. Let me know if you have any questions. Thanks!

  Read the article

• Python Django sites on Apache+mod_wsgi with nginx proxy: highly fluctuating performance

  - by Halfgaar

  I have an Ubuntu 10.04 box running several dozen Python Django sites using mod_wsgi (embedded mode; the faster mode, if properly configured). Performance highly fluctuates. Sometimes fast, sometimes several seconds delay. The smokeping graphs are al over the place. Recently, I also added an nginx proxy for the static content, in the hopes it would cure the highly fluctuating performance. But, even though it reduced the number of requests Apache has to process significantly, it didn't help with the main problem. When clicking around on websites while running htop, it can be seen that sometimes requests are almost instant, whereas sometimes it causes Apache to consume 100% CPU for a few seconds. I really don't understand where this fluctuation comes from. I have configured the mpm_worker for Apache like this: StartServers 1 MinSpareThreads 50 MaxSpareThreads 50 ThreadLimit 64 ThreadsPerChild 50 MaxClients 50 ServerLimit 1 MaxRequestsPerChild 0 MaxMemFree 2048 1 server with 50 threads, max 50 clients. Munin and apache2ctl -t both show a consistent presence of workers; they are not destroyed and created all the time. Yet, it behaves as such. This tells me that once a sub interpreter is created, it should remain in memory, yet it seems sites have to reload all the time. I also have a nginx+gunicorn box, which performs quite well. I would really like to know why Apache is so random. This is a virtual host config: <VirtualHost *:81> ServerAdmin [email protected] ServerName example.com DocumentRoot /srv/http/site/bla Alias /static/ /srv/http/site/static Alias /media/ /srv/http/site/media WSGIScriptAlias / /srv/http/site/passenger_wsgi.py <Directory /> AllowOverride None </Directory> <Directory /srv/http/site> Options -Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> Ubuntu 10.04 Apache 2.2.14 mod_wsgi 2.8 nginx 0.7.65 Edit: I've put some code in the settings.py file of a site that writes the date to a tmp file whenever it's loaded. I can now see that the site is not randomly reloaded all the time, so Apache must be keeping it in memory. So, that's good, except it doesn't bring me closer to an answer... Edit: I just found an error that might also be related to this: File "/usr/lib/python2.6/subprocess.py", line 633, in __init__ errread, errwrite) File "/usr/lib/python2.6/subprocess.py", line 1049, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory The server has 600 of 2000 MB free, which should be plenty. Is there a limit that is set on Apache or WSGI somewhere?

  Read the article

• My server is slower than the average user's computer, should I still offload Access queries to SQL Server? [closed]

  - by andrewb

  Possible Duplicate: How do you do Load Testing and Capacity Planning for Databases I have a database set up with MS Access 2007 front ends and an SQL Server 2005 back end. At the moment, all the queries are saved in the front end as I've only recently moved to an SQL Server backend. I'm wondering how much of those queries I should save as stored procedures/views on SQL Server. About the system The number of concurrent users is only a handful, though it could be as high as 25 at one time (very unlikely). The average computer has an Intel i3-2120 CPU running at 3.3 GHz, which gets a PassMark score of 3,987, whilst the server has an Intel Xeon E5335 running at 2.0 GHz, which gets a PassMark score of 2,637. Always an awkward situation when an i3 outperforms a Xeon... though the i3 is from Q1 2011 and the Xeon is Q2 2009. There is potential for a server upgrade in the future, though it wouldn't come easy. I'm inclined to move the queries to the back end, as they are beginning to take noticeable time and I figure that is a better way of doing things. I like the idea of throwing everything at the server, then pushing for a server upgrade. It makes more sense in my mind to be upgrading one server rather than 30 PCs. Or am I being overzealous? Why my question isn't a duplicate It seems that my question has been misinterpreted and labelled a duplicate of quite a different question, one about testing and capacity planning. I'll try explain how my question is very different from the linked question. The crux of my question is something like "Even though my server is technically slower, is it better to have it doing more of the queries?" There's two ways that people could have answered this: I agree the server is going to be slower, but the extra benefits of such and such (like the less Access the better) means you should move most to the server anyway. (OR no it doesn't outweigh the benefit, keep them in Access) Actually the server will be faster because of such and such. I'm hoping that people out there could provide some answers like this, and the question in the dupe link doesn't really provide either of these answers. Ok sure, I suppose I could do extensive performance testing to compare Access queries running on a local machine to SQL Server queries running on the server, but that sounds like a very hard task (particularly performance testing of access) compared to someone giving some quick general guidance, and again, my question is looking for a lot more than immediate performance benefit.

  Read the article

• I/O intensive MySql server on Amazon AWS

  - by rhossi

  We recently moved from a traditional Data Center to cloud computing on AWS. We are developing a product in partnership with another company, and we need to create a database server for the product we'll release. I have been using Amazon Web Services for the past 3 years, but this is the first time I received a spec with this very specific hardware configuration. I know there are trade-offs and that real hardware will always be faster than virtual machines, and knowing that fact forehand, what would you recommend? 1) Amazon EC2? 2) Amazon RDS? 3) Something else? 4) Forget it baby, stick to the real hardware Here is the hardware requirements This server will be focused on I/O and MySQL for the statistics, memory size and disk space for the images hosting. Server 1 I/O The very main part on this server will be I/O processing, FusionIO cards have proven themselves extremely efficient, this is currently the best you can have in this domain. o Fusion ioDrive2 MLC 365GB (http://www.fusionio.com/load/-media-/1m66wu/docsLibrary/FIO_ioDrive2_Datasheet.pdf) CPU MySQL will use less CPU cores than Apache but it will use them very hard, the E7 family has 30M Cache L3 wichi provide boost performance : o 1x Intel E7-2870 will be ok. Storage SAS will be good enough in terms of performance, especially considering the space required. o RAID 10 of 4 x SAS 10k or 15k for a total available space of 512 GB. Memory o 64 GB minimum is required on this server considering the size of the statistics database. Warning: the statistics database will grow quickly, if possible consider starting with 128 GB directly, it will help. This server will be focused on I/O and MySQL for the statistics, memory size and disk space for the images hosting. Server 2 I/O The very main part on this server will be I/O processing, FusionIO cards have proven themselves extremely efficient, this is currently the best you can have in this domain. o Fusion ioDrive2 MLC 365GB (http://www.fusionio.com/load/-media-/1m66wu/docsLibrary/FIO_ioDrive2_Datasheet.pdf) CPU MySQL will use less CPU cores than Apache but it will use them very hard, the E7 family has 30M Cache L3 wichi provide boost performance : o 1x Intel E7-2870 will be ok. Storage SAS will be good enough in terms of performance, especially considering the space required. o RAID 10 of 4 x SAS 10k or 15k for a total available space of 512 GB. Memory o 64 GB minimum is required on this server considering the size of the statistics database. Warning: the statistics database will grow quickly, if possible consider starting with 128 GB directly, it will help. Thanks in advance. Best,

  Read the article

< Previous Page | 156 157 158 159 160 161 162 163 164 165 166 167  | Next Page >