Search Results

Search found 1042 results on 42 pages for 'jonathan drain'.

Page 41/42 | < Previous Page | 37 38 39 40 41 42  | Next Page >

• C# SerialPort - Problems mixing ports with different baud rates.

  - by GrandAdmiral

  Greetings, I have two devices that I would like to connect over a serial interface, but they have incompatible connections. To get around this problem, I connected them both to my PC and I'm working on a C# program that will route traffic on COM port X to COM port Y and vice versa. The program connects to two COM ports. In the data received event handler, I read in incoming data and write it to the other COM port. To do this, I have the following code: private void HandleDataReceived(SerialPort inPort, SerialPort outPort) { byte[] data = new byte[1]; while (inPort.BytesToRead > 0) { // Read the data data[0] = (byte)inPort.ReadByte(); // Write the data if (outPort.IsOpen) { outPort.Write(data, 0, 1); } } } That code worked fine as long as the outgoing COM port operated at a higher baud rate than the incoming COM port. If the incoming COM port was faster than the outgoing COM port, I started missing data. I had to correct the code like this: private void HandleDataReceived(SerialPort inPort, SerialPort outPort) { byte[] data = new byte[1]; while (inPort.BytesToRead > 0) { // Read the data data[0] = (byte)inPort.ReadByte(); // Write the data if (outPort.IsOpen) { outPort.Write(data, 0, 1); while (outPort.BytesToWrite > 0); //<-- Change to fix problem } } } I don't understand why I need that fix. I'm new to C# (this is my first program), so I'm wondering if there is something I am missing. The SerialPort defaults to a 2048 byte write buffer and my commands are less than ten bytes. The write buffer should have the ability to buffer the data until it can be written to a slower COM port. In summary, I'm receiving data on COM X and writing the data to COM Y. COM X is connected at a faster baud rate than COM Y. Why doesn't the buffering in the write buffer handle this difference? Why does it seem that I need to wait for the write buffer to drain to avoid losing data? Thanks!

  Read the article

• What could cause this difference in behaviour from iphone OS3.0 to iOS4.0?

  - by frankodwyer

  I am getting a strange EXC_BAD_ACCESS error when running my app on iOS4. The app has been pretty solid on OS3.x for some time - not even seeing crash logs in this area of the code (or many at all) in the wild. I've tracked the error down to this code: main class: - (void) sendPost:(PostRequest*)request { NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; NSURLResponse* response; NSError* error; NSData *serverReply = [NSURLConnection sendSynchronousRequest:request.request returningResponse:&response error:&error]; ServerResponse* serverResponse=[[ServerResponse alloc] initWithResponse:response error:error data:serverReply]; [request.objectToNotifyWhenDone performSelectorOnMainThread:request.targetToNotifyWhenDone withObject:serverResponse waitUntilDone:YES]; [pool drain]; } (Note: sendPost is run on a separate thread for each invocation of it. PostRequest is just a class to encapsulate a request and a selector to notify when complete) ServerResponse.m: @synthesize response; @synthesize replyString; @synthesize error; @synthesize plist; - (ServerResponse*) initWithResponse:(NSURLResponse*)resp error:(NSError*)err data:(NSData*)serverReply { self.response=resp; self.error=err; self.plist=nil; self.replyString=nil; if (serverReply) { self.replyString = [[[NSString alloc] initWithBytes:[serverReply bytes] length:[serverReply length] encoding: NSASCIIStringEncoding] autorelease]; NSPropertyListFormat format; NSString *errorStr; plist = [NSPropertyListSerialization propertyListFromData:serverReply mutabilityOption:NSPropertyListImmutable format:&format errorDescription:&errorStr]; } return self; } ServerResponse.h: @property (nonatomic, retain) NSURLResponse* response; @property (nonatomic, retain) NSString* replyString; @property (nonatomic, retain) NSError* error; @property (nonatomic, retain) NSDictionary* plist; - (ServerResponse*) initWithResponse:(NSURLResponse*)response error:(NSError*)error data:(NSData*)serverReply; This reliably crashes with a bad access in the line: self.error=err; ...i.e. in the synthesized property setter! I'm stumped as to why this should be, given the code worked on the previous OS and hasn't changed since (even the binary compiled with the previous SDK crashes the same way, but not on OS3.0) - and given it is a simple property method. Any ideas? Could the NSError implementation have changed between releases or am I missing something obvious?

  Read the article

• Updating a deallocated UIWebView from a background thread

  - by Dan Ray

  As you can see from the title, I've programmed myself into a corner and I've got several things working against me... In a UIViewController subclass that manages a large and complex view. One part of it is a UIWebView that contains output from a web request that I had to build and execute, and manually assemble HTML from. Since it takes a second or two to run, I dropped it into the background by calling self performSelectorInBackground:. Then from that method I call there, I use self performSelectorOnMainThread: to get back to the surface of the thread stack to update the UIWebView with what I just got. Like this (which I've cut down to show only the relevant issues): -(void)locationManager:(CLLocationManager *)manager didUpdateToLocation:(CLLocation *)newLocation fromLocation:(CLLocation *)oldLocation { //then get mapquest directions NSLog(@"Got called to handle new location!"); [manager stopUpdatingLocation]; [self performSelectorInBackground:@selector(getDirectionsFromHere:) withObject:newLocation]; } - (void)getDirectionsFromHere:(CLLocation *)newLocation { NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; CLLocationCoordinate2D here = newLocation.coordinate; // assemble a call to the MapQuest directions API in NSString *dirURL // ...cut for brevity NSLog(@"Query is %@", dirURL); NSString *response = [NSString stringWithContentsOfURL:[NSURL URLWithString:dirURL] encoding:NSUTF8StringEncoding error:NULL]; NSMutableString *directionsOutput = [[NSMutableString alloc] init]; // assemble response into an HTML table in NSString *directionsOutput // ...cut for brevity [self performSelectorOnMainThread:@selector(updateDirectionsWithHtml:) withObject:directionsOutput waitUntilDone:NO]; [directionsOutput release]; [pool drain]; [pool release]; } - (void)updateDirectionsWithHtml:(NSString *)directionsOutput { [self.directionsWebView loadHTMLString:directionsOutput baseURL:nil]; } This all works totally great, UNLESS I've backed out of this view controller before CLLocationManager hits its delegate method. If this happens after I've already left this view, I get: 2010-06-07 16:38:08.508 EverWondr[180:760b] bool _WebTryThreadLock(bool), 0x1b6830: Tried to obtain the web lock from a thread other than the main thread or the web thread. This may be a result of calling to UIKit from a secondary thread. Crashing now... Despite what this says, I can repeatably cause this crash when I back out too early. I'm not at all convinced that attempting a UI update from a background thread is really the issue; I think it's that my UIWebView is deallocated. I suspect that the fact I was just IN a background thread makes the runtime suspect something's up about that, but I feel fairly sure that's not it. So how do I tell CLLocationManager not to worry about it, when I'm backing out of that view? I tried [self.locationManager stopUpdatingLocation] inside my viewWillDisappear method, but that didn't do it. (Incidentally, MapQuest's apis are FANTASTIC. Way WAY better than anything Google provides. I can't recommend them highly enough.)

  Read the article

• iPhone: Speeding up a search that's polling 17,000 Core Data objects

  - by randombits

  I have a class that conforms to UISearchDisplayDelegate and contains a UISearchBar. This view is responsible for allowing the user to poll a store of about 17,000 objects that are currently managed by Core Data. Everytime the user types in a character, I created an instance of a SearchOperation (subclasses NSOperation) that queries Core Data to find results that might match the search. The code in the search controller looks something like: - (void)filterContentForSearchText:(NSString*)searchText scope:(NSString*)scope { // Update the filtered array based on the search text and scope in a secondary thread if ([searchText length] < 3) { [filteredList removeAllObjects]; // First clear the filtered array. [self setFilteredList:NULL]; [self.tableView reloadData]; return; } NSDictionary *searchdict = [NSDictionary dictionaryWithObjectsAndKeys:scope, @"scope", searchText, @"searchText", nil]; [aSearchQueue cancelAllOperations]; SearchOperation *searchOp = [[SearchOperation alloc] initWithDelegate:self dataDict:searchdict]; [aSearchQueue addOperation:searchOp]; } And my search is rather straight forward. SearchOperation is a subclass of NSOperation. I overwrote the main method with the following code: - (void)main { if ([self isCancelled]) { return; } NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; NSEntityDescription *entity = [NSEntityDescription entityForName:@"MyEntity" inManagedObjectContext:managedObjectContext]; NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init]; [fetchRequest setEntity:entity]; NSPredicate *predicate = NULL; predicate = [NSPredicate predicateWithFormat:@"(someattr contains[cd] %@)", searchText]; [fetchRequest setPredicate:predicate]; NSError *error = NULL; NSArray *fetchResults = [managedObjectContext executeFetchRequest:fetchRequest error:&error]; [fetchRequest release]; if (self.delegate != nil) [self.delegate didFinishSearching:fetchResults]; [pool drain]; } This code works, but it has several issues. It's slow. Even though I have the search happening in a separate thread other than the UI thread, querying 17,000 objects is clearly not optimal. If I'm not careful, crashes can happen. I set the max concurrent searches in my NSOperationQueue to 1 to avoid this. What else can I do to make this search faster? I think preloading all 17,000 objects into memory might be risky. There has to be a smarter way to conduct this search to give results back to the user faster.

  Read the article

• Help with Arrays in Objective C.

  - by NJTechie

  Problem : Take an integer as input and print out number equivalents of each number from input. I hacked my thoughts to work in this case but I know it is not an efficient solution. For instance : 110 Should give the following o/p : one one zero Could someone throw light on effective usage of Arrays for this problem? #import <Foundation/Foundation.h> int main (int argc, const char * argv[]) { NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init]; int input, i=0, j,k, checkit; int temp[i]; NSLog(@"Enter an integer :"); scanf("%d", &input); checkit = input; while(input > 0) { temp[i] = input%10; input = input/10; i++; } if(checkit != 0) { for(j=i-1;j>=0;j--) { //NSLog(@" %d", temp[j]); k = temp[j]; //NSLog(@" %d", k); switch (k) { case 0: NSLog(@"zero"); break; case 1: NSLog(@"one"); break; case 2: NSLog(@"two"); break; case 3: NSLog(@"three"); break; case 4: NSLog(@"four"); break; case 5: NSLog(@"five"); break; case 6: NSLog(@"six"); break; case 7: NSLog(@"seven"); break; case 8: NSLog(@"eight"); break; case 9: NSLog(@"nine"); break; default: break; } } } else NSLog(@"zero"); [pool drain]; return 0; }

  Read the article

• Gradual memory leak in loop over contents of QTMovie

  - by Benji XVI

  I have a simple foundation tool that exports every frame of a movie as a .tiff file. Here is the relevant code: NSString* movieLoc = [NSString stringWithCString:argv[1]]; QTMovie *sourceMovie = [QTMovie movieWithFile:movieLoc error:nil]; int i=0; while (QTTimeCompare([sourceMovie currentTime], [sourceMovie duration]) != NSOrderedSame) { // save image of movie to disk NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init]; NSString *filePath = [NSString stringWithFormat:@"/somelocation_%d.tiff", i++]; NSData *currentImageData = [[sourceMovie currentFrameImage] TIFFRepresentation]; [currentImageData writeToFile:filePath atomically:NO]; NSLog(@"%@", filePath); [sourceMovie stepForward]; [arp release]; } [pool drain]; return 0; As you can see, in order to prevent very large memory buildups with the various transparently-autoreleased variables in the loop, we create, and flush, an autoreleasepool with every run through the loop. However, over the course of stepping through a movie, the amount of memory used by the program still gradually increases. Instruments is not detecting any memory leaks per se, but the object trace shows certain General Data blocks to be increasing in size. [Edited out reference to slowdown as it doesn't seem to be as much of a problem as I thought.] Edit: let's knock out some parts of the code inside the loop & see what we find out... Test 1 while (banana) { NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init]; NSString *filePath = [NSString stringWithFormat:@"/somelocation_%d.tiff", i++]; NSLog(@"%@", filePath); [sourceMovie stepForward]; [arp release]; } Here we simply loop over the whole movie, creating the filename and logging it. Memory characteristics: remains at 15MB usage for the duration. Test 2 while (banana) { NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init]; NSImage *image = [sourceMovie currentFrameImage]; [sourceMovie stepForward]; [arp release]; } Here we add back in the creation of the NSImage from the current frame. Memory characteristics: gradually increasing memory usage. RSIZE is at 60MB by frame 200; 75MB by f300. Test 3 while (banana) { NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init]; NSImage *image = [sourceMovie currentFrameImage]; NSData *imageData = [image TIFFRepresentation]; [sourceMovie stepForward]; [arp release]; } We've added back in the creation of an NSData object from the NSImage. Memory characteristics: Memory usage is again increasing: 62MB at f200; 75MB at f300. In other words, largely identical. It looks like a memory leak in the underlying system QTMovie uses to do currentFrameImage, to me.

  Read the article

• Are Objective-C initializers allowed to share the same name?

  - by NattKatt

  I'm running into an odd issue in Objective-C when I have two classes using initializers of the same name, but differently-typed arguments. For example, let's say I create classes A and B: A.h: #import <Cocoa/Cocoa.h> @interface A : NSObject { } - (id)initWithNum:(float)theNum; @end A.m: #import "A.h" @implementation A - (id)initWithNum:(float)theNum { self = [super init]; if (self != nil) { NSLog(@"A: %f", theNum); } return self; } @end B.h: #import <Cocoa/Cocoa.h> @interface B : NSObject { } - (id)initWithNum:(int)theNum; @end B.m: #import "B.h" @implementation B - (id)initWithNum:(int)theNum { self = [super init]; if (self != nil) { NSLog(@"B: %d", theNum); } return self; } @end main.m: #import <Foundation/Foundation.h> #import "A.h" #import "B.h" int main (int argc, const char * argv[]) { NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init]; A *a = [[A alloc] initWithNum:20.0f]; B *b = [[B alloc] initWithNum:10]; [a release]; [b release]; [pool drain]; return 0; } When I run this, I get the following output: 2010-04-26 20:44:06.820 FnTest[14617:a0f] A: 20.000000 2010-04-26 20:44:06.823 FnTest[14617:a0f] B: 1 If I reverse the order of the imports so it imports B.h first, I get: 2010-04-26 20:45:03.034 FnTest[14635:a0f] A: 0.000000 2010-04-26 20:45:03.038 FnTest[14635:a0f] B: 10 For some reason, it seems like it's using the data type defined in whichever @interface gets included first for both classes. I did some stepping through the debugger and found that the isa pointer for both a and b objects ends up the same. I also found out that if I no longer make the alloc and init calls inline, both initializations seem to work properly, e.g.: A *a = [A alloc]; [a initWithNum:20.0f]; If I use this convention when I create both a and b, I get the right output and the isa pointers seem to be different for each object. Am I doing something wrong? I would have thought multiple classes could have the same initializer names, but perhaps that is not the case.

  Read the article

• Objective C "do - while" question

  - by Rob

  The example for one of the exercises in the book I am reading shows the following code: #import <Foundation/Foundation.h> int main (int argc, const char * argv[]) { NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init]; int input, reverse, numberOfDigits; reverse = 0; numberOfDigits = 0; NSLog (@"Please input a multi-digit number:"); scanf ("%i", &input); if ( input < 0 ) { input = -input; NSLog (@"Minus"); } do { reverse = reverse * 10 + input % 10; numberOfDigits++; } while (input /= 10); do { switch ( reverse % 10 ) { case 0: NSLog (@"Zero"); break; case 1: NSLog (@"One"); break; case 2: NSLog (@"Two"); break; case 3: NSLog (@"Three"); break; case 4: NSLog (@"Four"); break; case 5: NSLog (@"Five"); break; case 6: NSLog (@"Six"); break; case 7: NSLog (@"Seven"); break; case 8: NSLog (@"Eight"); break; case 9: NSLog (@"Nine"); break; } numberOfDigits--; } while (reverse /= 10); while (numberOfDigits--) { NSLog (@"Zero"); } [pool drain]; return 0; } My question is this, the while statement shows (input /= 10) which, if I understand this correctly basically means (input = input / 10). Now, if that is true, why doesn't the loop just run continuously? I mean, even if you were to divide 0 by 10 then that would still extract a number. If the user was to input "50607", it would first cut off the "7", then the "0", and so on and so on, but why does it exit the loop after removing the "5". Wouldn't the response after the "5" be the same as the "0" between the 5 and the 6 to the program?

  Read the article

• C# .NET Why does my inherited listview keep drawing in LargeIcon View ?? Because Microsoft is Evil!!

  - by Bugz R us

  I have a inherited Listview which standard has to be in Tile Mode. When using this control, the DrawItem gives e.bounds which are clearly bounds of largeIcon view ?? When debugging to check the view it is actually set to, it says it's in Tile view ?? Yet e.DrawText draws LargeIcon view ?? ......... Edit: ................. This seems only to happen when the control is placed upon another usercontrol? ......... Edit 2: ................. It gets stranger ... When i add buttons next to the list to change the view at runtime, "Tile" is the same as "LargeIcon", and "List" view is the same as "SmallIcons" ??? I've also completely removed the ownerdraw ... .......... Edit 3: ................. MSDN Documentation: Tile view Each item appears as a full-sized icon with the item label and subitem information to the right of it. The subitem information that appears is specified by the application. This view is available only on Windows XP and the Windows Server 2003 family. On earlier operating systems, this value is ignored and the ListView control displays in the LargeIcon view. Well I am on XP ya damn liars ?!? Apparently if the control is within a usercontrol, this value is ignored too ... pff I'm getting enough of this Microsoft crap .... you just keep on hitting bugs ... another day down the drain ... public class InheritedListView : ListView { //Hiding members ... mwuahahahahaha //yeah i was still laughing then [BrowsableAttribute(false)] public new View View { get { return base.View; } } public InheritedListView() { base.View = View.Tile; this.OwnerDraw = true; base.DrawItem += new DrawListViewItemEventHandler(DualLineGrid_DrawItem); } void DualLineGrid_DrawItem(object sender, DrawListViewItemEventArgs e) { View v = this.View; //**when debugging, v is Tile, however e.DrawText() draws in LargeIcon mode, // e.Bounds also reflects LargeIcon mode ???? ** }

  Read the article

• SQL SERVER – CXPACKET – Parallelism – Usual Solution – Wait Type – Day 6 of 28

  - by pinaldave

  CXPACKET has to be most popular one of all wait stats. I have commonly seen this wait stat as one of the top 5 wait stats in most of the systems with more than one CPU. Books On-Line: Occurs when trying to synchronize the query processor exchange iterator. You may consider lowering the degree of parallelism if contention on this wait type becomes a problem. CXPACKET Explanation: When a parallel operation is created for SQL Query, there are multiple threads for a single query. Each query deals with a different set of the data (or rows). Due to some reasons, one or more of the threads lag behind, creating the CXPACKET Wait Stat. There is an organizer/coordinator thread (thread 0), which takes waits for all the threads to complete and gathers result together to present on the client’s side. The organizer thread has to wait for the all the threads to finish before it can move ahead. The Wait by this organizer thread for slow threads to complete is called CXPACKET wait. Note that not all the CXPACKET wait types are bad. You might experience a case when it totally makes sense. There might also be cases when this is unavoidable. If you remove this particular wait type for any query, then that query may run slower because the parallel operations are disabled for the query. Reducing CXPACKET wait: We cannot discuss about reducing the CXPACKET wait without talking about the server workload type. OLTP: On Pure OLTP system, where the transactions are smaller and queries are not long but very quick usually, set the “Maximum Degree of Parallelism” to 1 (one). This way it makes sure that the query never goes for parallelism and does not incur more engine overhead. EXEC sys.sp_configure N'cost threshold for parallelism', N'1' GO RECONFIGURE WITH OVERRIDE GO Data-warehousing / Reporting server: As queries will be running for long time, it is advised to set the “Maximum Degree of Parallelism” to 0 (zero). This way most of the queries will utilize the parallel processor, and long running queries get a boost in their performance due to multiple processors. EXEC sys.sp_configure N'cost threshold for parallelism', N'0' GO RECONFIGURE WITH OVERRIDE GO Mixed System (OLTP & OLAP): Here is the challenge. The right balance has to be found. I have taken a very simple approach. I set the “Maximum Degree of Parallelism” to 2, which means the query still uses parallelism but only on 2 CPUs. However, I keep the “Cost Threshold for Parallelism” very high. This way, not all the queries will qualify for parallelism but only the query with higher cost will go for parallelism. I have found this to work best for a system that has OLTP queries and also where the reporting server is set up. Here, I am setting ‘Cost Threshold for Parallelism’ to 25 values (which is just for illustration); you can choose any value, and you can find it out by experimenting with the system only. In the following script, I am setting the ‘Max Degree of Parallelism’ to 2, which indicates that the query that will have a higher cost (here, more than 25) will qualify for parallel query to run on 2 CPUs. This implies that regardless of the number of CPUs, the query will select any two CPUs to execute itself. EXEC sys.sp_configure N'cost threshold for parallelism', N'25' GO EXEC sys.sp_configure N'max degree of parallelism', N'2' GO RECONFIGURE WITH OVERRIDE GO Read all the post in the Wait Types and Queue series. Additionally a must read comment of Jonathan Kehayias. Note: The information presented here is from my experience and I no way claim it to be accurate. I suggest you all to read the online book for further clarification. All the discussion of Wait Stats over here is generic and it varies from system to system. It is recommended that you test this on the development server before implementing on the production server. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: DMV, Pinal Dave, PostADay, SQL, SQL Authority, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, SQL Wait Stats, SQL Wait Types, T SQL, Technology

  Read the article

• SQLIO Writes

  - by Grant Fritchey

  SQLIO is a fantastic utility for testing the abilities of the disks in your system. It has a very unfortunate name though, since it's not really a SQL Server testing utility at all. It really is a disk utility. They ought to call it DiskIO because they'd get more people using I think. Anyway, branding is not the point of this blog post. Writes are the point of this blog post. SQLIO works by slamming your disk. It performs as mean reads as it can or it performs as many writes as it can depending on how you've configured your tests. There are much smarter people than me who will get into all the various types of tests you should run. I'd suggest reading a bit of what Jonathan Kehayias (blog|twitter) has to say or wade into Denny Cherry's (blog|twitter) work. They're going to do a better job than I can describing all the benefits and mechanisms around using this excellent piece of software. My concerns are very focused. I needed to set up a series of tests to see how well our product SQL Storage Compress worked. I wanted to know the effects it would have on a system, the disk for sure, but also memory and CPU. How to stress the system? SQLIO of course. But when I set it up and ran it, following the documentation that comes with it, I was seeing better than 99% compression on the files. Don't get me wrong. Our product is magnificent, wonderful, all things great and beautiful, gets you coffee in the morning and is made mostly from bacon. But 99% compression. No, it's not that good. So what's up? Well, it's the configuration. The default mechanism is to load up a file, something large that will overwhelm your disk cache. You're instructed to load the file with a character 0x0. I never got a computer science degree. I went to film school. Because of this, I didn't memorize ASCII tables so when I saw this, I thought it was zero's or something. Nope. It's NULL. That's right, you're making a very large file, but you're filling it with NULL values. That's actually ok when all you're testing is the disk sub-system. But, when you want to test a compression and decompression, that can be an issue. I got around this fairly quickly. Instead of generating a file filled with NULL values, I just copied a database file for my tests. And to test it with SQL Storage Compress, I used a database file that had already been run through compression (about 40% compression on that file if you're interested). Now the reads were taken care of. I am seeing very realistic performance from decompressing the information for reads through SQLIO. But what about writes? Well, the issue is, what does SQLIO write? I don't have access to the code. But I do have access to the results. I did two different tests, just to be sure of what I was seeing. First test, use the .DAT file as described in the documentation. I opened the .DAT file after I was done with SQLIO, using WordPad. Guess what? It's a giant file full of air. SQLIO writes NULL values. What does that do to compression? I did the test again on a copy of an uncompressed database file. Then I ran the original and the SQLIO modified copy through ZIP to see what happened. I got better than 99% compression out of the SQLIO modified file (original file of 624,896kb went to 275,871kb compressed, after SQLIO it went to 608kb compressed). So, what does SQLIO write? It writes air. If you're trying to test it with compression or maybe some other type of file storage mechanism like dedupe, you need to know this because your tests really won't be valid. Should I find some other mechanism for testing? Yeah, if all I'm interested in is establishing performance to my own satisfaction, yes. But, I want to be able to compare my results with other people's results and we all need to be using the same tool in order for that to happen. SQLIO is the common mechanism that most people I know use to establish disk performance behavior. It'd be better if we could get SQLIO to do writes in some other fashion. Oh, and before I go, I get to brag a bit. Measuring IOPS, SQL Storage Compress outperforms my disk alone by about 30%.

  Read the article

• How to get rid of a stubborn 'removed' device in mdadm

  - by T.J. Crowder

  One of my server's drives failed and so I removed the failed drive from all three relevant arrays, had the drive swapped out, and then added the new drive to the arrays. Two of the arrays worked perfectly. The third added the drive back as a spare, and there's an odd "removed" entry in the mdadm details. I tried both mdadm /dev/md2 --remove failed and mdadm /dev/md2 --remove detached as suggested here and here, neither of which complained, but neither of which had any effect, either. Does anyone know how I can get rid of that entry and get the drive added back properly? (Ideally without resyncing a third time, I've already had to do it twice and it takes hours. But if that's what it takes, that's what it takes.) The new drive is /dev/sda, the relevant partition is /dev/sda3. Here's the detail on the array: # mdadm --detail /dev/md2 /dev/md2: Version : 0.90 Creation Time : Wed Oct 26 12:27:49 2011 Raid Level : raid1 Array Size : 729952192 (696.14 GiB 747.47 GB) Used Dev Size : 729952192 (696.14 GiB 747.47 GB) Raid Devices : 2 Total Devices : 2 Preferred Minor : 2 Persistence : Superblock is persistent Update Time : Tue Nov 12 17:48:53 2013 State : clean, degraded Active Devices : 1 Working Devices : 2 Failed Devices : 0 Spare Devices : 1 UUID : 2fdbf68c:d572d905:776c2c25:004bd7b2 (local to host blah) Events : 0.34665 Number Major Minor RaidDevice State 0 0 0 0 removed 1 8 19 1 active sync /dev/sdb3 2 8 3 - spare /dev/sda3 If it's relevant, it's a 64-bit server. It normally runs Ubuntu, but right now I'm in the data centre's "rescue" OS, which is Debian 7 (wheezy). The "removed" entry was there the last time I was in Ubuntu (it won't, currently, boot from the disk), so I don't think that's not some Ubuntu/Debian conflict (and they are, of course, closely related). Update: Having done extensive tests with test devices on a local machine, I'm just plain getting anomalous behavior from mdadm with this array. For instance, with /dev/sda3 removed from the array again, I did this: mdadm /dev/md2 --grow --force --raid-devices=1 And that got rid of the "removed" device, leaving me just with /dev/sdb3. Then I nuked /dev/sda3 (wrote a file system to it, so it didn't have the raid fs anymore), then: mdadm /dev/md2 --grow --raid-devices=2 ...which gave me an array with /dev/sdb3 in slot 0 and "removed" in slot 1 as you'd expect. Then mdadm /dev/md2 --add /dev/sda3 ...added it — as a spare again. (Another 3.5 hours down the drain.) So with the rebuilt spare in the array, given that mdadm's man page says RAID-DEVICES CHANGES ... When the number of devices is increased, any hot spares that are present will be activated immediately. ...I grew the array to three devices, to try to activate the "spare": mdadm /dev/md2 --grow --raid-devices=3 What did I get? Two "removed" devices, and the spare. And yet when I do this with a test array, I don't get this behavior. So I nuked /dev/sda3 again, used it to create a brand-new array, and am copying the data from the old array to the new one: rsync -r -t -v --exclude 'lost+found' --progress /mnt/oldarray/* /mnt/newarray This will, of course, take hours. Hopefully when I'm done, I can stop the old array entirely, nuke /dev/sdb3, and add it to the new array. Hopefully, it won't get added as a spare!

  Read the article

• Kernel panic when bringing up DRBD resource

  - by sc.

  I'm trying to set up two machines synchonizing with DRBD. The storage is setup as follows: PV - LVM - DRBD - CLVM - GFS2. DRBD is set up in dual primary mode. The first server is set up and running fine in primary mode. The drives on the first server have data on them. I've set up the second server and I'm trying to bring up the DRBD resources. I created all the base LVM's to match the first server. After initializing the resources with `` drbdadm create-md storage I'm bringing up the resources by issuing drbdadm up storage After issuing that command, I get a kernel panic and the server reboots in 30 seconds. Here's a screen capture. My configuration is as follows: OS: CentOS 6 uname -a Linux host.structuralcomponents.net 2.6.32-279.5.2.el6.x86_64 #1 SMP Fri Aug 24 01:07:11 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux rpm -qa | grep drbd kmod-drbd84-8.4.1-2.el6.elrepo.x86_64 drbd84-utils-8.4.1-2.el6.elrepo.x86_64 cat /etc/drbd.d/global_common.conf global { usage-count yes; # minor-count dialog-refresh disable-ip-verification } common { handlers { pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b /proc/sysrq-trigger ; reboot -f"; pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b /proc/sysrq-trigger ; reboot -f"; local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o /proc/sysrq-trigger ; halt -f"; # fence-peer "/usr/lib/drbd/crm-fence-peer.sh"; # split-brain "/usr/lib/drbd/notify-split-brain.sh root"; # out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root"; # before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k"; # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh; } startup { # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb become-primary-on both; wfc-timeout 30; degr-wfc-timeout 10; outdated-wfc-timeout 10; } options { # cpu-mask on-no-data-accessible } disk { # size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes # disk-drain md-flushes resync-rate resync-after al-extents # c-plan-ahead c-delay-target c-fill-target c-max-rate # c-min-rate disk-timeout } net { # protocol timeout max-epoch-size max-buffers unplug-watermark # connect-int ping-int sndbuf-size rcvbuf-size ko-count # allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri # after-sb-1pri after-sb-2pri always-asbp rr-conflict # ping-timeout data-integrity-alg tcp-cork on-congestion # congestion-fill congestion-extents csums-alg verify-alg # use-rle protocol C; allow-two-primaries yes; after-sb-0pri discard-zero-changes; after-sb-1pri discard-secondary; after-sb-2pri disconnect; } } cat /etc/drbd.d/storage.res resource storage { device /dev/drbd0; meta-disk internal; on host.structuralcomponents.net { address 10.10.1.120:7788; disk /dev/vg_storage/lv_storage; } on host2.structuralcomponents.net { address 10.10.1.121:7788; disk /dev/vg_storage/lv_storage; } /var/log/messages is not logging anything about the crash. I've been trying to find a cause of this but I've come up with nothing. Can anyone help me out? Thanks.

  Read the article

• Issues configuring Exchange 2010 as well as SSL problems.

  - by Eric Smith

  Possibly-Relevant Background Info: I've recently moved up from icky shared hosting to a glorious, Remote Desktop-administrated VPS server running Windows Server 2008 R2. Even though I'm only 21 now and a computer science major, I've tried to play with every Windows Server release since '03, just to learn new things. What usually happens is inevitably I'll do something wrong and pretty much ruin the install. You're dealing with an amateur here :) Through the past few months of working with my new server, I've mastered DNS, IIS, got Team Foundation Server running (yay!), and can install all of the other basics like SQL Server and Active Directory. The Problem: Now, these last few weeks I've been trying to install Exchange Server 2010 (SP1). To make a long story short, it took me several attempts, and I even had to get my server wiped just so I could start fresh since Exchange decided uninstalling properly was for sissies (cost me $20, bah). Today, at long last, I got Exchange mostly working. There were two main problems left, however, that left me unsatisfied: Exchange installed itself and all of its child sites into Default Web Site. I wanted to access Exchange via mail.domain.com, but instead everything was configured to domain.com. My limited server admin knowledge was not enough to configure IIS or Exchange to move itself over to the website I had set up for it, appropriately titled 'mail.domain.com', which I had bound to a dedicated IP address (I was told this was necessary, but he may have been wrong). I have two SSL certificates: one for my main domain and one for my mail subdomain. For whatever reason, I had issues geting Exchange to use my mail certificate, even though I had assigned the proper roles in the MMC. I did, at one point, get it to work (or mostly work, anyways. Frankly, my memory of today is clouded by intense frustration). Additionally, I was confused which type of SSL certificate I should be using for Exchange. My SSL provider, GoDaddy, allows me to request a new certificate whenever, so I can use either the certificate request provided by IIS or the more complicated and specific request you can create with Exchange. Which type should I be using, the IIS or Exchange certificate? If I must use the Exchange certificate, will that 1) cause issues when I bind that certificate to my mail.domain.com subdomain or 2) is that an unnecessary step? The SSL Certificate Strikes Back When I thought I had the proper SSL certificate assigned for those brief, sweet moments, Google Chrome reported the correct mail.domain.com certificate when browsing https://mail.domain.com. However, Outlook 2010 threw up an error when trying to configure my email account claiming that the certificate didn't match the domain of "mail.domain.com". Is this an issue that will be resolved by problem #2 or is it a separate one entirely? Apologies for the massive wall of text, but I wanted to provide as much info as I possibly could. Exchange is the last thing I'd like installed on my server, and naturally it's turning out to be the hardest. Thanks for any info at all. Even a point in a vague direction would be a huge help at this point. Thanks! -Eric P.S.: The reason I keep ruining my install is that when I attempt to uninstall Exchange, something invariably goes wrong. The last time the uninstaller complained that there was still a mailbox active and it couldn't proceed until I deleted it. ... The only mailbox left was the Administrator account, the built-in one I couldn't delete. So I attempted to manually uninstall it following several guides online only to now be stuck unable to launch the installer and have to get my system wiped AGAIN for the second time today ($40 down the drain, bah!). I do not understand at all why "uninstall" just can't mean "hey, you, delete everything and go away". There's not even a force uninstall option, only a "recover system" option that just fails to fix anything and makes it so I can't even use the GUI uninstaller. </rant>

  Read the article

• TFS, G.I. Joe and Under-doing

  If I were to rank the most consistently irritating parts of my work day, using TFS would come in first by a wide margin. Even repeated network outages this week seem like a pleasant reprieve from this monolithic beast. This is not a reflexive anti-Microsoft feeling, that attitude just wouldnt work for a consultant who does .NET development. It is also not an utter dismissal of TFS as worthless; Ive seen people use it effectively on several projects. So why? Ill start with a laundry list of shortcomings. An out of the box UI for work items that is insultingly bad, a source control system that is confoundingly fragile when handling merges, folder renames and long file names, the arcane XML wizardry necessary to customize a template and a build system that adds an extra layer of oddness on top of msbuild. Im sure my legion of readers will soon point out to me how I can work around all these issues, how this is fixed in TFS 2010 or with this add-in, and how once you have everything set up, youre fine. And theyd be right, any one of these problems could be worked around. If not dirty laundry, what else? I thought about it for a while, and came to the conclusion that TFS is so irritating to me because it represents a vision of software development that I find unappealing. To expand upon this, lets start with some wisdom from those great PSAs at the end of the G.I. Joe cartoons of the 80s: Now you know, and knowing is half the battle. In software development, Id go further and say knowing is more than half the battle. Understanding the dimensions of the problem you are trying to solve, the needs of the users, the value that your software can provide are more than half the battle. Implementation of this understanding is not easy, but it is not even possible without this knowledge. Assuming we have a fixed amount of time and mental energy for any project, why does this spell trouble for TFS? If you think about what TFS is doing, its offering you a huge array of options to track the day to day implementation of your project. From tasks, to code churn, to test coverage. All valuable metrics, but only in exchange for valuable time to get it all working. In addition, when you have a shiny toy like TFS, the temptation is to feel obligated to use it. So the push from TFS is to encourage a project manager and team to focus on process and metrics around process. You can get great visibility, and graphs to show your project stakeholders, but none of that is important if you are not implementing the right product. Not just unimportant, these activities can be harmful as they drain your time and sap your creativity away from the rest of the project. To be more concrete, lets suppose your organization has invested the time to create a template for your projects and trained people in how to use it, so there is no longer a big investment of time for each project to get up and running. First, Id challenge if that template could be specific enough to be full featured and still applicable for any project. Second, the very existence of this template would be a indication to a project manager that the success of their project was somehow directly related to fitting management of that project into this format. Again, while the capabilities are wonderful, the mirage is there; just get everything into TFS and your project will run smoothly. Ill close the loop on this first topic by proposing a thought experiment. Think of the projects youve worked on. How many times have you been chagrined to discover youve implemented the wrong feature, misunderstood how a feature should work or just plain spent too much time on a screen that nobody uses? That sounds like a really worthwhile area to invest time in improving. How about going back to these projects and thinking about how many times you wished you had optimized the state change flow of your tasks or been embarrassed to not have a code churn report linked back to the latest changeset? With thanks to the Real American Heroes, Ill move on to a more current influence, that of the developers at 37signals, and their philosophy towards software development. This philosophy, fully detailed in the books Getting Real and Rework, is a vision of software that under does the competition. This is software that is deliberately limited in functionality in order to concentrate fully on making sure ever feature that is there is awesome and needed. Why is this relevant? Well, in one of those fun seeming paradoxes in life, constraints can be a spark for creativity. Think Twitter, the small screen of an iPhone, the limitations of HTML for applications, the low memory limits of older or embedded system. As long as there is some freedom within those constraints, amazing things emerge. For project management, some of the most respected people in the industry recommend using just index cards, pens and tape. They argue that with change the constant in software development, your process should be as limited (yet rigorous) as possible. Looking at TFS, this is not a system designed to under do anybody. It is a big jumble of components and options, with every feature you could think of. Predictably this means many basic functions are hard to use. For task management, many people just use an Excel spreadsheet linked up to TFS. Not a stirring endorsement of the tooling there. TFS as a whole would be far more appealing to me if there was less of it, but better. Id cut 50% of the features to make the other half really amaze and inspire me. And thats really the heart of the matter. TFS has great promise and I want to believe it can work better. But ultimately it focuses your attention on a lot of stuff that doesnt really matter and then clamps down your creativity in a mess of forms and dialogs obscuring what does.   --- Relevant Links --- All those great G.I. Joe PSAs are on YouTube, including lots of mashed up versions. A simple Google search will get you on the right track.Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

  Read the article

• Which functions in the C standard library commonly encourage bad practice?

  - by Ninefingers

  Hello all, This is inspired by this question and the comments on one particular answer in that I learnt that strncpy is not a very safe string handling function in C and that it pads zeros, until it reaches n, something I was unaware of. Specifically, to quote R.. strncpy does not null-terminate, and does null-pad the whole remainder of the destination buffer, which is a huge waste of time. You can work around the former by adding your own null padding, but not the latter. It was never intended for use as a "safe string handling" function, but for working with fixed-size fields in Unix directory tables and database files. snprintf(dest, n, "%s", src) is the only correct "safe strcpy" in standard C, but it's likely to be a lot slower. By the way, truncation in itself can be a major bug and in some cases might lead to privilege elevation or DoS, so throwing "safe" string functions that truncate their output at a problem is not a way to make it "safe" or "secure". Instead, you should ensure that the destination buffer is the right size and simply use strcpy (or better yet, memcpy if you already know the source string length). And from Jonathan Leffler Note that strncat() is even more confusing in its interface than strncpy() - what exactly is that length argument, again? It isn't what you'd expect based on what you supply strncpy() etc - so it is more error prone even than strncpy(). For copying strings around, I'm increasingly of the opinion that there is a strong argument that you only need memmove() because you always know all the sizes ahead of time and make sure there's enough space ahead of time. Use memmove() in preference to any of strcpy(), strcat(), strncpy(), strncat(), memcpy(). So, I'm clearly a little rusty on the C standard library. Therefore, I'd like to pose the question: What C standard library functions are used inappropriately/in ways that may cause/lead to security problems/code defects/inefficiencies? In the interests of objectivity, I have a number of criteria for an answer: Please, if you can, cite design reasons behind the function in question i.e. its intended purpose. Please highlight the misuse to which the code is currently put. Please state why that misuse may lead towards a problem. I know that should be obvious but it prevents soft answers. Please avoid: Debates over naming conventions of functions (except where this unequivocably causes confusion). "I prefer x over y" - preference is ok, we all have them but I'm interested in actual unexpected side effects and how to guard against them. As this is likely to be considered subjective and has no definite answer I'm flagging for community wiki straight away. I am also working as per C99.

  Read the article

• how to use SQL wildcard % with Queryset extra>select?

  - by tylias

  I'm trying to add weights to search terms I'm using to filter a queryset. Using the '%' wildcard is causing me some problems. I'm using the extra() modifier to add a weight parameter to the queryset, which I will be using to inform a sort ordering. (See http://docs.djangoproject.com/en/1.1/ref/models/querysets/#extra-select-none-where-none-params-none-tables-none-order-by-none-select-params-none ) Here's the gist of the code: def viewname(request) ... exact_matchstrings="" exact_matchstrings.append("(accountprofile.first_name LIKE '" + term + "')") exact_matchstrings.append("(accountprofile.first_name LIKE '" + term + '\%' + "')") extraquerystring = " + ".join(exact_matchstrings) return_queryset = return_queryset.extra( select = { 'match_weight': extraquerystring }, ) The effect I'm going for is that if the search term matches exactly, the weight associated with the record is 2, but if the term merely starts with the search term and isn't an exact match, the weight is 1. (for example, if 'term' = 'Jon', an entry with first_name='Jon' gets a weight of 2 but an entry with an entry with first_name = 'Jonathan' gets a weight of 1.) I can test the statement in SQL and it seems to work well enough. If I make this SQL query from the mysql shell, no problem: select (first_name like "Carl") + (first_name like "Car%") from accountprofile; But trying to run it via the extra() modifier in my view code and evaluating the resulting queryset gives me the following error: Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 68, in __repr__ data = list(self[:REPR_OUTPUT_SIZE + 1]) File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 83, in __len__ self._result_cache.extend(list(self._iter)) File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 238, in iterator for row in self.query.results_iter(): File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 287, in results_iter for rows in self.execute_sql(MULTI): File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 2369, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py", line 22, in execute sql = self.db.ops.last_executed_query(self.cursor, sql, params) File "/usr/local/lib/python2.6/dist-packages/django/db/backends/__init__.py", line 217, in last_executed_query return smart_unicode(sql) % u_params ValueError: unsupported format character ''' (0x27) at index 309 I've tried it escaping and not escaping % wildcard but that doesn't solve the problem. Doesn't seem to affect it at all, really. Any ideas?

  Read the article

• Installing my sdist from PyPI puts the files in the wrong places

  - by Tartley

  Hey. My problem is that when I upload my Python package to PyPI, and then install it from there using pip, my app breaks because it installs my files into completely different locations than when I simply install the exact same package from a local sdist. Installing from the local sdist puts files on my system like this: /Python27/ Lib/ site-packages/ gloopy-0.1.alpha-py2.7.egg/ (egg and install info files) data/ (images and shader source) doc/ (html) examples/ (.py scripts that use the library) gloopy/ (source) This is much as I'd expect, and works fine (e.g. my source can find my data dir, because they lie next to each other, just like they do in development.) If I upload the same sdist to PyPI and then install it from there, using pip, then things look very different: /Python27/ data/ (images and shader source) doc/ (html) Lib/ site-packages/ gloopy-0.1.alpha-py2.7.egg/ (egg and install info files) gloopy/ (source files) examples/ (.py scripts that use the library) This doesn't work at all - my app can't find its data files, plus obviously it's a mess, polluting the top-level /python27 directory with all my junk. What am I doing wrong? How do I make the pip install behave like the local sdist install? Is that even what I should be trying to achieve? Details I have setuptools installed, and also distribute, and I'm calling distribute_setup.use_setuptools() WindowsXP, Python2.7. My development directory looks like this: /gloopy /data (image files and GLSL shader souce read at runtime) /doc (html files) /examples (some scripts to show off the library) /gloopy (the library itself) My MANIFEST.in mentions all the files I want to be included in the sdist, including everything in the data, examples and doc directories: recursive-include data *.* recursive-include examples *.py recursive-include doc/html *.html *.css *.js *.png include LICENSE.txt include TODO.txt My setup.py is quite verbose, but I guess the best thing is to include it here, right? I also includes duplicate references to the same data / doc / examples directories as are mentioned in the MANIFEST.in, because I understand this is required in order for these files to be copied from the sdist to the system during install. NAME = 'gloopy' VERSION= __import__(NAME).VERSION RELEASE = __import__(NAME).RELEASE SCRIPT = None CONSOLE = False def main(): import sys from pprint import pprint from setup_utils import distribute_setup from setup_utils.sdist_setup import get_sdist_config distribute_setup.use_setuptools() from setuptools import setup description, long_description = read_description() config = dict( name=name, version=version, description=description, long_description=long_description, keywords='', packages=find_packages(), data_files=[ ('examples', glob('examples/*.py')), ('data/shaders', glob('data/shaders/*.*')), ('doc', glob('doc/html/*.*')), ('doc/_images', glob('doc/html/_images/*.*')), ('doc/_modules', glob('doc/html/_modules/*.*')), ('doc/_modules/gloopy', glob('doc/html/_modules/gloopy/*.*')), ('doc/_modules/gloopy/geom', glob('doc/html/_modules/gloopy/geom/*.*')), ('doc/_modules/gloopy/move', glob('doc/html/_modules/gloopy/move/*.*')), ('doc/_modules/gloopy/shapes', glob('doc/html/_modules/gloopy/shapes/*.*')), ('doc/_modules/gloopy/util', glob('doc/html/_modules/gloopy/util/*.*')), ('doc/_modules/gloopy/view', glob('doc/html/_modules/gloopy/view/*.*')), ('doc/_static', glob('doc/html/_static/*.*')), ('doc/_api', glob('doc/html/_api/*.*')), ], classifiers=[ 'Development Status :: 1 - Planning', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: Microsoft :: Windows', 'Programming Language :: Python :: 2.7', ], # see classifiers http://pypi.python.org/pypi?:action=list_classifiers ) config.update(dict( author='Jonathan Hartley', author_email='[email protected]', url='http://bitbucket.org/tartley/gloopy', license='New BSD', ) ) if '--verbose' in sys.argv: pprint(config) setup(**config) if __name__ == '__main__': main()

  Read the article

• Efficient list compacting

  - by Patrik

  Suppose you have a list of unsigned ints. Suppose some elements are equal to 0 and you want to push them back. Currently I use this code (list is a pointer to a list of unsigned ints of size n for (i = 0; i < n; ++i) { if (list[i]) continue; int j; for (j = i + 1; j < n && !list[j]; ++j); int z; for (z = j + 1; z < n && list[z]; ++z); if (j == n) break; memmove(&(list[i]), &(list[j]), sizeof(unsigned int) * (z - j))); int s = z - j + i; for(j = s; j < z; ++j) list[j] = 0; i = s - 1; } Can you think of a more efficient way to perform this task? The snippet is purely theoretical, in the production code, each element of list is a 64 bytes struct EDIT: I'll post my solution. Many thanks to Jonathan Leffler. void RemoveDeadParticles(int * list, int * n) { int i, j = *n - 1; for (; j >= 0 && list[j] == 0; --j); for (i = 0; i < j; ++i) { if (list[i]) continue; memcpy(&(list[i]), &(list[j]), sizeof(int)); list[j] = 0; for (; j >= 0 && list[j] == 0; --j); if (i == j) break; } *n = i + 1; }

  Read the article

• Listing common SQL Code Smells.

  - by Phil Factor

  Once you’ve done a number of SQL Code-reviews, you’ll know those signs in the code that all might not be well. These ’Code Smells’ are coding styles that don’t directly cause a bug, but are indicators that all is not well with the code. . Kent Beck and Massimo Arnoldi seem to have coined the phrase in the "OnceAndOnlyOnce" page of www.C2.com, where Kent also said that code "wants to be simple". Bad Smells in Code was an essay by Kent Beck and Martin Fowler, published as Chapter 3 of the book ‘Refactoring: Improving the Design of Existing Code’ (ISBN 978-0201485677) Although there are generic code-smells, SQL has its own particular coding habits that will alert the programmer to the need to re-factor what has been written. See Exploring Smelly Code   and Code Deodorants for Code Smells by Nick Harrison for a grounding in Code Smells in C# I’ve always been tempted by the idea of automating a preliminary code-review for SQL. It would be so useful to trawl through code and pick up the various problems, much like the classic ‘Lint’ did for C, and how the Code Metrics plug-in for .NET Reflector by Jonathan 'Peli' de Halleux is used for finding Code Smells in .NET code. The problem is that few of the standard procedural code smells are relevant to SQL, and we need an agreed list of code smells. Merrilll Aldrich made a grand start last year in his blog Top 10 T-SQL Code Smells.However, I'd like to make a start by discovering if there is a general opinion amongst Database developers what the most important SQL Smells are. One can be a bit defensive about code smells. I will cheerfully write very long stored procedures, even though they are frowned on. I’ll use dynamic SQL occasionally. You can only use them as an aid for your own judgment and it is fine to ‘sign them off’ as being appropriate in particular circumstances. Also, whole classes of ‘code smells’ may be irrelevant for a particular database. The use of proprietary SQL, for example, is only a ‘code smell’ if there is a chance that the database will have to be ported to another RDBMS. The use of dynamic SQL is a risk only with certain security models. As the saying goes,  a CodeSmell is a hint of possible bad practice to a pragmatist, but a sure sign of bad practice to a purist. Plamen Ratchev’s wonderful article Ten Common SQL Programming Mistakes lists some of these ‘code smells’ along with out-and-out mistakes, but there are more. The use of nested transactions, for example, isn’t entirely incorrect, even though the database engine ignores all but the outermost: but it does flag up the possibility that the programmer thinks that nested transactions are supported. If anything requires some sort of general agreement, the definition of code smells is one. I’m therefore going to make this Blog ‘dynamic, in that, if anyone twitters a suggestion with a #SQLCodeSmells tag (or sends me a twitter) I’ll update the list here. If you add a comment to the blog with a suggestion of what should be added or removed, I’ll do my best to oblige. In other words, I’ll try to keep this blog up to date. The name against each 'smell' is the name of the person who Twittered me, commented about or who has written about the 'smell'. it does not imply that they were the first ever to think of the smell! Use of deprecated syntax such as *= (Dave Howard) Denormalisation that requires the shredding of the contents of columns. (Merrill Aldrich) Contrived interfaces Use of deprecated datatypes such as TEXT/NTEXT (Dave Howard) Datatype mis-matches in predicates that rely on implicit conversion.(Plamen Ratchev) Using Correlated subqueries instead of a join   (Dave_Levy/ Plamen Ratchev) The use of Hints in queries, especially NOLOCK (Dave Howard /Mike Reigler) Few or No comments. Use of functions in a WHERE clause. (Anil Das) Overuse of scalar UDFs (Dave Howard, Plamen Ratchev) Excessive ‘overloading’ of routines. The use of Exec xp_cmdShell (Merrill Aldrich) Excessive use of brackets. (Dave Levy) Lack of the use of a semicolon to terminate statements Use of non-SARGable functions on indexed columns in predicates (Plamen Ratchev) Duplicated code, or strikingly similar code. Misuse of SELECT * (Plamen Ratchev) Overuse of Cursors (Everyone. Special mention to Dave Levy & Adrian Hills) Overuse of CLR routines when not necessary (Sam Stange) Same column name in different tables with different datatypes. (Ian Stirk) Use of ‘broken’ functions such as ‘ISNUMERIC’ without additional checks. Excessive use of the WHILE loop (Merrill Aldrich) INSERT ... EXEC (Merrill Aldrich) The use of stored procedures where a view is sufficient (Merrill Aldrich) Not using two-part object names (Merrill Aldrich) Using INSERT INTO without specifying the columns and their order (Merrill Aldrich) Full outer joins even when they are not needed. (Plamen Ratchev) Huge stored procedures (hundreds/thousands of lines). Stored procedures that can produce different columns, or order of columns in their results, depending on the inputs. Code that is never used. Complex and nested conditionals WHILE (not done) loops without an error exit. Variable name same as the Datatype Vague identifiers. Storing complex data  or list in a character map, bitmap or XML field User procedures with sp_ prefix (Aaron Bertrand)Views that reference views that reference views that reference views (Aaron Bertrand) Inappropriate use of sql_variant (Neil Hambly) Errors with identity scope using SCOPE_IDENTITY @@IDENTITY or IDENT_CURRENT (Neil Hambly, Aaron Bertrand) Schemas that involve multiple dated copies of the same table instead of partitions (Matt Whitfield-Atlantis UK) Scalar UDFs that do data lookups (poor man's join) (Matt Whitfield-Atlantis UK) Code that allows SQL Injection (Mladen Prajdic) Tables without clustered indexes (Matt Whitfield-Atlantis UK) Use of "SELECT DISTINCT" to mask a join problem (Nick Harrison) Multiple stored procedures with nearly identical implementation. (Nick Harrison) Excessive column aliasing may point to a problem or it could be a mapping implementation. (Nick Harrison) Joining "too many" tables in a query. (Nick Harrison) Stored procedure returning more than one record set. (Nick Harrison) A NOT LIKE condition (Nick Harrison) excessive "OR" conditions. (Nick Harrison) User procedures with sp_ prefix (Aaron Bertrand) Views that reference views that reference views that reference views (Aaron Bertrand) sp_OACreate or anything related to it (Bill Fellows) Prefixing names with tbl_, vw_, fn_, and usp_ ('tibbling') (Jeremiah Peschka) Aliases that go a,b,c,d,e... (Dave Levy/Diane McNurlan) Overweight Queries (e.g. 4 inner joins, 8 left joins, 4 derived tables, 10 subqueries, 8 clustered GUIDs, 2 UDFs, 6 case statements = 1 query) (Robert L Davis) Order by 3,2 (Dave Levy) MultiStatement Table functions which are then filtered 'Sel * from Udf() where Udf.Col = Something' (Dave Ballantyne) running a SQL 2008 system in SQL 2000 compatibility mode(John Stafford)

  Read the article

• Ingredient Substitutes while Baking

  - by Rekha

  In our normal cooking, we substitute the vegetables for the gravies we prepare. When we start baking, we look for a good recipe. At least one or two ingredient will be missing. We do not know where to substitute what to bring same output. So we finally drop the plan of baking. Again after a month, we get the interest in baking. Again one or two lack of ingredient and that’s it. We keep on doing this for months. When I was going through the cooking blogs, I came across a site with the Ingredient Substitutes for Baking: (*) is to indicate that this substitution is ideal from personal experience. Flour Substitutes ( For 1 cup of Flour) All Purpose Flour 1/2 cup white cake flour plus 1/2 cup whole wheat flour 1 cup self-rising flour (omit using salt and baking powder if the recipe calls for it since self raising flour has it already) 1 cup plus 2 tablespoons cake flour 1/2 cup (75 grams) whole wheat flour 7/8 cup (130 grams) rice flour (starch) (do not replace all of the flour with the rice flour) 7/8 cup whole wheat Bread Flour 1 cup all purpose flour 1 cup all purpose flour plus 1 teaspoon wheat gluten (*) Cake Flour Place 2 tbsp cornstarch in 1 cup and fill the rest up with All Purpose flour (*) 1 cup all purpose flour minus 2 tablespoons Pastry flour Place 2 tbsp cornstarch in 1 cup and fill the rest up with All Purpose flour Equal parts of All purpose flour plus cake flour (*) Self-rising Flour 1½ teaspoons of baking powder plus ½ teaspoon of salt plus 1 cup of all-purpose flour. Cornstarch (1 tbsp) 2 tablespoons all-purpose flour 1 tablespoon arrowroot 4 teaspoons quick-cooking tapioca 1 tablespoon potato starch or rice starch or flour Tapioca (1 tbsp) 1 – 1/2 tablespoons all-purpose flour Cornmeal (stone ground) polenta OR corn flour (gives baked goods a lighter texture) if using cornmeal for breading,crush corn chips in a blender until they have the consistency of cornmeal. maize meal Corn grits Sweeteners ( for Every 1 cup ) * * (HV) denotes Healthy Version for low fat or fat free substitution in Baking Light Brown Sugar 2 tablespoons molasses plus 1 cup of white sugar Dark Brown Sugar 3 tablespoons molasses plus 1 cup of white sugar Confectioner’s/Powdered Sugar Process 1 cup sugar plus 1 tablespoon cornstarch Corn Syrup 1 cup sugar plus 1/4 cup water 1 cup Golden Syrup 1 cup honey (may be little sweeter) 1 cup molasses Golden Syrup Combine two parts light corn syrup plus one part molasses 1/2 cup honey plus 1/2 cup corn syrup 1 cup maple syrup 1 cup corn syrup Honey 1- 1/4 cups sugar plus 1/4 cup water 3/4 cup maple syrup plus 1/2 cup granulated sugar 3/4 cup corn syrup plus 1/2 cup granulated sugar 3/4 cup light molasses plus 1/2 cup granulated white sugar 1 1/4 cups granulated white or brown sugar plus 1/4 cup additional liquid in recipe plus 1/2 teaspoon cream of tartar Maple Syrup 1 cup honey,thinned with water or fruit juice like apple 3/4 cup corn syrup plus 1/4 cup butter 1 cup Brown Rice Syrup 1 cup Brown sugar (in case of cereals) 1 cup light molasses (on pancakes, cereals etc) 1 cup granulated sugar for every 3/4 cup of maple syrup and increase liquid in the recipe by 3 tbsp for every cup of sugar.If baking soda is used, decrease the amount by 1/4 teaspoon per cup of sugar substituted, since sugar is less acidic than maple syrup Molasses 1 cup honey 1 cup dark corn syrup 1 cup maple syrup 3/4 cup brown sugar warmed and dissolved in 1/4 cup of liquid ( use this if taste of molasses is important in the baked good) Cocoa Powder (Natural, Unsweetened) 3 tablespoons (20 grams) Dutch-processed cocoa plus 1/8 teaspoon cream of tartar, lemon juice or white vinegar 1 ounce (30 grams) unsweetened chocolate (reduce fat in recipe by 1 tablespoon) 3 tablespoons (20 grams) carob powder Semisweet baking chocolate (1 oz) 1 oz unsweetened baking chocolate plus 1 Tbsp sugar Unsweetened baking chocolate (1 oz ) 3 Tbsp baking cocoa plus 1 Tbsp vegetable oil or melted shortening or margarine Semisweet chocolate chips (1 cup) 6 oz semisweet baking chocolate, chopped (Alternatively) For 1 cup of Semi sweet chocolate chips you can use : 6 tablespoons unsweetened cocoa powder, 7 tablespoons sugar ,1/4 cup fat (butter or oil) Leaveners and Diary * * (HV) denotes Healthy Version for low fat or fat free substitution in Baking Compressed Yeast (1 cake) 1 envelope or 2 teaspoons active dry yeast 1 packet (1/4 ounce) Active Dry yeast 1 cake fresh compressed yeast 1 tablespoon fast-rising active yeast Baking Powder (1 tsp) 1/3 teaspoon baking soda plus 1/2 teaspoon cream of tartar 1/2 teaspoon baking soda plus 1/2 cup buttermilk or plain yogurt 1/4 teaspoon baking soda plus 1/3 cup molasses. When using the substitutions that include liquid, reduce other liquid in recipe accordingly Baking Soda(1 tsp) 3 tsp Baking Powder ( and reduce the acidic ingredients in the recipe. Ex Instead of buttermilk add milk) 1 tsp potassium bicarbonate Ideal substitution – 2 tsp Baking powder and omit salt in recipe Cream of tartar (1 tsp) 1 teaspoon white vinegar 1 tsp lemon juice Notes from What’s Cooking America – If cream of tartar is used along with baking soda in a cake or cookie recipe, omit both and use baking powder instead. If it calls for baking soda and cream of tarter, just use baking powder.Normally, when cream of tartar is used in a cookie, it is used together with baking soda. The two of them combined work like double-acting baking powder. When substituting for cream of tartar, you must also substitute for the baking soda. If your recipe calls for baking soda and cream of tarter, just use baking powder. One teaspoon baking powder is equivalent to 1/4 teaspoon baking soda plus 5/8 teaspoon cream of tartar. If there is additional baking soda that does not fit into the equation, simply add it to the batter. Buttermilk (1 cup) 1 tablespoon lemon juice or vinegar (white or cider) plus enough milk to make 1 cup (let stand 5-10 minutes) 1 cup plain or low fat yogurt 1 cup sour cream 1 cup water plus 1/4 cup buttermilk powder 1 cup milk plus 1 1/2 – 1 3/4 teaspoons cream of tartar Plain Yogurt (1 cup) 1 cup sour cream 1 cup buttermilk 1 cup crème fraiche 1 cup heavy whipping cream (35% butterfat) plus 1 tablespoon freshly squeezed lemon juice Whole Milk (1 cup) 1 cup fat free milk plus 1 tbsp unsaturated Oil like canola (HV) 1 cup low fat milk (HV) Heavy Cream (1 cup) 3/4 cup milk plus 1/3 cup melted butter.(whipping wont work) Sour Cream (1 cup) (pls refer also Substitutes for Fats in Baking below) 7/8 cup buttermilk or sour milk plus 3 tablespoons butter. 1 cup thickened yogurt plus 1 teaspoon baking soda. 3/4 cup sour milk plus 1/3 cup butter. 3/4 cup buttermilk plus 1/3 cup butter. Cooked sauces: 1 cup yogurt plus 1 tablespoon flour plus 2 teaspoons water. Cooked sauces: 1 cup evaporated milk plus 1 tablespoon vinegar or lemon juice. Let stand 5 minutes to thicken. Dips: 1 cup yogurt (drain through a cheesecloth-lined sieve for 30 minutes in the refrigerator for a thicker texture). Dips: 1 cup cottage cheese plus 1/4 cup yogurt or buttermilk, briefly whirled in a blender. Dips: 6 ounces cream cheese plus 3 tablespoons milk,briefly whirled in a blender. Lower fat: 1 cup low-fat cottage cheese plus 1 tablespoon lemon juice plus 2 tablespoons skim milk, whipped until smooth in a blender. Lower fat: 1 can chilled evaporated milk whipped with 1 teaspoon lemon juice. 1 cup plain yogurt plus 1 tablespoon cornstarch 1 cup plain nonfat yogurt Substitutes for Fats in Baking * * (HV) denoted Healthy Version for low fat or fat free substitution in Baking Butter (1 cup) 1 cup trans-free vegetable shortening 3/4 cups of vegetable oil (example. Canola oil) Fruit purees (example- applesauce, pureed prunes, baby-food fruits). Add it along with some vegetable oil and reduce any other sweeteners needed in the recipe since fruit purees are already sweet. 1 cup polyunsaturated margarine (HV) 3/4 cup polyunsaturated oil like safflower oil (HV) 1 cup mild olive oil (not extra virgin)(HV) Note: Butter creates the flakiness and the richness which an oil/purees cant provide. If you don’t want to compromise that much to taste, replace half the butter with the substitutions. Shortening(1 cup) 1 cup polyunsaturated margarine like Earth Balance or Smart Balance(HV) 1 cup + 2tbsp Butter ( better tasting than shortening but more expensive and has cholesterol and a higher level of saturated fat; makes cookies less crunchy, bread crusts more crispy) 1 cup + 2 tbsp Margarine (better tasting than shortening but more expensive; makes cookies less crunchy, bread crusts tougher) 1 Cup – 2tbsp Lard (Has cholesterol and a higher level of saturated fat) Oil equal amount of apple sauce stiffly beaten egg whites into batter equal parts mashed banana equal parts yogurt prune puree grated raw zucchini or seeds removed if cooked. Works well in quick breads/muffins/coffee cakes and does not alter taste pumpkin puree (if the recipe can handle the taste change) Low fat cottage cheese (use only half of the required fat in the recipe). Can give rubbery texture to the end result Silken Tofu – (use only half of the required fat in the recipe). Can give rubbery texture to the end result Equal parts of fruit juice Note: Fruit purees can alter the taste of the final product is used in large quantities. Cream Cheese (1 cup) 4 tbsps. margarine plus 1 cup low-fat cottage cheese – blended. Add few teaspoons of fat-free milk if needed (HV) Heavy Cream (1 cup) 1 cup evaporated skim milk (or full fat milk) 1/2 cup low fat Yogurt plus 1/2 low fat Cottage Cheese (HV) 1/2 cup Yogurt plus 1/2 Cottage Cheese Sour Cream (1 cup) 1 cup plain yogurt (HV) 3/4 cup buttermilk or plain yogurt plus 1/3 cup melted butter 1 cup crème fraiche 1 tablespoon lemon juice or vinegar plus enough whole milk to fill 1 cup (let stand 5-10 minutes) 1/2 cup low-fat cottage cheese plus 1/2 cup low-fat or nonfat yogurt (HV) 1 cup fat-free sour cream (HV) Note: How to Make Maple Syrup Substitute at home For 1 Cup Maple Syrup 1/2 cup granulated sugar 1 cup brown sugar, firmly packed 1 cup boiling water 1 teaspoon butter 1 teaspoon maple extract or vanilla extract Method In a heavy saucepan, place the granulated sugar and keep stirring until it melts and turns slightly brown. Alternatively in another pan, place brown sugar and water and bring to a boil without stirring. Now mix both the sugars and simmer in low heat until they come together as one thick syrup. Remove from heat, add butter and the extract. Use this in place of maple syrup. Store it in a fridge in an air tight container. Even though this was posted in their site long back, I found it helpful. So posting it for you. via chefinyou . cc image credit: flickr/zetrules

  Read the article

• Fragmented Log files could be slowing down your database

  - by Fatherjack

  Something that is sometimes forgotten by a lot of DBAs is the fact that database log files get fragmented in the same way that you get fragmentation in a data file. The cause is very different but the effect is the same – too much effort reading and writing data. Data files get fragmented as data is changed through normal system activity, INSERTs, UPDATEs and DELETEs cause fragmentation and most experienced DBAs are monitoring their indexes for fragmentation and dealing with it accordingly. However, you don’t hear about so many working on their log files. How can a log file get fragmented? I’m glad you asked. When you create a database there are at least two files created on the disk storage; an mdf for the data and an ldf for the log file (you can also have ndf files for extra data storage but that’s off topic for now). It is wholly possible to have more than one log file but in most cases there is little point in creating more than one as the log file is written to in a ‘wrap-around’ method (more on that later). When a log file is created at the time that a database is created the file is actually sub divided into a number of virtual log files (VLFs). The number and size of these VLFs depends on the size chosen for the log file. VLFs are also created in the space added to a log file when a log file growth event takes place. Do you have your log files set to auto grow? Then you have potentially been introducing many VLFs into your log file. Let’s get to see how many VLFs we have in a brand new database. USE master GO CREATE DATABASE VLF_Test ON ( NAME = VLF_Test, FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10.ROCK_2008\MSSQL\DATA\VLF_Test.mdf', SIZE = 100, MAXSIZE = 500, FILEGROWTH = 50 ) LOG ON ( NAME = VLF_Test_Log, FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10.ROCK_2008\MSSQL\DATA\VLF_Test_log.ldf', SIZE = 5MB, MAXSIZE = 250MB, FILEGROWTH = 5MB ); go USE VLF_Test go DBCC LOGINFO; The results of this are firstly a new database is created with specified files sizes and the the DBCC LOGINFO results are returned to the script editor. The DBCC LOGINFO results have plenty of interesting information in them but lets first note there are 4 rows of information, this relates to the fact that 4 VLFs have been created in the log file. The values in the FileSize column are the sizes of each VLF in bytes, you will see that the last one to be created is slightly larger than the others. So, a 5MB log file has 4 VLFs of roughly 1.25 MB. Lets alter the CREATE DATABASE script to create a log file that’s a bit bigger and see what happens. Alter the code above so that the log file details are replaced by LOG ON ( NAME = VLF_Test_Log, FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10.ROCK_2008\MSSQL\DATA\VLF_Test_log.ldf', SIZE = 1GB, MAXSIZE = 25GB, FILEGROWTH = 1GB ); With a bigger log file specified we get more VLFs What if we make it bigger again? LOG ON ( NAME = VLF_Test_Log, FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10.ROCK_2008\MSSQL\DATA\VLF_Test_log.ldf', SIZE = 5GB, MAXSIZE = 250GB, FILEGROWTH = 5GB ); This time we see more VLFs are created within our log file. We now have our 5GB log file comprised of 16 files of 320MB each. In fact these sizes fall into all the ranges that control the VLF creation criteria – what a coincidence! The rules that are followed when a log file is created or has it’s size increased are pretty basic. If the file growth is lower than 64MB then 4 VLFs are created If the growth is between 64MB and 1GB then 8 VLFs are created If the growth is greater than 1GB then 16 VLFs are created. Now the potential for chaos comes if the default values and settings for log file growth are used. By default a database log file gets a 1MB log file with unlimited growth in steps of 10%. The database we just created is 6 MB, let’s add some data and see what happens. USE vlf_test go -- we need somewhere to put the data so, a table is in order IF OBJECT_ID('A_Table') IS NOT NULL DROP TABLE A_Table go CREATE TABLE A_Table ( Col_A int IDENTITY, Col_B CHAR(8000) ) GO -- Let's check the state of the log file -- 4 VLFs found EXECUTE ('DBCC LOGINFO'); go -- We can go ahead and insert some data and then check the state of the log file again INSERT A_Table (col_b) SELECT TOP 500 REPLICATE('a',2000) FROM sys.columns AS sc, sys.columns AS sc2 GO -- insert 500 rows and we get 22 VLFs EXECUTE ('DBCC LOGINFO'); go -- Let's insert more rows INSERT A_Table (col_b) SELECT TOP 2000 REPLICATE('a',2000) FROM sys.columns AS sc, sys.columns AS sc2 GO 10 -- insert 2000 rows, in 10 batches and we suddenly have 107 VLFs EXECUTE ('DBCC LOGINFO'); Well, that escalated quickly! Our log file is split, internally, into 107 fragments after a few thousand inserts. The same happens with any logged transactions, I just chose to illustrate this with INSERTs. Having too many VLFs can cause performance degradation at times of database start up, log backup and log restore operations so it’s well worth keeping a check on this property. How do we prevent excessive VLF creation? Creating the database with larger files and also with larger growth steps and actively choosing to grow your databases rather than leaving it to the Auto Grow event can make sure that the growths are made with a size that is optimal. How do we resolve a situation of a database with too many VLFs? This process needs to be done when the database is under little or no stress so that you don’t affect system users. The steps are: BACKUP LOG YourDBName TO YourBackupDestinationOfChoice Shrink the log file to its smallest possible size DBCC SHRINKFILE(FileNameOfTLogHere, TRUNCATEONLY) * Re-size the log file to the size you want it to, taking in to account your expected needs for the coming months or year. ALTER DATABASE YourDBName MODIFY FILE ( NAME = FileNameOfTLogHere, SIZE = TheSizeYouWantItToBeIn_MB) * – If you don’t know the file name of your log file then run sp_helpfile while you are connected to the database that you want to work on and you will get the details you need. The resize step can take quite a while This is already detailed far better than I can explain it by Kimberley Tripp in her blog 8-Steps-to-better-Transaction-Log-throughput.aspx. The result of this will be a log file with a VLF count according to the bullet list above. Knowing when VLFs are being created By complete coincidence while I have been writing this blog (it’s been quite some time from it’s inception to going live) Jonathan Kehayias from SQLSkills.com has written a great article on how to track database file growth using Event Notifications and Service Broker. I strongly recommend taking a look at it as this is going to catch any sneaky auto grows that take place and let you know about them right away. Hassle free monitoring of VLFs If you are lucky or wise enough to be using SQL Monitor or another monitoring tool that let’s you write your own custom metrics then you can keep an eye on this very easily. There is a custom metric for VLFs (written by Stuart Ainsworth) already on the site and there are some others there are very useful so take a moment or two to look around while you are there. Resources MSDN – http://msdn.microsoft.com/en-us/library/ms179355(v=sql.105).aspx Kimberly Tripp from SQLSkills.com – http://www.sqlskills.com/BLOGS/KIMBERLY/post/8-Steps-to-better-Transaction-Log-throughput.aspx Thomas LaRock at Simple-Talk.com – http://www.simple-talk.com/sql/database-administration/monitoring-sql-server-virtual-log-file-fragmentation/ Disclosure I am a Friend of Red Gate. This means that I am more than likely to say good things about Red Gate DBA and Developer tools. No matter how awesome I make them sound, take the time to compare them with other products before you contact the Red Gate sales team to make your order.

  Read the article

• Try a sample: Using the counter predicate for event sampling

  - by extended_events

  Extended Events offers a rich filtering mechanism, called predicates, that allows you to reduce the number of events you collect by specifying criteria that will be applied during event collection. (You can find more information about predicates in Using SQL Server 2008 Extended Events (by Jonathan Kehayias)) By evaluating predicates early in the event firing sequence we can reduce the performance impact of collecting events by stopping event collection when the criteria are not met. You can specify predicates on both event fields and on a special object called a predicate source. Predicate sources are similar to action in that they typically are related to some type of global information available from the server. You will find that many of the actions available in Extended Events have equivalent predicate sources, but actions and predicates sources are not the same thing. Applying predicates, whether on a field or predicate source, is very similar to what you are used to in T-SQL in terms of how they work; you pick some field/source and compare it to a value, for example, session_id = 52. There is one predicate source that merits special attention though, not just for its special use, but for how the order of predicate evaluation impacts the behavior you see. I’m referring to the counter predicate source. The counter predicate source gives you a way to sample a subset of events that otherwise meet the criteria of the predicate; for example you could collect every other event, or only every tenth event. Simple CountingThe counter predicate source works by creating an in memory counter that increments every time the predicate statement is evaluated. Here is a simple example with my favorite event, sql_statement_completed, that only collects the second statement that is run. (OK, that’s not much of a sample, but this is for demonstration purposes. Here is the session definition: CREATE EVENT SESSION counter_test ON SERVERADD EVENT sqlserver.sql_statement_completed    (ACTION (sqlserver.sql_text)    WHERE package0.counter = 2)ADD TARGET package0.ring_bufferWITH (MAX_DISPATCH_LATENCY = 1 SECONDS) You can find general information about the session DDL syntax in BOL and from Pedro’s post Introduction to Extended Events. The important part here is the WHERE statement that defines that I only what the event where package0.count = 2; in other words, only the second instance of the event. Notice that I need to provide the package name along with the predicate source. You don’t need to provide the package name if you’re using event fields, only for predicate sources. Let’s say I run the following test queries: -- Run three statements to test the sessionSELECT 'This is the first statement'GOSELECT 'This is the second statement'GOSELECT 'This is the third statement';GO Once you return the event data from the ring buffer and parse the XML (see my earlier post on reading event data) you should see something like this: event_name sql_text sql_statement_completed SELECT ‘This is the second statement’ You can see that only the second statement from the test was actually collected. (Feel free to try this yourself. Check out what happens if you remove the WHERE statement from your session. Go ahead, I’ll wait.) Percentage Sampling OK, so that wasn’t particularly interesting, but you can probably see that this could be interesting, for example, lets say I need a 25% sample of the statements executed on my server for some type of QA analysis, that might be more interesting than just the second statement. All comparisons of predicates are handled using an object called a predicate comparator; the simple comparisons such as equals, greater than, etc. are mapped to the common mathematical symbols you know and love (eg. = and >), but to do the less common comparisons you will need to use the predicate comparators directly. You would probably look to the MOD operation to do this type sampling; we would too, but we don’t call it MOD, we call it divides_by_uint64. This comparator evaluates whether one number is divisible by another with no remainder. The general syntax for using a predicate comparator is pred_comp(field, value), field is always first and value is always second. So lets take a look at how the session changes to answer our new question of 25% sampling: CREATE EVENT SESSION counter_test_25 ON SERVERADD EVENT sqlserver.sql_statement_completed    (ACTION (sqlserver.sql_text)    WHERE package0.divides_by_uint64(package0.counter,4))ADD TARGET package0.ring_bufferWITH (MAX_DISPATCH_LATENCY = 1 SECONDS)GO Here I’ve replaced the simple equivalency check with the divides_by_uint64 comparator to check if the counter is evenly divisible by 4, which gives us back every fourth record. I’ll leave it as an exercise for the reader to test this session. Why order matters I indicated at the start of this post that order matters when it comes to the counter predicate – it does. Like most other predicate systems, Extended Events evaluates the predicate statement from left to right; as soon as the predicate statement is proven false we abandon evaluation of the remainder of the statement. The counter predicate source is only incremented when it is evaluated so whether or not the counter is incremented will depend on where it is in the predicate statement and whether a previous criteria made the predicate false or not. Here is a generic example: Pred1: (WHERE statement_1 AND package0.counter = 2)Pred2: (WHERE package0.counter = 2 AND statement_1) Let’s say I cause a number of events as follows and examine what happens to the counter predicate source. Iteration Statement Pred1 Counter Pred2 Counter A Not statement_1 0 1 B statement_1 1 2 C Not statement_1 1 3 D statement_1 2 4 As you can see, in the case of Pred1, statement_1 is evaluated first, when it fails (A & C) predicate evaluation is stopped and the counter is not incremented. With Pred2 the counter is evaluated first, so it is incremented on every iteration of the event and the remaining parts of the predicate are then evaluated. In this example, Pred1 would return an event for D while Pred2 would return an event for B. But wait, there is an interesting side-effect here; consider Pred2 if I had run my statements in the following order: Not statement_1 Not statement_1 statement_1 statement_1 In this case I would never get an event back from the system because the point at which counter=2, the rest of the predicate evaluates as false so the event is not returned. If you’re using the counter target for sampling and you’re not getting the expected events, or any events, check the order of the predicate criteria. As a general rule I’d suggest that the counter criteria should be the last element of your predicate statement since that will assure that your sampling rate will apply to the set of event records defined by the rest of your predicate. Aside: I’m interested in hearing about uses for putting the counter predicate criteria earlier in the predicate statement. If you have one, post it in a comment to share with the class. - Mike Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

  Read the article

• Workshops, online content show how Oracle infuses simplicity, mobility, extensibility into user experience

  - by mvaughan

  By Kathy Miedema & Misha Vaughan, Oracle Applications User Experience Oracle has made a huge investment into the user experience of its many different software product families, and recent releases showcase big changes and features that aim to promote end user engagement and efficiency by streamlining navigation and simplifying the user interface. But making Oracle’s enterprise software great-looking and usable doesn’t stop when Oracle products go out the door. The Applications User Experience (UX) team recognizes that our customers may need to customize software to fit their work processes. And that’s why we provide tools such as user experience design patterns to help you maintain the Oracle user experience as you tailor your application to fit your business needs. Often, however, customers may need some context around user experience. How has the Oracle user experience been designed and constructed? Why is a good user experience important for users? How does understanding what goes into the user experience benefit the people who purchase the software for users? There’s a short answer to these questions, and you can read about it on Usable Apps. But truly understanding Oracle’s investment and seeing how it applies across product families occasionally requires a deeper dive into the Oracle user experience, especially if you’re an influencer or decision-maker about Oracle products. To help frame these decisions, the Communications & Outreach team has developed several targeted workshops that explore what Oracle means when it talks about user experience, and provides a roadmap into where the Oracle user experience is going. These workshops require non-disclosure agreements, and have been delivered to Oracle sales folks, Oracle partners, Oracle ACE Directors and ACEs, and a few customers. Some of these audience members have been developers or have a technical background; just as many did not. Here’s a breakdown of the kind of training you can get around the Oracle user experience from the OAUX Communications & Outreach team.For Partners: George Papazzian, Principal, Naviscent with Joyce Ohgi, Oracle Oracle Fusion Applications HCM Pre-Sales Seminar:  In concert with Worldwide Alliances  and  Channels under Applications Partner Enablement Director Jonathan Vinoskey’s guidance, the Applications User Experience team delivers a two-day workshop.  Day one focuses on Oracle Fusion Applications HCM and pre-sales strategy, and Day two focuses on positioning and leveraging Oracle’s investment in the Oracle Fusion Applications user experience.  The next workshops will occur on the following dates: December 4-5, 2013 @ Manchester, UK January 29-30, 2014 @ Reston, Virginia February 2014 @ Guadalajara, Mexico (email: Shannon Whiteman) March 11-12, 2014 @ Dubai, United Arab Emirates April 1-2, 2014 @ Chicago, Illinois Partner Advisory Board: A two-day board meeting in the U.S. and U.K. to discuss four main user experience areas for Oracle Fusion Applications: simplicity, visualization & analytics, mobility, & futures. This event is limited to Oracle Diamond Partners, UX bloggers, and key UX influencers and requires legal documentation.  We will be talking about the Oracle applications UX strategy and roadmap. Partner Implementation Training on User Interface: How to Build Great-Looking, Usable Apps:  In this two-day, hands-on workshop built around Oracle’s Application Development Framework, learn how to build desktop and mobile user interfaces and mobile user interfaces based on Oracle’s experience with Fusion Applications. This workshop is for partners with a technology background who are looking for ways to tailor Fusion Applications using ADF, or have built their own custom solutions using ADF. It includes an introduction to UX design patterns and provides tools to build usability-tested UX designs. Nov 5-6, 2013 @ Redwood Shores, CA, USA January 28-29th, 2014 @ Reston, Virginia, USA February 25-26, 2014 @ Guadalajara, Mexico March 9-10, 2014 @ Dubai, United Arab Emirates To register, contact [email protected] Simplified UI Customization & Extensibility:  Pilot workshop:  We will be reviewing the proposed content for communicating the user experience tool kit available with the next release of Oracle Fusion Applications.  Our core focus will be on what toolkit components our system implementors and independent software vendors will need to respond to customer demand, whether they are extending Fusion Applications, or building custom applications, that will need to leverage the simplified UI. Dec 11th, 2013 @ Reading, UK For information: contact [email protected] Private lab tour and demos: Interested in seeing what’s going on in the Apps UX Labs?  If you are headed to the San Francisco Bay Area, let us know. We can arrange a spin through our usability labs at headquarters. OAUX Expo: This open-house forum gives partners a look at what the UX team is working on, and showcases the next-generation user experiences in a demo environment where attendees can see and touch the applications. UX Direct: Use the same methods that Oracle uses to develop its own user experiences. We help you define your users and their needs, and then provide direction on how to tailor the best user experience you can for them. For CustomersAngela Johnston, Gozel Aamoth, Teena Singh, and Yen Chan, Oracle Lab tours: See demos of soon-to-be-released products, and take a spin on usability research equipment such as our eye-tracker. Watch this video to get an idea of what you’ll see. Get our newsletter: Learn about newly released products and see where you can meet us at user group conferences. Participate in a feedback session: Join a focus group or customer feedback session to get an early look at user experience designs for the next generation of software, and provide your thoughts on how well it will work. Join the OUAB: The Oracle Usability Advisory Board meets several times a year to discuss trends in the workforce and provide direction on user experience designs. UX Direct: Use the same methods that Oracle uses to develop its own user experiences. We help you define your users and their needs, and then provide direction on how to tailor the best user experience you can for them. For Developers (customers, partners, and consultants): Plinio Arbizu, SP Solutions, Richard Bingham, Oracle, Balaji Kamepalli, EiSTechnoogies, Praveen Pillalamarri, EiSTechnologies How to Build Great-Looking, Usable Apps: This workshop is for attendees with a strong technology background who are looking for ways to tailor customer software using ADF. It includes an introduction to UX design patterns and provides tools to build usability-tested UX designs.  See above for dates and times. UX design patterns web site: Cut the length of your project down by months. Use these patterns to build out the task flow you need to develop for your users. The patterns have already been usability-tested and represent the best practices that the Oracle UX research team has found in its studies. UX Direct: Use the same methods that Oracle uses to develop its own user experiences. We help you define your users and their needs, and then provide direction on how to tailor the best user experience you can for them. For Oracle Sales Mike Klein, Jeremy Ashley, Brent White, Oracle Contact your local sales person for more information about the Oracle user experience and the training available from the Applications User Experience Communications & Outreach team. See customer-friendly user experience collateral ranging from the new simplified UI in Oracle Fusion Applications Release 7, to E-Business Suite user experience highlights, to Siebel, PeopleSoft, and JD Edwards user experience highlights.   Receive access to the same pre-sales and implementation training we provide to partners. For Oracle Sales only: Oracle-only training on the Oracle Fusion Applications UX Innovation Sales Kit.

  Read the article

• Agile Testing Days 2012 – Day 2 – Learn through disagreement

  - by Chris George

  I think I was in the right place! During Day 1 I kept on reading tweets about Lean Coffee that has happened earlier that morning. It intrigued me and I figured in for a penny in for a pound, and set my alarm for 6:45am. Following the award night the night before, it was _really_ hard getting up when it went off, but I did and after a very early breakfast, set off for the 10 min walk to the Dorint. With Lean Coffee due to start at 07:30, I arrived at the hotel and made my way to one of the hotel bars. I soon realised I was in the right place as although the bar was empty, there was a table with post-it’s and pens! This MUST be the place! The premise of Lean Coffee is to have several small timeboxed discussions. Everyone writes down what they would like to discuss on post-its that are then briefly explained and submitted to the pile. Once everyone is done, the group dot-votes on the topics. The topics are then sorted by the dot vote counts and the discussions begin. Each discussion had 8 mins to start with, which meant it prevented the discussions getting off topic too much. After the time elapsed, the group had a vote whether to extend the discussion by a further 4 mins or move on. Several discussion were had around training, soft skills etc. The conversations were really interesting and there were quite a few good ideas. Overall it was a very enjoyable experience, certainly worth the early start! Make Melly Happy Following Lean Coffee was real coffee, and much needed that was! The first keynote of the day was “Let’s help Melly (Changing Work into Life)”by Jurgen Appelo. Draw lines to track happiness This was a very interesting presentation, and set the day nicely. The theme to the keynote was projects are about the people, more-so than the actual tasks. So he started by showing a photo of an employee ‘Melly’ who looked happy enough. He then stated that she looked happy but actually hated her job. In fact 50% of Americans hate their jobs. He went on to say that the world over 50% of people hate Americans their jobs. Jurgen talked about many ways to reduce the feedback cycle, not only of the project, but of the people management. Ideas such as Happiness doors, happiness tracking (drawing lines on a wall indicating your happiness for that day), kudo boxes (to compliment a colleague for good work). All of these (and more) ideas stimulate conversation amongst the team, lead to early detection of issues and investigation of solutions. I’ve massively simplified Jurgen’s keynote and have certainly not done it justice, so I will post a link to the video once it’s available. Following more coffee, the next talk was “How releasing faster changes testing” by Alexander Schwartz. This is a topic very close to our hearts at the moment, so I was eager to find out any juicy morsels that could help us achieve more frequent releases, and Alex did not disappoint. He started off by confirming something that I have been a firm believer in for a number of years now; adding more people can do more harm than good when trying to release. This is for a number of reasons, but just adding new people to a team at such a critical time can be more of a drain on resources than they add. The alternative is to have the whole team have shared responsibility for faster delivery. So the whole team is responsible for quality and testing. Obviously you will have the test engineers on the project who have the specialist skills, but there is no reason that the entire team cannot do exploratory testing on the product. This links nicely with the Developer Exploratory testing presented by Sigge on Day 1, and certainly something that my team are really striving towards. Focus on cycle time, so what can be done to reduce the time between dev cycles, release cycles. What’s stops a release, what delays a release? all good solid questions that can be answered. Alex suggested that perhaps the product doesn’t need to be fully tested. Doing less testing will reduce the cycle time therefore get the release out faster. He suggested a risk-based approach to planning what testing needs to happen. Reducing testing could have an impact on revenue if it causes harm to customers, so test the ‘right stuff’! Determine a set of tests that are ‘face saving’ or ‘smoke’ tests. These tests cover the core functionality of the product and aim to prevent major embarrassment if these areas were to fail! Amongst many other very good points, Alex suggested that a good approach would be to release after every new feature is added. So do a bit of work -> release, do some more work -> release. By releasing small increments of work, the impact on the customer of bugs being introduced is reduced. Red Pill, Blue Pill The second keynote of the day was “Adaptation and improvisation – but your weakness is not your technique” by Markus Gartner and proved to be another very good presentation. It started off quoting lines from the Matrix which relate to adapting, improvising, realisation and mastery. It has alot of nerds in the room smiling! Markus went on to explain how through deliberate practice ( and a lot of it!) you can achieve mastery, but then you never stop learning. Through methods such as code retreats, testing dojos, workshops you can continually improve and learn. The code retreat idea was one that interested me. It involved pairing to write an automated test for, say, 45 mins, they deleting all the code, finding a different partner and writing the same test again! This is another keynote where the video will speak louder than anything I can write here! Markus did elaborate on something that Lisa and Janet had touched on yesterday whilst busting the myth that “Testers Must Code”. Whilst it is true that to be a tester, you don’t need to code, it is becoming more common that there is this crossover happening where more testers are coding and more programmers are testing. Markus made a special distinction between programmers and developers as testers develop tests code so this helped to make that clear. “Extending Continuous Integration and TDD with Continuous Testing” by Jason Ayers was my next talk after lunch. We already do CI and a bit of TDD on my project team so I was interested to see what this continuous testing thing was all about and whether it would actually work for us. At the start of the presentation I was of the opinion that it just would not work for us because our tests are too slow, and that would be the case for many people. Jason started off by setting the scene and saying that those doing TDD spend between 10-15% of their time waiting for tests to run. This can be reduced by testing less often, reducing the test time but this then increases the risk of introduced bugs not being spotted quickly. Therefore, in comes Continuous Testing (CT). CT systems run your unit tests whenever you save some code and runs them in the background so you can continue working. This is a really nice idea, but to do this, your tests must be fast, independent and reliable. The latter two should be the case anyway, and the first is ideal, but hard! Jason makes several suggestions to make tests fast. Firstly keep the scope of the test small, secondly spin off any expensive tests into a suite which is run, perhaps, overnight or outside of the CT system at any rate. So this started to change my mind, perhaps we could re-engineer our tests, and continuously run the quick ones to give an element of coverage. This talk was very interesting and I’ve already tried a couple of the tools mentioned on our product (Mighty Moose and NCrunch). Sadly due to the way our solution is built, it currently doesn’t work, but we will look at whether we can make this work because this has the potential to be a mini-game-changer for us. Using the wrong data Gojko’s Hierarchy of Quality The final keynote of the day was “Reinventing software quality” by Gojko Adzic. He opened the talk with the statement “We’ve got quality wrong because we are using the wrong data”! Gojko then went on to explain that we should judge a bug by whether the customer cares about it, not by whether we think it’s important. Why spend time fixing issues that the customer just wouldn’t care about and releasing months later because of this? Surely it’s better to release now and get customer feedback? This was another reference to the idea of how it’s better to build the right thing wrong than the wrong thing right. Get feedback early to make sure you’re making the right thing. Gojko then showed something which was very analogous to Maslow’s heirachy of needs. Successful – does it contribute to the business? Useful – does it do what the user wants Usable – does it do what it’s supposed to without breaking Performant/Secure – is it secure/is the performance acceptable Deployable Functionally ok – can it be deployed without breaking? He then explained that User Stories should focus on change. In other words they should focus on the users needs, not the users process. Describe what the change will be, how that change will happen then measure it! Networking and Beer Following the day’s closing keynote, there were drinks and nibble for the ‘Networking’ evening. This was a great opportunity to talk to people. I find approaching strangers very uncomfortable but once again, when in Rome! Pete Walen and I had a long conversation about only fixing issues that the customer cares about versus fixing issues that make you proud of your software! Without saying much, and asking the right questions, Pete made me re-evaluate my thoughts on the matter. Clever, very clever!  Oh and he ‘bought’ me a beer! My Takeaway Triple from Day 2: release small and release often to minimize issues creeping in and get faster feedback from ‘the real world’ Focus on issues that the customers care about, not what we think is important It’s okay to disagree with someone, even if they are well respected agile testing gurus, that’s how discussion and learning happens!  

  Read the article

< Previous Page | 37 38 39 40 41 42  | Next Page >