Search Results

Search found 65558 results on 2623 pages for 'large data'.

Page 482/2623 | < Previous Page | 478 479 480 481 482 483 484 485 486 487 488 489  | Next Page >

  • Is there a way to combine streaming data retrieval with hibernate?

    - by Steve B.
    For the purposes of handling very large collections (and by very large I just mean "likely to throw OutOfMemory exception"), it seems problematic to use Hibernate because normally collection retrieval is done in a block, i.e. List values=session.createQuery("from X").list(), where you monolithically grab all N-million values and then process them. What I'd prefer to do is to retrieve the values as an iterator so that I grab 1000 or so (or whatever's a reasonable page size) at a time. Apart from writing my own iteration (which seems like it's likely to be re-inventing the wheel) is there a hibernate-native way to handle this?

    Read the article

  • Is is possible to parse a web page from the client side for a large number of words and if so, how?

    - by Technoh
    I have a list of keywords, about 25,000 of them. I would like people who add a certain < script tag on their web page to have these keywords transformed into links. What would be the best way to go and achieve this? I have tried the simple javascript approach (an array with lots of elements and regexping/replacing each) and it obviously slows down the browser. I could always process the content server-side if there was a way, from the client, to send the page's content to a cross-domain server script (I'm partial to PHP but it could be anything) but I don't know of any way to do this. Any other working solution is also welcome.

    Read the article

  • CoreGraphics taking a while to show on a large view - can i get it to repeat pixels?

    - by Andrew
    This is my coregraphics code: void drawTopPaperBackground(CGContextRef context, CGRect rect) { CGRect paper3 = CGRectMake(10, 14, 300, rect.size.height - 14); CGRect paper2 = CGRectMake(13, 12, 294, rect.size.height - 12); CGRect paper1 = CGRectMake(16, 10, 288, rect.size.height - 10); //Shadow CGContextSetShadowWithColor(context, CGSizeMake(0,0), 10, [[UIColor colorWithWhite:0 alpha:0.5]CGColor]); CGPathRef path = createRoundedRectForRect(paper3, 0); CGContextSetFillColorWithColor(context, [[UIColor blackColor] CGColor]); CGContextAddPath(context, path); CGContextFillPath(context); //Layers of paper //CGContextSaveGState(context); drawPaper(context, paper3); drawPaper(context, paper2); drawPaper(context, paper1); //CGContextRestoreGState(context); } void drawPaper(CGContextRef context, CGRect rect) { //Shadow CGContextSaveGState(context); CGContextSetShadowWithColor(context, CGSizeMake(0,0), 1, [[UIColor colorWithWhite:0 alpha:0.5]CGColor]); CGPathRef path = createRoundedRectForRect(rect, 0); CGContextSetFillColorWithColor(context, [[UIColor blackColor] CGColor]); CGContextAddPath(context, path); CGContextFillPath(context); //CGContextRestoreGState(context); //Gradient //CGContextSaveGState(context); CGColorRef startColor = [UIColor colorWithWhite:0.92 alpha:1.0].CGColor; CGColorRef endColor = [UIColor colorWithWhite:0.94 alpha:1.0].CGColor; CGRect firstHalf = CGRectMake(rect.origin.x, rect.origin.y, rect.size.width / 2, rect.size.height); CGRect secondHalf = CGRectMake(rect.origin.x + (rect.size.width / 2), rect.origin.y, rect.size.width / 2, rect.size.height); drawVerticalGradient(context, firstHalf, startColor, endColor); drawVerticalGradient(context, secondHalf, endColor, startColor); //CGContextRestoreGState(context); //CGContextSaveGState(context); CGRect redRect = rectForRectWithInset(rect, -1); CGMutablePathRef redPath = createRoundedRectForRect(redRect, 0); //CGContextSaveGState(context); CGContextSetStrokeColorWithColor(context, [[UIColor blackColor] CGColor]); CGContextAddPath(context, path); CGContextClip(context); CGContextAddPath(context, redPath); CGContextSetShadowWithColor(context, CGSizeMake(0, 0), 15.0, [[UIColor colorWithWhite:0 alpha:0.1] CGColor]); CGContextStrokePath(context); CGContextRestoreGState(context); } The view is a UIScrollView, which contains a textview. Every time the user types something and goes onto a new line, I call [self setNeedsDisplay]; and it redraws the code. But when the view starts to get long - around 1000 height, it has very noticeable lag. How can i make this code more efficient? Can i take a line of pixels and make it just repeat that, or stretch it, all the way down?

    Read the article

  • How to display data into datagridview using multi thread?

    - by Mark
    Hi, I have application where I read/receive data all the time (text) and I need to display this data into datagridview, what is the best way to do that in real time, so the data will be changed all the time. I thought about multi threading, if this is a good idea can you guide me with link to explain how to implement it. Thanks

    Read the article

  • Retrieving data from database. Retrieve only when needed or get everything?

    - by RHaguiuda
    I have a simple application to store Contacts. This application uses a simple relational database to store Contact information, like Name, Address and other data fields. While designing it, I question came to my mind: When designing programs that uses databases, should I retrieve all database records and store them in objects in my program, so I have a very fast performance or I should always gather data only when required? Of course, retrieving all data can only be done if it`s not too many, but do you use this approach when you make sure that the database will be small (< 300 records for example)? I have designed once a similar application that fetches data only when needed, but that was slow (using a Access database). Thanks for all help.

    Read the article

  • Setting up SVN (subvsersion) to manage our companies files, how to exclude large files from being ve

    - by Roeland
    Me and two other guys recently started our own web development company. We each work from our homes and have decided we want to keep one central location for all of our files. These files include word documents, spreadsheets, client files, designs.. etc. Anything pertaining to our company. I have a pretty solid internet connection and a windows 2008 server box sitting at home so I set up a subversion repository. Our file repository will look something like this. Clients Company A Design (photoshop files, wireframes, concepts) Documents ( logins, quotes, proposals etc) Site Backups Company B Design Documents Site Backups Prospects Company C Company D Our Company Our Website Documents (contract, operating procudres) My question is in regards to design files. The photoshop files that my designer works with range in sizes from 10mb to 100mb. I don't think we need to keep these files version-ed as this would eat up space incredibly fast. How do I go about controlling which files get version-ed, and which files are just stored. What I am thinking is that all documents need to be version-ed, and any files other then that should not be. Any help would be appreciated, thanks! Edit I am also curious whether this is the way to go. I just like this system since it keeps version of all my documents and at the same time. Also essentially I will have 3 backups in 3 different locations (3 local copies) so no need for backing it up. I am unsure of how svn would perform as purely a huge file repository.

    Read the article

  • Does anybody have any suggestions on which of these two approaches is better for large delete?

    - by RPS
    Approach #1: DECLARE @count int SET @count = 2000 DECLARE @rowcount int SET @rowcount = @count WHILE @rowcount = @count BEGIN DELETE TOP (@count) FROM ProductOrderInfo WHERE ProductId = @product_id AND bCopied = 1 AND FileNameCRC = @localNameCrc SELECT @rowcount = @@ROWCOUNT WAITFOR DELAY '000:00:00.400' Approach #2: DECLARE @count int SET @count = 2000 DECLARE @rowcount int SET @rowcount = @count WHILE @rowcount = @count BEGIN DELETE FROM ProductOrderInfo WHERE ProductId = @product_id AND FileNameCRC IN ( SELECT TOP(@count) FileNameCRC FROM ProductOrderInfo WITH (NOLOCK) WHERE bCopied = 1 AND FileNameCRC = @localNameCrc ) SELECT @rowcount = @@ROWCOUNT WAITFOR DELAY '000:00:00.400' END

    Read the article

  • using spring, hibernate and scala, is there a better way to load test data than dbunit?

    - by egervari
    Here are some things I really dislike about dbunit: 1) You cannot specify the exact ordering the inserts because dbunit likes to group your inserts by table name, and not by the order you define them in the XML file. This is a problem when you have records depending on other records in other tables, so you have to disable foreign key constraints during your tests... which actually sucks because these foreign key constraints will get fired in production while your tests won't be aware of them! 2) They seem hellbent on forcing you to use an xml namespace to define your xml... and I honestly can't be bothered to do this. I like the data.xml without any namespace. It works. But they are so hellbent on deprecating it. 3) Creating different xml files is hard on a per test basis, so it actually encourages creating data for your entire app. Unfortunately, this process is a little bloated too once the data grows in size and things get inter tangled. There has got to be a better way to split up your test data into chunks without having to copy/paste a lot of the test data across all of your tests. 4) Keeping track of id references in a big xml file is just impossible. If you have 130 domain classes, it just gets bewildering. This model simply does not scale. Is there something less bloated and better in the Spring/Hibernate space? db unit has worn out its welcome and I'm really looking for something better.

    Read the article

  • How can I get a iterable resultset from the database using pdo, instead of a large array?

    - by Tchalvak
    I'm using PDO inside a database abstraction library function query. I'm using fetchAll(), which if you have a lot of results, can get memory intensive, so I want to provide an argument to toggle between a fetchAll associative array and a pdo result set that can be iterated over with foreach and requires less memory (somehow). I remember hearing about this, and I searched the PDO docs, but I couldn't find any useful way to do that. Does anyone know how to get an iterable resultset back from PDO instead of just a flat array? And am I right that using an iterable resultset will be easier on memory? I'm using Postgresql, if it matters in this case. . . . The current query function is as follows, just for clarity. /** * Running bound queries on the database. * * Use: query('select all from players limit :count', array('count'=>10)); * Or: query('select all from players limit :count', array('count'=>array(10, PDO::PARAM_INT))); **/ function query($sql_query, $bindings=array()){ DatabaseConnection::getInstance(); $statement = DatabaseConnection::$pdo->prepare($sql_query); foreach($bindings as $binding => $value){ if(is_array($value)){ $statement->bindParam($binding, $value[0], $value[1]); } else { $statement->bindValue($binding, $value); } } $statement->execute(); // TODO: Return an iterable resultset here, and allow switching between array and iterable resultset. return $statement->fetchAll(PDO::FETCH_ASSOC); }

    Read the article

  • Is there a performance gain from defining routes in app.yaml versus one large mapping in a WSGIAppli

    - by jgeewax
    Scenario 1 This involves using one "gateway" route in app.yaml and then choosing the RequestHandler in the WSGIApplication. app.yaml - url: /.* script: main.py main.py from google.appengine.ext import webapp class Page1(webapp.RequestHandler): def get(self): self.response.out.write("Page 1") class Page2(webapp.RequestHandler): def get(self): self.response.out.write("Page 2") application = webapp.WSGIApplication([ ('/page1/', Page1), ('/page2/', Page2), ], debug=True) def main(): wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main() Scenario 2: This involves defining two routes in app.yaml and then two separate scripts for each (page1.py and page2.py). app.yaml - url: /page1/ script: page1.py - url: /page2/ script: page2.py page1.py from google.appengine.ext import webapp class Page1(webapp.RequestHandler): def get(self): self.response.out.write("Page 1") application = webapp.WSGIApplication([ ('/page1/', Page1), ], debug=True) def main(): wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main() page2.py from google.appengine.ext import webapp class Page2(webapp.RequestHandler): def get(self): self.response.out.write("Page 2") application = webapp.WSGIApplication([ ('/page2/', Page2), ], debug=True) def main(): wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main() Question What are the benefits and drawbacks of each pattern? Is one much faster than the other?

    Read the article

  • Flex Tree with infinite parents and children

    - by Tempname
    I am working on a tree component and I am having a bit of the issue with populating the data-provider for this tree. The data that I get back from my database is a simple array of value objects. Each value object has 2 properties. ObjectID and ParentID. For parents the ParentID is null and for children the ParentID is the ObjectID of the parent. Any help with this is greatly appreciated. Essentially the tree should look something like this: Parent1 Child1 Child1 Child2 Child1 Child2 Parent2 Child1 Child2 Child3 Child1 This is the current code that I am testing with: public function setDataProvider(data:Array):void { var tree:Array = new Array(); for(var i:Number = 0; i < data.length; i++) { // do the top level array if(!data[i].parentID) { tree.push(data[i], getChildren(data[i].objectID, data)); } } function getChildren(objectID:Number, data:Array):Array { var childArr:Array = new Array(); for(var k:Number = 0; k < data.length; k++) { if(data[k].parentID == objectID) { childArr.push(data[k]); //getChildren(data[k].objectID, data); } } return childArr; } trace(ObjectUtil.toString(tree)); } Here is a cross section of my data: ObjectID ParentID 1 NULL 10 NULL 8 NULL 6 NULL 4 6 3 6 9 6 2 6 11 7 7 8 5 8

    Read the article

  • How do you specify a really large character in UIButton?

    - by Epsilon Prime
    I have a series of buttons that have suit symbols on them. Currently I provide these suit symbols as bitmaps. In preparation for iPhone 4 I'd like to use text instead. However Interface Builder rescales the button to account for whitespace underneath the symbol so I can't get the image to fill the button completely. Any hints on getting Interface Builder to behave?

    Read the article

  • What data structure would be the least painful DataTable replacement?

    - by MatthewMartin
    I'm storing a lot of sorted ~10 row 2 column/key value pairs in ASP.NET cache-- they're the data for dropdownlists. Right now they are all DataTables, which isn't very space efficient (the rule of thumb is 10x increase in size when data is strored in a dataset). Old Code DataTable table = dataAccess.GetDataTable(); dropDownList.DataSource = table; Hoped for new Code Unknown data = dataAccess.GetSomethingMoreSpaceEfficient(); dropDownList.DataSource = data; What pre-existing datastructures are similar enough to DataTable that would minimize code breakage and reduce the serialized size when stored in ASP.NET cache?

    Read the article

  • Combining FileStream and MemoryStream to avoid disk accesses/paging while receiving gigabytes of data?

    - by w128
    I'm receiving a file as a stream of byte[] data packets (total size isn't known in advance) that I need to store somewhere before processing it immediately after it's been received (I can't do the processing on the fly). Total received file size can vary from as small as 10 KB to over 4 GB. One option for storing the received data is to use a MemoryStream, i.e. a sequence of MemoryStream.Write(bufferReceived, 0, count) calls to store the received packets. This is very simple, but obviously will result in out of memory exception for large files. An alternative option is to use a FileStream, i.e. FileStream.Write(bufferReceived, 0, count). This way, no out of memory exceptions will occur, but what I'm unsure about is bad performance due to disk writes (which I don't want to occur as long as plenty of memory is still available) - I'd like to avoid disk access as much as possible, but I don't know of a way to control this. I did some testing and most of the time, there seems to be little performance difference between say 10 000 consecutive calls of MemoryStream.Write() vs FileStream.Write(), but a lot seems to depend on buffer size and the total amount of data in question (i.e the number of writes). Obviously, MemoryStream size reallocation is also a factor. Does it make sense to use a combination of MemoryStream and FileStream, i.e. write to memory stream by default, but once the total amount of data received is over e.g. 500 MB, write it to FileStream; then, read in chunks from both streams for processing the received data (first process 500 MB from the MemoryStream, dispose it, then read from FileStream)? Another solution is to use a custom memory stream implementation that doesn't require continuous address space for internal array allocation (i.e. a linked list of memory streams); this way, at least on 64-bit environments, out of memory exceptions should no longer be an issue. Con: extra work, more room for mistakes. So how do FileStream vs MemoryStream read/writes behave in terms of disk access and memory caching, i.e. data size/performance balance. I would expect that as long as enough RAM is available, FileStream would internally read/write from memory (cache) anyway, and virtual memory would take care of the rest. But I don't know how often FileStream will explicitly access a disk when being written to. Any help would be appreciated.

    Read the article

  • What's the most scalable way to handle somewhat large file uploads in a Python webapp?

    - by Jason Baker
    We have a web application that takes file uploads for some parts. The file uploads aren't terribly big (mostly word documents and such), but they're much larger than your typical web request and they tend to tie up our threaded servers (zope 2 servers running behind an Apache proxy). I'm mostly in the brainstorming phase right now and trying to figure out a general technique to use. Some ideas I have are: Using a python asynchronous server like tornado or diesel or gunicorn. Writing something in twisted to handle it. Just using nginx to handle the actual file uploads. It's surprisingly difficult to find information on which approach I should be taking. I'm sure there are plenty of details that would be needed to make an actual decision, but I'm more worried about figuring out how to make this decision than anything else. Can anyone give me some advice about how to proceed with this?

    Read the article

  • Exhaustive (or even just large) list of languages/stacks used for popular sites?

    - by jacko
    As a result of a conversation with a colleague today, I've been searching (unsuccessfully) for a large'ish list of what technology stacks are being used popular websites and standalone applications today. We're aware of the big ones like Facebook (php/ ), Twitter (scala/cassandra), Youtube (python/?), Digg (php/cassandra), stackoverflow (.net mvc/sqlserver), but we're looking for a more complete list. It would also be interesting to hear about any stats for desktop apps also? Can anyone help?

    Read the article

  • Does Compressed Sensing bring anything new to data Compression?

    - by anon
    Compressed sensing is great for situations where capturing data is expensive (either in energy or time), and thus less samples can now be taken. However, in situations like image compression, given that the data is already on the computer -- does compressed sensing offer anything? For example, would it offer better data compression? Would it result in better image search?... (Note: If you don't know what the field of Compressed Sensing is, please do not respond.)

    Read the article

  • Syncing large personal school-material -git-repo with things such as casual notes? Rsync, wget and Git -- or some ready tool?

    - by hhh
    My friend wants to store electrically her school -notes and process them fast, with backups. She has over 2GB -size repo already and growing all the time (mostly appended material i.e. more school notes, different formats, pdf, pictures and scanned, some text -files, etc). The goal of my friend is to process fast the notes. I suggested command like this here i.e. "# crontab -e @weekly wget --random-wait -e robots=off -U mozilla -mirror http://VeryLong.com". But I think plugging in Rsync somewhere could make it much better with Git. How would you help my friend to process and store the school -material under Git-version-controlling and still keep the size reasonable? Perhaps related rsync .git directory rsync git big repository Different scope Git/rsync mix for projects with large binaries and text files What's a good way to organize a large collection of personal scripts using git?

    Read the article

  • How to use PHP fgetcsv to create an array for each piece of data in csv file?

    - by Olivia
    I'm trying to import data from a csv file to some html code to use it in a graph I have already coded. I'm trying to use PHP and fgetcsv to create arrays for each separate piece of data to use PHP to put it into the html code. I know how to open the csv file and to print it using PHP, and how to print each separate row, but not each piece of data separated by a comma. Is there a way to do this? If this helps, this is the csv data I am trying to import. May 10,72,12,60 May 11,86,24,62 May 12,67,32,34 May 13,87,12,75 May 14,112,23,89 May 17,69,21,48 May 18,98,14,84 May 19,115,18,97 May 20,101,13,88 May 21,107,32,75 I hope that makes sense.

    Read the article

  • How to select number of lines from large text files?

    - by MiNdFrEaK
    I was wondering how to select number of lines from a certain text file. As an example: I have a text file containing the following lines: branch 27 : rect id 23400 rect: -115.475609 -115.474907 31.393650 31.411301 branch 28 : rect id 23398 rect: -115.474907 -115.472282 31.411301 31.417351 branch 29 : rect id 23396 rect: -115.472282 -115.468033 31.417351 31.427151 branch 30 : rect id 23394 rect: -115.468033 -115.458733 31.427151 31.438181 Non-Leaf Node: level=1 count=31 address=53 branch 0 : rect id 42 rect: -115.768539 -106.251556 31.425039 31.717550 branch 1 : rect id 50 rect: -109.559479 -106.009361 31.296721 31.775299 branch 2 : rect id 51 rect: -110.937401 -106.226143 31.285870 31.771971 branch 3 : rect id 54 rect: -109.584412 -106.069092 31.285240 31.775230 branch 4 : rect id 56 rect: -109.570961 -106.000954 31.296721 31.780769 branch 5 : rect id 58 rect: -115.806213 -106.366188 31.400450 31.687519 branch 6 : rect id 59 rect: -113.173859 -106.244057 31.297440 31.627750 branch 7 : rect id 60 rect: -115.811478 -106.278252 31.400450 31.679470 branch 8 : rect id 61 rect: -109.953888 -106.020111 31.325319 31.775270 branch 9 : rect id 64 rect: -113.070969 -106.015968 31.331841 31.704750 branch 10 : rect id 68 rect: -113.065689 -107.034576 31.326300 31.770809 branch 11 : rect id 71 rect: -112.333344 -106.059860 31.284081 31.662920 branch 12 : rect id 73 rect: -115.071083 -106.309677 31.267879 31.466850 branch 13 : rect id 74 rect: -116.094414 -106.286308 31.236290 31.424770 branch 14 : rect id 75 rect: -115.423264 -106.286308 31.229691 31.415510 branch 15 : rect id 76 rect: -116.111656 -106.313110 31.259390 31.478300 branch 16 : rect id 77 rect: -116.247467 -106.309677 31.240231 31.451799 branch 17 : rect id 78 rect: -116.170792 -106.094543 31.156429 31.391781 branch 18 : rect id 79 rect: -116.225723 -106.292709 31.239960 31.442850 branch 19 : rect id 80 rect: -116.268013 -105.769913 31.157240 31.378111 branch 20 : rect id 82 rect: -116.215424 -105.827202 31.198441 31.383421 branch 21 : rect id 83 rect: -116.095734 -105.826439 31.197460 31.373819 branch 22 : rect id 84 rect: -115.423264 -105.815018 31.182640 31.368891 branch 23 : rect id 85 rect: -116.221527 -105.776512 31.160931 31.389830 branch 24 : rect id 86 rect: -116.203369 -106.473831 31.168350 31.367611 branch 25 : rect id 87 rect: -115.727631 -106.501587 31.189100 31.395941 branch 26 : rect id 88 rect: -116.237289 -105.790756 31.164780 31.358959 branch 27 : rect id 89 rect: -115.791344 -105.990044 31.072620 31.349529 branch 28 : rect id 90 rect: -115.736847 -106.495079 31.187969 31.376900 branch 29 : rect id 91 rect: -115.721710 -106.000130 31.160351 31.354601 branch 30 : rect id 92 rect: -115.792236 -106.000793 31.166620 31.378811 Leaf Node: level=0 count=21 address=42 branch 0 : rect id 18312 rect: -106.412270 -106.401367 31.704750 31.717550 branch 1 : rect id 18288 rect: -106.278252 -106.253387 31.520321 31.548361 I just want those lines which are in between Non-Leaf Node level=1 to Leaf Node Level=0 and also there are a lot of segments like this and I need them all.

    Read the article

  • Why does s ++ t not lead to a stack overflow for large s?

    - by martingw
    I'm wondering why Prelude> head $ reverse $ [1..10000000] ++ [99] 99 does not lead to a stack overflow error. The ++ in the prelude seems straight forward and non-tail-recursive: (++) :: [a] -> [a] -> [a] (++) [] ys = ys (++) (x:xs) ys = x : xs ++ ys So just with this, it should run into a stack overflow, right? So I figure it probably has something to do with the ghc magic that follows the definition of ++: {-# RULES "++" [~1] forall xs ys. xs ++ ys = augment (\c n -> foldr c n xs) ys #-} Is that what helps avoiding the stack overflow? Could someone provide some hint for what's going on in this piece of code?

    Read the article

  • Is it ok to store large objects (java component for example) in an Application variable?

    - by DustMason
    I am developing an app right now which creates and stores a connection to a local XMPP server in the Application scope. The connection methods are stored in a cfc that makes sure the Application.XMPPConnection is connected and authorized each time it is used, and makes use of the connection to send live events to users. As far as I can tell, this is working fine. BUT it hasn't been tested under any kind of stress. My question is: Will this set up cause problems later on? I only ask because I can't find evidence of other people using Application variables in this way. If I weren't using railo I would be using CF's event gateway instead to accomplish the same task.

    Read the article

< Previous Page | 478 479 480 481 482 483 484 485 486 487 488 489  | Next Page >