Search Results

Search found 12282 results on 492 pages for 'memory deallocation'.

Page 109/492 | < Previous Page | 105 106 107 108 109 110 111 112 113 114 115 116  | Next Page >

• Memory efficient import many data files into panda DataFrame in Python

  - by richardh

  I import into a panda DataFrame a directory of |-delimited.dat files. The following code works, but I eventually run out of RAM with a MemoryError:. import pandas as pd import glob temp = [] dataDir = 'C:/users/richard/research/data/edgar/masterfiles' for dataFile in glob.glob(dataDir + '/master_*.dat'): print dataFile temp.append(pd.read_table(dataFile, delimiter='|', header=0)) masterAll = pd.concat(temp) Is there a more memory efficient approach? Or should I go whole hog to a database? (I will move to a database eventually, but I am baby stepping my move to pandas.) Thanks! FWIW, here is the head of an example .dat file: cik|cname|ftype|date|fileloc 1000032|BINCH JAMES G|4|2011-03-08|edgar/data/1000032/0001181431-11-016512.txt 1000045|NICHOLAS FINANCIAL INC|10-Q|2011-02-11|edgar/data/1000045/0001193125-11-031933.txt 1000045|NICHOLAS FINANCIAL INC|8-K|2011-01-11|edgar/data/1000045/0001193125-11-005531.txt 1000045|NICHOLAS FINANCIAL INC|8-K|2011-01-27|edgar/data/1000045/0001193125-11-015631.txt 1000045|NICHOLAS FINANCIAL INC|SC 13G/A|2011-02-14|edgar/data/1000045/0000929638-11-00151.txt

  Read the article

• Allocate constant memory

  - by Vlad

  I'm trying to set my simulation params in constant memory but without luck (CUDA.NET). cudaMemcpyToSymbol function returns cudaErrorInvalidSymbol. The first parameter in cudaMemcpyToSymbol is string... Is it symbol name? actualy I don't understand how it could be resolved. Any help appreciated. //init, load .cubin float[] arr = new float[1]; arr[0] = 0.0f; int size = Marshal.SizeOf(arr[0]) * arr.Length; IntPtr ptr = Marshal.AllocHGlobal(size); Marshal.Copy(arr, 0, ptr, arr.Length); var error = CUDARuntime.cudaMemcpyToSymbol("param", ptr, 4, 0, cudaMemcpyKind.cudaMemcpyHostToDevice); my .cu file contain __constant__ float param;

  Read the article

• Which is the most memory leak safe approach.

  - by MattC

  I have a table of frequently updated information. This is presented using a container div with a div for each row, each row containing 10 divs. I am using setInterval to call a an asmx webservice that returns some json formatted information. On the success callback I call $("#myContainer").empty(); on the container div and recreate the rows and 10 nested divs for each row's columns. This page may be left to run for a whole day, so I am wary of updating the DOM like this as I have noticed that memory does rise for the browser over time (IE8). The other approach I am considering is to add an idea to the row div. When new results process each item of data, look for the corresponding row, if it exists overwrite the data in each div. If it doesn't exist (new data for example), append the row. What approaches have others used for this sort of long lived pseudo realtime information display. TIA

  Read the article

• How do you implement Software Transactional Memory?

  - by Joseph Garvin

  In terms of actual low level atomic instructions and memory fences (I assume they're used), how do you implement STM? The part that's mysterious to me is that given some arbitrary chunk of code, you need a way to go back afterward and determine if the values used in each step were valid. How do you do that, and how do you do it efficiently? This would also seem to suggest that just like any other 'locking' solution you want to keep your critical sections as small as possible (to decrease the probability of a conflict), am I right? Also, can STM simply detect "another thread entered this area while the computation was executing, therefore the computation is invalid" or can it actually detect whether clobbered values were used (and thus by luck sometimes two threads may execute the same critical section simultaneously without need for rollback)?

  Read the article

• Python In-memory table

  - by nsharish

  What is the right way to forming in-memory table in python with direct lookups for rows and columns.I thought of using dict of dicts this way, class Table(dict): def __getitem__(self, key): if key not in self: self[key]={} return dict.__getitem__(self, key) table = Table() table['row1']['column1'] = 'value11' table['row1']['column2'] = 'value12' table['row2']['column1'] = 'value21' table['row2']['column2'] = 'value22' >>>table {'row1':{'column1':'value11','column2':'value12'},'row2':{'column1':'value21','column2':'value22'}} I had difficulty in looking up for values in columns. >>>'row1' in table True >>>'value11' in table['row1'].values() True Now how do I do lookup if 'column1' has 'value11' Is this method of forming tables wrong?Is there a better way to implement such tables with easier lookups?.Thanks

  Read the article

• phpDocumentor alternative consuming less memory

  - by Michal Cihar

  Okay, I'm fed up with phpDocumentator. It consumes way much more memory and time than I'm willing to give it. Does there exist some really compatible program to generate documentation for PHP code? I've tried PHPDoctor, which looks nice, but it has some missing features. I've also tried PhpDocGen, but it just bails out with some Perl errors, which I'm too lazy to study. Doxygen also does not seem to play well with our sources. PS: The documentation would be for phpMyAdmin, a little bit outdated documentation is here.

  Read the article

• remove ViewController from memory

  - by user262325

  hello everyone I hope to load an ViewController and do something then unload it from memory. if (self.vViewController5.view.superview==nil) { ViewController5 *blueController = [[ViewController5 alloc] initWithNibName:@"View5" bundle:nil]; self.vViewController5 = blueController; [self.vViewController5 setDelegate:self]; [blueController release]; } [self presentModalViewController:vViewController5 animated:YES]; later, call [self dismissModalViewControllerAnimated:YES]; but I found that dismissModalViewControllerAnimated does not trigger the event viewDidUnload of Viewcontroller5. I try function release but it caused program collapse. I also try removeFromSuperView but it does not trigger the event ViewDidUnload neither. Welcome any comment Thanks interdev

  Read the article

• Out of Memory When Loading Java Entities

  - by Hugh Buchanan

  I have a terrible problem that hopefully has a very simple answer. I am running out of memory when I perform a basic If I have code like this: MyEntity myEntity; for (Object id: someIdList) { myEntity = find(id); // do something basic with myEntity } And the find() method is a standard EntityManager related method: public MyEntity find(Object id) { return em.find(mycorp.ejb.entity.MyEntity.class, id); } This code worked a couple of weeks ago, and works fine if there are fewer items in the database. The resulting error I am facing is: java.lang.OutOfMemoryError: GC overhead limit exceeded The exception is coming from oracle toplink calling some oracle jdbc methods. The loop exists because an EJBQL such as "select object(o) from MyEntity as o" will overload the application server when there are lots of records.

  Read the article

• Bitmapdata heavy usage - memory disaster (spark/FB4)

  - by keyle

  I've got a flex component which works pretty well but unfortunately turns into a disaster once used in a datagroup item renderer of about 40-50 items. Essentially it uses bitmapdata to take screenshot of a fully-rendered webpage in mx:HTML (this version of webkit rocks btw, miles better than flex 3). The code is pretty self-explanatory I think. http://noben.org/show/PageGrabber.mxml I've optimized it all I could, browsed, search for answers and already trimmed it down a lot, I'm desparate to reduce the memory usage (about 600mb after 100 draw) The Garbage collector has little effect. Thanks! Nic

  Read the article

• Logging Mechanism using memory mapping technique

  - by Tushar

  Just create a mapping of the file of the required size (CreateFileMapping or mmap), write the lines in the buffer and start over when the maximum number is reached. -- Your answer for write-a-circular-file-in-c. I am also writing the LogWriter module. In this caase i am mapping the whole file to the memory using mmap(). I am maintaining the Read and Write pointers.I want to write the log to the file in append mode. Then when logger service is started first time it writes it appends the logs. But when system gets shutdown next time when i run the service it doesn't append the data at the end. I want to maintain the write and read offsets even if system shut down.How to achieve this ..? How to find the how much data is written to the log file. ??

  Read the article

• Out of Memory on Update or Delete of Service Reference

  - by Kelly

  I have a Service Reference for a WCF project that has just over a hundred endpoints in my ServiceHost web.config. Every time I attempt to update or delete the Service Reference, it fails with an out of memory exception. I am running Vista Ultimate SP2 64-bit with 8GB RAM. I can work around it by going outside the project and deleting the Service References folder, then coming back in and re-adding the Reference. Is this the only workaround that you know of? Thanks!

  Read the article

• Write wave files to memory in Java

  - by Cliff

  I'm trying to figure out why my servlet code creates wave files with improper headers. I use: AudioSystem.write( new AudioInputStream( new ByteArrayInputStream(memoryBytes), new AudioFormat(22000, 16, 1, true,false), memoryBytes.length ), AudioFileFormat.Type.WAVE, servletOutputStream ); taking a byte array from memory containing raw PCM samples and a servlet output stream that gets returned to the client. In the result I get a normal wave file but with zeros in the chunk size fields. Is the API broken? I would think that the size could be filled in using the size passed in the audio input stream. But now, after typing this out I'm thinking its not making this info available to the outer write() method on AudioSystem. It seems like the AudioSystem.write call needs a size parameter unless it is able to pull the size from the stream... which wouldn't work with an arbitrary sized stream. Does anyone know how to make this example work?

  Read the article

• Create an assembly in memory

  - by Jared I

  I'd like to create an assembly in memory, using an using the classes in Reflection.Emit Currently, I can create the assembly and get it's bytes using something like AssemblyBuilder builder = AppDomain.CurrentDomain.DefineDynamicAssembly(..., AssemblyBuilderAccess.Save); ... create the assembly ... builder.Save(targetFileName); using(FileStream fs = File.Open(targetFileName, FileMode.Open)) { ... read the bytes from the file stream ... } However, it does so by creating a file on the local filesystem. I don't actually need the file, just the bytes that would be in the file. Is it possible to create the assembly without writing any files?

  Read the article

• What are the pros and cons of using an in memory DB rather than a ThreadLocal

  - by Pangea

  We have been using ThreadLocal so far to carry some data so as to not clutter the API. However below are some of issues of using thread local that which I don't like: 1) Over the years the data items being carried in thread local has increased 2) Since we started using threads (for some light weight processing), we have also migrating these data to the threads in the pool and copying them back again I am thinking of using an in memory DB for these (we doesn't want to add this to the API). I wondering if this approach is good. What are the pros and cons? thx in advance.

  Read the article

• HSQLDB Constraint Violation & SQL Query Log for an HSQLDB in-memory setup

  - by shipmaster

  We have a setup where we are using an embedded HSQLDB for backing Hibernate/JPA unit tests in java, and we are using the in-memory database mode since we simply want the database thrown away after the test run. My problem is that one of the tests is failing due to a constraint violation and HSQLDB lists the column as SYS_CT_286, and the query that appears in the log is the prepared statement where I cant see what the actual parameter values are (they are replaced by '?'). My questions are: 1- Is there a way in which I can see the actual SQL being executed? (like the mysql query log for example?). 2- What exactly is SYS_CT_286? it is not one of my columns, is it a generated column? is there something obvious that may be wrong with it? Thanks.

  Read the article

• AVAudioPlayer Memory Leak - Media Player Framework

  - by Krishnan

  Hi Friends, I am using AVAudioPlayer object to play an audio. I create an audioPlayer object initially. I play an animation and when ever animation starts I play the audio and pause the audio when the animation is finished. I initially found three memory Leaks using Instruments. (The responsible caller mentioned was RegisterEmbedCodecs). After suggestion from a "ahmet emrah" in this forum to add MediaPlayer framework, the number of leaks reduced to one. And is there any way to completely get rid of it? Thanks and regards, krishnan.

  Read the article

• realloc()ing memory for a buffer used in recv()

  - by Hristo

  I need to recv() data from a socket and store it into a buffer, but I need to make sure get all of the data so I have things in a loop. So to makes sure I don't run out of room in my buffer, I'm trying to use realloc to resize the memory allocated to the buffer. So far I have: // receive response int i = 0; int amntRecvd = 0; char *pageContentBuffer = (char*) malloc(4096 * sizeof(char)); while ((amntRecvd = recv(proxySocketFD, pageContentBuffer + i, 4096, 0)) > 0) { i += amntRecvd; realloc(pageContentBuffer, 4096 + sizeof(pageContentBuffer)); } However, this doesn't seem to be working properly since Valgrind is complaining "valgrind: the 'impossible' happened:". Any advice as to how this should be done properly? Thanks, Hristo

  Read the article

• Memcache won't flush or clear memory

  - by pedalpete

  I've been trying to clear my memcache as I'm noticing the storage taking up almost 30% of server memory when using ps -aux. So I ran the following php code. $memcache = new Memcache; $memcache-connect("localhost",11211); $memcache-flush(); print_r($memcache-getStats()); This results in the output of ( [pid] = 4936 [uptime] = 27318915 [time] = 1255318611 [version] = 1.2.2 [pointer_size] = 64 [rusage_user] = 9.659531 [rusage_system] = 49.770433 [curr_items] = 57864 [total_items] = 128246 [bytes] = 1931734247 [curr_connections] = 1 [total_connections] = 128488 [connection_structures] = 17 [cmd_get] = 170288 [cmd_set] = 128246 [get_hits] = 45464 [get_misses] = 124824 [evictions] = 1009 [bytes_read] = 5607431213 [bytes_written] = 1806543589 [limit_maxbytes] = 2147483648 [threads] = 1 ) This should be fairly basic, but clearly, I'm missing something.

  Read the article

• Lucene.Net memory consumption and slow search when too many clauses used

  - by Umer

  I have a DB having text file attributes and text file primary key IDs and indexed around 1 million text files along with their IDs (primary keys in DB). Now, I am searching at two levels. First is straight forward DB search, where i get primary keys as result (roughly 2 or 3 million IDs) Then i make a Boolean query for instance as following +Text:"test*" +(pkID:1 pkID:4 pkID:100 pkID:115 pkID:1041 .... ) and search it in my Index file. The problem is that such query (having 2 million clauses) takes toooooo much time to give result and consumes reallly too much memory.... Is there any optimization solution for this problem ?

  Read the article

• XmlDocument caching memory usage

  - by mdsharpe

  We are seeing very high memory usage in .NET web applications which use XmlDocument. A small (~5MB) XML document is loaded into an XmlDocument object and stored in HttpContext.Cache for easy querying and XSLT transformation on each page load. The XML is modified on disk periodically so a cache has a dependency on the file. Such an application appears to be using hundreds of megabytes of RAM. I have experimented with requesting garbage collection on each request start, and this keeps the RAM usage far lower but I cannot imagine this is good practise. Does anyone have any suggestions as to how we can achieve the same goal but with lower RAM usage?

  Read the article

• Memory section handling error

  - by Basilevs

  I'm getting a link time error: WARNING: /home/gulevich/development/camac-fedorov/camac/linux/k0607-lsi6/camac-k0607-lsi6.o (.ctors): unexpected non-allocatable section. Did you forget to use "ax"/"aw" in a .S file? Note that for example <linux/init.h> contains section definitions for use in .S files. The code causing the error (assembly in C source): # if defined(__ELF__) # define __SECTION_FLAGS ", \"aw\" , @progbits" /* writable flag needed for ld ".[cd]tors" sections bug workaround) */ # elif defined(__COFF__) # define __SECTION_FLAGS ", \"dr\"" /* untested, may be writable flag needed */ # endif asm ( ".section .ctors" __SECTION_FLAGS "\n" ".globl __ctors_begin__\n" "__ctors_begin__:\n" ".previous\n" ); Is there any way to fix this? The idea is to put a varaible __ctors_begin__ at the beginning of a certain memory section. This code is a legacy that worked fine using a different build system and older compiler. Meaning of this assembly code explained in an answer to my previous question.

  Read the article

• Struts/JSP/J2EE performance and memory profiling and issues

  - by Berlin Brown

  We are using Struts and having performance issues. And making heavy use of jsp includes, tiles, EL expressions. I am sure this is eating up a lot of memory and processing time. What are some approaches to profile the JSP page? What tools could I use? What should I look for when profiling? I have seen the code generated JSP Java Servlet Code and I see the bottlenecks but would rather measure it more accurately. This is under JDK1.5 and IBM Websphere 6.1 (RAD7)

  Read the article

• NSDate & Memory management

  - by iFloh

  Hi, memory management still gives me grief. This time it is an NSDate iVar that I init using NSDate *myNSDate = [[NSDate date] firstDayOfMonth]; with a method call to - (NSDate *)firstDayOfMonth { NSDateComponents *tmpDateComponents = [[NSCalendar currentCalendar] components:NSYearCalendarUnit | NSMonthCalendarUnit | NSEraCalendarUnit | NSWeekCalendarUnit | NSWeekdayOrdinalCalendarUnit fromDate:self]; [tmpDateComponents setDay:1]; [tmpDateComponents setHour:0]; [tmpDateComponents setMinute:0]; [tmpDateComponents setSecond:0]; return [[NSCalendar currentCalendar] dateFromComponents:tmpDateComponents]; } At the end of the init call the retain count is at 1 (Note the iVar is not defined as a property). When I step into the viewWillAppear method the myNSDate has vanished. I tried to do an explicit retain on it, but that only lasts until I update the iVar using the above method again. I though - ok - I add the retain to the return of the function, but that makes the leak analyser throw up an error. What am I doing wrong?

  Read the article

• Can we view objects in the JVM memory?

  - by Sebastien Lorber

  Hey, At work we found that on some instances (particulary the slow ones) we have a different behaviour, acquired at the reboot. We guess a cache is not initialized correctly, or maybe a concurrency problem... Anyway it's not reproductible in any other env than production. We actually don't have loggers to activate... it's an old component... Thus i'd like to know if there are tools that can help us to see the different objets present in the JVM memory in order to check the content of the cache... Thank you!

  Read the article

• Static DB Provider in ASP.NET MVC Causing Memory Leak

  - by user364685

  Hi, I have got an app I'm going to write in ASP.NET MVC and I want to create a DatabaseFactory object something like this:- public class DatabaseFactory { private string dbConn get { return <gets from config file>; } public IDatabaseTableObject GetDatabaseTable() { IDatabaseTableObject databaseTableObject = new SQLDatabaseObject(dbConn); return databaseTableObject; } } and this works fine, but I obviously have to instantiate the DatabaseFactory in every controller that needs it. If I made this static, so I could, in theory just call DatabaseFactory.GetDatabaseTable() it would cause a memory leak, wouldn't it?

  Read the article

< Previous Page | 105 106 107 108 109 110 111 112 113 114 115 116  | Next Page >