Search Results

Search found 14113 results on 565 pages for 'memory stick'.

Page 63/565 | < Previous Page | 59 60 61 62 63 64 65 66 67 68 69 70  | Next Page >

• How to debug JBoss out of memory problem?

  - by user561733

  Hello, I am trying to debug a JBoss out of memory problem. When JBoss starts up and runs for a while, it seems to use memory as intended by the startup configuration. However, it seems that when some unknown user action is taken (or the log file grows to a certain size) using the sole web application JBoss is serving up, memory increases dramatically and JBoss freezes. When JBoss freezes, it is difficult to kill the process or do anything because of low memory. When the process is finally killed via a -9 argument and the server is restarted, the log file is very small and only contains outputs from the startup of the newly started process and not any information on why the memory increased so much. This is why it is so hard to debug: server.log does not have information from the killed process. The log is set to grow to 2 GB and the log file for the new process is only about 300 Kb though it grows properly during normal memory circumstances. This is information on the JBoss configuration: JBoss (MX MicroKernel) 4.0.3 JDK 1.6.0 update 22 PermSize=512m MaxPermSize=512m Xms=1024m Xmx=6144m This is basic info on the system: Operating system: CentOS Linux 5.5 Kernel and CPU: Linux 2.6.18-194.26.1.el5 on x86_64 Processor information: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, 8 cores This is good example information on the system during normal pre-freeze conditions a few minutes after the jboss service startup: Running processes: 183 CPU load averages: 0.16 (1 min) 0.06 (5 mins) 0.09 (15 mins) CPU usage: 0% user, 0% kernel, 1% IO, 99% idle Real memory: 17.38 GB total, 2.46 GB used Virtual memory: 19.59 GB total, 0 bytes used Local disk space: 113.37 GB total, 11.89 GB used When JBoss freezes, system information looks like this: Running processes: 225 CPU load averages: 4.66 (1 min) 1.84 (5 mins) 0.93 (15 mins) CPU usage: 0% user, 12% kernel, 73% IO, 15% idle Real memory: 17.38 GB total, 17.18 GB used Virtual memory: 19.59 GB total, 706.29 MB used Local disk space: 113.37 GB total, 11.89 GB used

  Read the article

• What is the maximum memory that an IIS6 web site/app pool can use?

  - by Robin M

  I have an IIS 6 server running on Windows 2003 SP2 x86. The server has 4GB of RAM and runs consistently with 2GB allocated. I realise that with x86, the server won't utilize all of the 4GB RAM and the application space is also limited but the IIS processes seem to be limited elsewhere. w3wp.exe never has more than 500MB allocated and I occasionally get OutOfMemory exceptions from a busy .NET application (there are several applications running, each with a separate application pool). What is the maximum memory that an IIS6 web site/app pool can use?

  Read the article

• Is a memory upgrade a viable option to fix performance issues? [closed]

  - by ratchet freak

  I'm currently seeing my PC getting bogged down by Firefox 11.0 alone with only one hundred tabs open. Resulting in a memory use of over 530M , VM size of over 800M and an insane amount of page faults (easily reaching 100 million over the course of the day). The PF delta during normal operation easily reaches 7k with peaks to 15k sometimes reaching over 20k. This leads to a (real) deterioration to response time when switching, opening and closing tabs, opening menus, typing, ... My question is: Am I right in assuming that plugging in more RAM (either adding 2x1GB or replacing the existing RAM with 2x2GB or 4x1GB) will solve this problem? My specs: Windows XP Home Edition SP3 (32 bit) Intel Core Duo 2,4 GHz 2x512MB RAM 800MHz DDR2 (dual channel) 4MB unified cache 320GB HDD Intel G33 (X3100) onboard graphics (no graphics card but PCI express x16 slot is available)

  Read the article

• Tracking down memory issues affecting a website

  - by gaoshan88

  I've got a website (Wordpress based) that became unresponsive. I SSH'd into the server and saw that we were out of memory. Errors in my apache log files indicated the same... things failing to be allocated due to lack of memory). Restarting the server fixes it. So I look in access.log and error.log around the time of the incident but I see nothing strange. No extra traffic, no unusual requests. In fact the only request around the time of the problem was one from Googlebot for an rss feed... at that point I start to see 500 response codes in the logs until the machine was rebooted. I look in message.log hoping to see something but there is nothing at all for that entire day (which is odd as there are entries for every other day). The site has a large amount of memory allocated to it and normally runs using about 30% of what is available. My question... how would you go about trying to track this down at this point? What are some other log files I could check or strategies I could take?

  Read the article

• Representing a world in memory

  - by user9993

  I'm attempting to write a chunk based map system for a game, where as the player moves around chunks are loaded/unloaded, so that the program doesn't run out of memory by having the whole map in memory. I have this part mostly working, however I've hit a wall regarding how to represent the contents of each chunk in memory because of my so far limited understanding of OOP languages. The design I have currently has a ChunkManager class that uses a .NET List type to store instances of Chunk classes. The "chunks" consist of "blocks". It's similar to a Minecraft style game. Inside the Chunk classes, I have some information such as the chunk's X/Y coordinate etc, and I also have a three dimensional array of Block objects. (Three dimensional because I need XYZ values) Here's the problem: The Block class has some basic properties, and I had planned on making different types of blocks inherit from this "base" class. So for example, I would have "StoneBlock", "WaterBlock" etc. So because I have blocks of many different types, I don't know how I would create an array with different object types in each cell. This is how I currently have the three dimensional array declared in my Chunk class: private Block[][][] ArrayOfBlocks; But obviously this will only accept Block objects, not any of the other classes that inherit from Block. How would I go about creating this? Or am I doing it completely wrong and there's a better way?

  Read the article

• High Sqlservr.exe Memory Usage

  - by user18576

  I have a problem with sqlservr.exe (version 2008). It use a more memory. I checked on windows taskbar manager, sqlservr.exe usage ( Mem usage - 8GB Ram). I dont know how can I fix it.Got the following metrics of the server using Perfmon: SQLServer:Buffer Manager Buffer cache hit ratio 13 SQLServer:Buffer Manager Page lookups/sec 46026128096 SQLServer:Buffer Manager Free pages 129295 SQLServer:Buffer Manager Total pages 997309 SQLServer:Buffer Manager Target pages 1053560 SQLServer:Buffer Manager Database pages 484117 SQLServer:Buffer Manager Reserved pages 0 SQLServer:Buffer Manager Stolen pages 383897 SQLServer:Buffer Manager Lazy writes/sec 384369 SQLServer:Buffer Manager Readahead pages/sec 69315446 SQLServer:Buffer Manager Page reads/sec 71280353 SQLServer:Buffer Manager Page writes/sec 12408371 SQLServer:Buffer Manager Checkpoint pages/sec 7053801 SQLServer:Buffer Manager Page life expectancy 735262 SQLServer:General Statistics Active Temp Tables 161 SQLServer:General Statistics Temp Tables Creation Rate 3131845 SQLServer:General Statistics Logins/sec 2336011 SQLServer:General Statistics Logouts/sec 2335984 SQLServer:General Statistics User Connections 27 SQLServer:General Statistics Transactions 0 SQLServer:Access Methods Full Scans/sec 34422821 SQLServer:Access Methods Range Scans/sec 2027247756 SQLServer:Access Methods Workfiles Created/sec 49771600 SQLServer:Access Methods Worktables Created/sec 28205828 SQLServer:Access Methods Index Searches/sec 4890715219 SQLServer:Access Methods FreeSpace Scans/sec 21178928 SQLServer:Access Methods FreeSpace Page Fetches/sec 21226653 SQLServer:Access Methods Pages Allocated/sec 41483279 SQLServer:Access Methods Extents Allocated/sec 4743504 SQLServer:Access Methods Extent Deallocations/sec 4806606 SQLServer:Access Methods Page Deallocations/sec 41419137 SQLServer:Access Methods Page Splits/sec 23834799 SQLServer:Memory Manager SQL Cache Memory (KB) 29160 SQLServer:Memory Manager Target Server Memory (KB) 8428480 SQLServer:Memory Manager Total Server Memory (KB) 7978472 Some body could help me please.And I really want to know the cause for the above.

  Read the article

• Setting kernel memory for installing postgresql

  - by Matthieu Taymans

  My question is about setting the kernel shared memory for installing postgresql on mac osx 10.6.8. In the readme file of postgresql it is said: Shared Memory PostgreSQL uses shared memory extensively for caching and inter-process communication. Unfortunately, the default configuration of Mac OS X does not allow suitable amounts of shared memory to be created to run the database server. Before running the installation, please ensure that your system is configured to allow the use of larger amounts of shared memory. Note that this does not 'reserve' any memory so it is safe to configure much higher values than you might initially need. You can do this by editting the file /etc/sysctl.conf - e.g. % sudo vi /etc/sysctl.conf On a MacBook Pro with 2GB of RAM, the author's sysctl.conf contains: kern.sysv.shmmax=1610612736 kern.sysv.shmall=393216 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.maxprocperuid=512 kern.maxproc=2048 Note that (kern.sysv.shmall * 4096) should be greater than or equal to kern.sysv.shmmax. kern.sysv.shmmax must also be a multiple of 4096. Once you have edited (or created) the file, reboot before continuing with the installation. If you wish to check the settings currently being used by the kernel, you can use the sysctl utility: % sysctl -a The database server can now be installed. I'm a real beginner with all this but need to instal postgresql for academic purposes do you know how i can set this kernel shared memory. Won't that be harmful for my system? Thank you in advance. Matthieu

  Read the article

• High memory utilization with firebird + windows server 2008 r2

  - by chesterman

  i have a Windows Server 2008 R2 (64bit) running a 64bit installation of Firebird 2.1.4.18393_0 in a 4GB phisical server. After a while, the task manager show that all memory is used, but the sum of the memory of all process does not stack to the half of the memory. Unfortunally, it's start swapping. Using RAMMAP, i can see that my entire database file is mapped into the memory. This only occours in windows server 2008 r2 and windows 7 64 bit. i can use firebird 32 or 64bit installations, doesn't matter. How can i prevent this? Why this only occours in w2k8r2 and w7? tks in advance ** UPDATE Aparently, this occours by the use of all memory by the file system cache. The microsoft documentations explain that this WAS a issue in windows xp, 2k3, vista and 2k8, but it was solved in 7 and 2k8r2. also adds that this issue is more common in 64bit hosts. (http://support.microsoft.com/kb/976618) there are some tools (DynCache, setcache and the Get/SetSystemFileCacheSize system calls from windows API) that allows me to fix a upper limit to the memory usage by the fscache, but the documentation argues that i should not do this in w2k8r2 because it will severely impact on the overall system performance. anyway, i tried, the performance remained the same shit and the use of the page file remained, although there is now more the 1gb of free memory.

  Read the article

• Is it possible to store pointers in shared memory without using offsets?

  - by Joseph Garvin

  When using shared memory, each process may mmap the shared region into a different area of their address space. This means that when storing pointers within the shared region, you need to store them as offsets of the start of the shared region. Unfortunately, this complicates use of atomic instructions (e.g. if you're trying to write a lock free algorithm). For example, say you have a bunch of reference counted nodes in shared memory, created by a single writer. The writer periodically atomically updates a pointer 'p' to point to a valid node with positive reference count. Readers want to atomically write to 'p' because it points to the beginning of a node (a struct) whose first element is a reference count. Since p always points to a valid node, incrementing the ref count is safe, and makes it safe to dereference 'p' and access other members. However, this all only works when everything is in the same address space. If the nodes and the 'p' pointer are stored in shared memory, then clients suffer a race condition: x = read p y = x + offset Increment refcount at y During step 2, p may change and x may no longer point to a valid node. The only workaround I can think of is somehow forcing all processes to agree on where to map the shared memory, so that real pointers rather than offsets can be stored in the mmap'd region. Is there any way to do that? I see MAP_FIXED in the mmap documentation, but I don't know how I could pick an address that would be safe.

  Read the article

• How do I keep from running out of memory on graphics for an Android app?

  - by user279112

  I've been working on an Android app in Eclipse, and so far, my program hasn't really grown past midget size. However I've already run into an issue with an Out of Memory error. You see, I've been using graphics comprised solely of bitmaps and PNGs in this program, and recently, when I tried to add a little bit more functionality to the program (mainly including a few more bitmaps and causing an extra sprite to be created), it started crashing in the graphics thread's constructor - sprite's constructor. When I tracked the problem down, it turned out to be an Out of Memory error that is seemingly caused by adding too many picture files to the program and creating Drawables out of them. This would be a problem, as I really don't have that many picture resources worked into that program...maybe 20 or so. I haven't even started to include sound yet. These images aren't all that fancy. My questions are this: 1) Are programs for the Android phone really that limited on how much memory they can employ, or is it probably something other than the 20-30 resource pictures causing that error? 2) If the memory for Android apps is so awful it can't even handle 20-30 picture resources being loaded into Drawables that exist at the same time, then how in the world are you supposed to make decent graphics and sound for that thing? Thanks.

  Read the article

• GetIpAddrTable() leaks memory. How to resolve that?

  - by Stabledog

  On my Windows 7 box, this simple program causes the memory use of the application to creep up continuously, with no upper bound. I've stripped out everything non-essential, and it seems clear that the culprit is the Microsoft Iphlpapi function "GetIpAddrTable()". On each call, it leaks some memory. In a loop (e.g. checking for changes to the network interface list), it is unsustainable. There seems to be no async notification API which could do this job, so now I'm faced with possibly having to isolate this logic into a separate process and recycle the process periodically -- an ugly solution. Any ideas? // IphlpLeak.cpp - demonstrates that GetIpAddrTable leaks memory internally: run this and watch // the memory use of the app climb up continuously with no upper bound. #include <stdio.h> #include <windows.h> #include <assert.h> #include <Iphlpapi.h> #pragma comment(lib,"Iphlpapi.lib") void testLeak() { static unsigned char buf[16384]; DWORD dwSize(sizeof(buf)); if (GetIpAddrTable((PMIB_IPADDRTABLE)buf, &dwSize, false) == ERROR_INSUFFICIENT_BUFFER) { assert(0); // we never hit this branch. return; } } int main(int argc, char* argv[]) { for ( int i = 0; true; i++ ) { testLeak(); printf("i=%d\n",i); Sleep(1000); } return 0; }

  Read the article

• How to detect .NET WPF memory leak or GC long run?

  - by Néstor Sánchez A.

  I have the next very strange situation and problem: .NET 4.0 application for diagram editing (WPF). Runs ok in my PC: 8GM RAM, 3.0GHz, i7 quad-core. While creating objects (mostly diagram nodes and connectors, plus all the undo/redo information) the TaskManager show, as expected, some memory usage "jumps" (up and down). These mem-usage "jumps" also remains executing AFTER user interaction ended. Maybe this is the GC cleaning/regorganizing memory? To see what is going on, I've used the Ants mem profiler, but somewhat it prevents those "jumps" to happen after user interaction. PROBLEM: It Freezes/Hangs after seconds or minutes of usage in some slow/weak laptos/netbooks of my beta testers (under 2GHz of speed and under 2GB of RAM). I was thinking of a memory leak, but... EDIT: Also, there is the case that the memory usage grows and grows until collapse (only in slow machines). In a Windows XP Mode machine (VM in Win 7) with only 512MB of RAM Assigned it works fine without mem-usage "jumps" after user interaction (no GC cleaning?!). So, I really have a big trouble because I cannot reproduce the error, only see these strange behaviour (mem jumps), and the tool supposed to show me what is happening is hiding the problem (like the "observer's paradox"). Any ideas on what's happening and how to solve it?

  Read the article

• Huge burst of memory in c# service, what could be the cause?

  - by Daniel

  I'm working on a c# service application and i have this problem where out of no where and for no obvious reason, the memory for the process will climb from 150mb to almost 2gb in about 5 seconds and then back to 150mb. But nothing in our system should be using any where near that amount of memory (so its probably a bug somewhere). It might be a tight while true loop somewhere but the cpu usage at the time was very low so i thought i'd look for other ideas. Now the weirder thing is when i compile the service for 64bit, the same massive burst will occur except it exceeded 10gb of ram (paging most of it) and it just caused lots of problems with the computer and everything running on it. After a while it shuts down but it looks like windows is still willing to give it more memory. Would you have any ideas or tools that i can use in order to find this? Yes it has lots of logging however nothing in the logs stand out as to why this is happening. I can run the service in a console app mode, so my next test was going to be running it in visual studio debugger and see if i can find anything. It only happens occasionally but usually about 10-20 minutes after startup. On 32bit mode it cleans up and continues on like normally. 64bit mode it crashes after a while and uses stupid amounts of memory. But i'm really stumped as to why this is happening!!!

  Read the article

• Users take over a minute to log onto a 2008 windows server. LSM.exe running at 100MB+ memory.

  - by seanyboy

  We've a 64bit Windows Server 2008 running Remote Desktop. The application lsm.exe (the local session manager) appears to be leaking memory. Although the memory usage is quite low when the server is rebooted, this continues to climb until people can no longer log in. The server has no audio card and does not have any AV software installed. The server is fully service packed. (Service pack 2) What could be causing this, and how would I fix it?

  Read the article

• SQL SERVER – Data Pages in Buffer Pool – Data Stored in Memory Cache

  - by pinaldave

  This will drop all the clean buffers so we will be able to start again from there. Now, run the following script and check the execution plan of the query. Have you ever wondered what types of data are there in your cache? During SQL Server Trainings, I am usually asked if there is any way one can know how much data in a table is stored in the memory cache? The more detailed question I usually get is if there are multiple indexes on table (and used in a query), were the data of the single table stored multiple times in the memory cache or only for a single time? Here is a query you can run to figure out what kind of data is stored in the cache. USE AdventureWorks GO SELECT COUNT(*) AS cached_pages_count, name AS BaseTableName, IndexName, IndexTypeDesc FROM sys.dm_os_buffer_descriptors AS bd INNER JOIN ( SELECT s_obj.name, s_obj.index_id, s_obj.allocation_unit_id, s_obj.OBJECT_ID, i.name IndexName, i.type_desc IndexTypeDesc FROM ( SELECT OBJECT_NAME(OBJECT_ID) AS name, index_id ,allocation_unit_id, OBJECT_ID FROM sys.allocation_units AS au INNER JOIN sys.partitions AS p ON au.container_id = p.hobt_id AND (au.type = 1 OR au.type = 3) UNION ALL SELECT OBJECT_NAME(OBJECT_ID) AS name, index_id, allocation_unit_id, OBJECT_ID FROM sys.allocation_units AS au INNER JOIN sys.partitions AS p ON au.container_id = p.partition_id AND au.type = 2 ) AS s_obj LEFT JOIN sys.indexes i ON i.index_id = s_obj.index_id AND i.OBJECT_ID = s_obj.OBJECT_ID ) AS obj ON bd.allocation_unit_id = obj.allocation_unit_id WHERE database_id = DB_ID() GROUP BY name, index_id, IndexName, IndexTypeDesc ORDER BY cached_pages_count DESC; GO Now let us run the query above and observe the output of the same. We can see in the above query that there are four columns. Cached_Pages_Count lists the pages cached in the memory. BaseTableName lists the original base table from which data pages are cached. IndexName lists the name of the index from which pages are cached. IndexTypeDesc lists the type of index. Now, let us do one more experience here. Please note that you should not run this test on a production server as it can extremely reduce the performance of the database. DBCC DROPCLEANBUFFERS This will drop all the clean buffers and we will be able to start again from there. Now run following script and check the execution plan for the same. USE AdventureWorks GO SELECT UnitPrice, ModifiedDate FROM Sales.SalesOrderDetail WHERE SalesOrderDetailID BETWEEN 1 AND 100 GO The execution plans contain the usage of two different indexes. Now, let us run the script that checks the pages cached in SQL Server. It will give us the following output. It is clear from the Resultset that when more than one index is used, datapages related to both or all of the indexes are stored in Memory Cache separately. Let me know what you think of this article. I had a great pleasure while writing this article because I was able to write on this subject, which I like the most. In the next article, we will exactly see what data are cached and those that are not cached, using a few undocumented commands. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: DMV, Pinal Dave, SQL, SQL Authority, SQL Optimization, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, T SQL, Technology Tagged: SQL DMV

  Read the article

• Game Object Factory: Fixing Memory Leaks

  - by Bunkai.Satori

  Dear all, this is going to be tough: I have created a game object factory that generates objects of my wish. However, I get memory leaks which I can not fix. Memory leaks are generated by return new Object(); in the bottom part of the code sample. static BaseObject * CreateObjectFunc() { return new Object(); } How and where to delete the pointers? I wrote bool ReleaseClassType(). Despite the factory works well, ReleaseClassType() does not fix memory leaks. bool ReleaseClassTypes() { unsigned int nRecordCount = vFactories.size(); for (unsigned int nLoop = 0; nLoop < nRecordCount; nLoop++ ) { // if the object exists in the container and is valid, then render it if( vFactories[nLoop] != NULL) delete vFactories[nLoop](); } return true; } Before taking a look at the code below, let me help you in that my CGameObjectFactory creates pointers to functions creating particular object type. The pointers are stored within vFactories vector container. I have chosen this way because I parse an object map file. I have object type IDs (integer values) which I need to translate them into real objects. Because I have over 100 different object data types, I wished to avoid continuously traversing very long Switch() statement. Therefore, to create an object, I call vFactoriesnEnumObjectTypeID via CGameObjectFactory::create() to call stored function that generates desired object. The position of the appropriate function in the vFactories is identical to the nObjectTypeID, so I can use indexing to access the function. So the question remains, how to proceed with garbage collection and avoid reported memory leaks? #ifndef GAMEOBJECTFACTORY_H_UNIPIXELS #define GAMEOBJECTFACTORY_H_UNIPIXELS //#include "MemoryManager.h" #include <vector> template <typename BaseObject> class CGameObjectFactory { public: // cleanup and release registered object data types bool ReleaseClassTypes() { unsigned int nRecordCount = vFactories.size(); for (unsigned int nLoop = 0; nLoop < nRecordCount; nLoop++ ) { // if the object exists in the container and is valid, then render it if( vFactories[nLoop] != NULL) delete vFactories[nLoop](); } return true; } // register new object data type template <typename Object> bool RegisterClassType(unsigned int nObjectIDParam ) { if(vFactories.size() < nObjectIDParam) vFactories.resize(nObjectIDParam); vFactories[nObjectIDParam] = &CreateObjectFunc<Object>; return true; } // create new object by calling the pointer to the appropriate type function BaseObject* create(unsigned int nObjectIDParam) const { return vFactories[nObjectIDParam](); } // resize the vector array containing pointers to function calls bool resize(unsigned int nSizeParam) { vFactories.resize(nSizeParam); return true; } private: //DECLARE_HEAP; template <typename Object> static BaseObject * CreateObjectFunc() { return new Object(); } typedef BaseObject*(*factory)(); std::vector<factory> vFactories; }; //DEFINE_HEAP_T(CGameObjectFactory, "Game Object Factory"); #endif // GAMEOBJECTFACTORY_H_UNIPIXELS

  Read the article

• Is it possible to share a C struct in shared memory between apps compiled with different compilers?

  - by Joseph Garvin

  I realize that in general the C and C++ standards gives compiler writers a lot of latitude. But in particular it guarantees that POD types like C struct members have to be laid out in memory the same order that they're listed in the structs definition, and most compilers provide extensions letting you fix the alignment of members. So if you had a header that defined a struct and manually specified the alignment of its members, then compiled two apps with different compilers using the header, shouldn't one app be able to write an instance of the struct into shared memory and the other app be able to read it without errors? I am assuming though that the size of the types contained is consistent across two compilers on the same architecture (it has to be the same platform already since we're talking about shared memory). I realize that this is not always true for some types (e.g. long vs. long long in GCC and MSVC 64-bit) but nowadays there are uint16_t, uint32_t, etc. types, and float and double are specified by IEEE standards.

  Read the article

• What are some of best Javascript memory detecting tools?

  - by Philip Fourie

  Our team is faced with slow but serious Javascript memory leak. We have read up on the normal causes for memory leaks in Javascript (eg. closures and circular references). We tried to avoid those pitfalls in the code but it likely we still have unknown mistakes left in our code. I started my search for available tools but would like input from people with actual experience with these tools. Some of the tools I found so far (but have no idea how good and useful they would be for our problem): Sieve Drip JavaScript Memory Leak Detector Our search is not limited to free tools, it will be a bonus, but more importantly something that will get the job done. We do the following in our Javascript code: AJAX calls to a .NET WCF back-end that send back JSON data Manipulate the DOM Keep a fairly sized object model in the Javascript to store current state

  Read the article

• Can SHA-1 algorithm be computed on a stream? With low memory footprint?

  - by raoulsson

  I am looking for a way to compute SHA-1 checksums of very large files without having to fully load them into memory at once. I don't know the details of the SHA-1 implementation and therefore would like to know if it is even possible to do that. If you know the SAX XML parser, then what I look for would be something similar: Computing the SHA-1 checksum by only always loading a small part into memory at a time. All the examples I found, at least in Java, always depend on fully loading the file/byte array/string into memory. If you even know implementations (any language), then please let me know!

  Read the article

• How do I force my std::map to deallocate memory used?

  - by monkeyking

  I'm using a std::map, and I can't seem to free the memory back to the OS. It looks like, int main(){ aMap m; while(keepGoing){ while(fillUpMap){ //populate m } doWhatIwantWithMap(m); m.clear(); //flush some buffered values into map for next iteration flushIntoMap(m); } } Each (fillUpmap) allocates around 1gig, so I'm very much interested in getting this back to my system before it eats up all my memory. Ive experienced the same with std::vector, but there I could force it to free by doing a swap with an empty std::vector. This doesn't work with map. When I use valgrind it says that all memory is freed, so its not a problem with a leak, since everything is cleared up nicely after a run.

  Read the article

• How do you make your Java application memory efficient?

  - by Boune

  How do you optimize the heap size usage of an application that has a lot (millions) of long-lived objects? (big cache, loading lots of records from a db) Use the right data type Avoid java.lang.String to represent other data types Avoid duplicated objects Use enums if the values are known in advance Use object pools String.intern() (good idea?) Load/keep only the objects you need I am looking for general programming or Java specific answers. No funky compiler switch. Edit: Optimize the memory representation of a POJO that can appear millions of times in the heap. Use cases Load a huge csv file in memory (converted into POJOs) Use hibernate to retrieve million of records from a database Resume of answers: Use flyweight pattern Copy on write Instead of loading 10M objects with 3 properties, is it more efficient to have 3 arrays (or other data structure) of size 10M? (Could be a pain to manipulate data but if you are really short on memory...)

  Read the article

• Which is faster in memory, ints or chars? And file-mapping or chunk reading?

  - by Nick

  Okay, so I've written a (rather unoptimized) program before to encode images to JPEGs, however, now I am working with MPEG-2 transport streams and the H.264 encoded video within them. Before I dive into programming all of this, I am curious what the fastest way to deal with the actual file is. Currently I am file-mapping the .mts file into memory to work on it, although I am not sure if it would be faster to (for example) read 100 MB of the file into memory in chunks and deal with it that way. These files require a lot of bit-shifting and such to read flags, so I am wondering that when I reference some of the memory if it is faster to read 4 bytes at once as an integer or 1 byte as a character. I thought I read somewhere that x86 processors are optimized to a 4-byte granularity, but I'm not sure if this is true... Thanks!

  Read the article

• How much memory is reserved when i declare a string?

  - by Bhagya

  What exactly happens, in terms of memory, when i declare something like: char arr[4]; How many bytes are reserved for arr? How is null string accommodated when I 'strcpy' a string of length 4 in arr? I was writing a socket program, and when I tried to suffix NULL at arr[4] (i.e. the 5th memory location), I ended up replacing the values of some other variables of the program (overflow) and got into a big time mess. Any descriptions of how compilers (gcc is what I used) manage memory?

  Read the article

• Which of FILE* or ifstream has better memory usage?

  - by Viet

  I need to read fixed number of bytes from files, whose sizes are around 50MB. To be more precise, read a frame from YUV 4:2:0 CIF/QCIF files (~25KB to ~100KB per frame). Not very huge number but I don't want whole file to be in the memory. I'm using C++, in such a case, which of FILE* or ifstream has better (less/minimal) memory usage? Please kindly advise. Thanks! EDIT: I read fixed number of bytes: 25KB or 100KB (depending on QCIF/CIF format). The reading is in binary mode and forward-only. No seeking needed. No writing needed, only reading. EDIT: If identifying better of them is hard, which one does not require loading the whole file into memory?

  Read the article

• How to protect against GHC7 compiled programs taking all memory?

  - by Petr Pudlák

  When playing with various algorithms in Haskell it often happens to me that I create a program with a memory leak, as it often happens with lazy evaluation. The program taking all the memory isn't really fun, I often have difficulty killing it if I realize it too late. When using GHC6 I simply had export GHCRTS='-M384m' in my .bashrc. But in GHC7 they added a security measure that unless a program is compiled with -rtsopts, it simply fails when it is given any RTS option either on a command line argument or in GHCRTS. Unfortunately, almost no Haskell programs are compiled with this flag, so setting this variable makes everything to fail (as I discovered in After upgrading to GHC7, all programs suddenly fail saying "Most RTS options are disabled. Link with -rtsopts to enable them."). Any ideas how to make any use of GHCRTS with GHC7, or another convenient way how to prevent my programs taking all memory?

  Read the article

< Previous Page | 59 60 61 62 63 64 65 66 67 68 69 70  | Next Page >