Search Results

Search found 41025 results on 1641 pages for 'in memory database'.

Page 118/1641 | < Previous Page | 114 115 116 117 118 119 120 121 122 123 124 125  | Next Page >

• Can SHA-1 algorithm be computed on a stream? With low memory footprint?

  - by raoulsson

  I am looking for a way to compute SHA-1 checksums of very large files without having to fully load them into memory at once. I don't know the details of the SHA-1 implementation and therefore would like to know if it is even possible to do that. If you know the SAX XML parser, then what I look for would be something similar: Computing the SHA-1 checksum by only always loading a small part into memory at a time. All the examples I found, at least in Java, always depend on fully loading the file/byte array/string into memory. If you even know implementations (any language), then please let me know!

  Read the article

• Sqlite3 Database versus populating Arrays

  - by Kenoy

  hi, I am working on a program that requires me to input values for 12 objects, each with 4 arrays, each with 100 values. (4800) values. The 4 arrays represent possible outcomes based on 2 boolean values... i.e. YY, YN, NN, NY and the 100 values to the array are what I want to extract based on another inputted variable. I previously have all possible outcomes in a csv file, and have imported these into sqlite where I can query then for the value using sql. However, It has been suggested to me that sqlite database is not the way to go, and instead I should populate using arrays hardcoded. Which would be better during run time and for memory management?

  Read the article

• How do I force my std::map to deallocate memory used?

  - by monkeyking

  I'm using a std::map, and I can't seem to free the memory back to the OS. It looks like, int main(){ aMap m; while(keepGoing){ while(fillUpMap){ //populate m } doWhatIwantWithMap(m); m.clear(); //flush some buffered values into map for next iteration flushIntoMap(m); } } Each (fillUpmap) allocates around 1gig, so I'm very much interested in getting this back to my system before it eats up all my memory. Ive experienced the same with std::vector, but there I could force it to free by doing a swap with an empty std::vector. This doesn't work with map. When I use valgrind it says that all memory is freed, so its not a problem with a leak, since everything is cleared up nicely after a run.

  Read the article

• How do you make your Java application memory efficient?

  - by Boune

  How do you optimize the heap size usage of an application that has a lot (millions) of long-lived objects? (big cache, loading lots of records from a db) Use the right data type Avoid java.lang.String to represent other data types Avoid duplicated objects Use enums if the values are known in advance Use object pools String.intern() (good idea?) Load/keep only the objects you need I am looking for general programming or Java specific answers. No funky compiler switch. Edit: Optimize the memory representation of a POJO that can appear millions of times in the heap. Use cases Load a huge csv file in memory (converted into POJOs) Use hibernate to retrieve million of records from a database Resume of answers: Use flyweight pattern Copy on write Instead of loading 10M objects with 3 properties, is it more efficient to have 3 arrays (or other data structure) of size 10M? (Could be a pain to manipulate data but if you are really short on memory...)

  Read the article

• Which of FILE* or ifstream has better memory usage?

  - by Viet

  I need to read fixed number of bytes from files, whose sizes are around 50MB. To be more precise, read a frame from YUV 4:2:0 CIF/QCIF files (~25KB to ~100KB per frame). Not very huge number but I don't want whole file to be in the memory. I'm using C++, in such a case, which of FILE* or ifstream has better (less/minimal) memory usage? Please kindly advise. Thanks! EDIT: I read fixed number of bytes: 25KB or 100KB (depending on QCIF/CIF format). The reading is in binary mode and forward-only. No seeking needed. No writing needed, only reading. EDIT: If identifying better of them is hard, which one does not require loading the whole file into memory?

  Read the article

• Which is faster in memory, ints or chars? And file-mapping or chunk reading?

  - by Nick

  Okay, so I've written a (rather unoptimized) program before to encode images to JPEGs, however, now I am working with MPEG-2 transport streams and the H.264 encoded video within them. Before I dive into programming all of this, I am curious what the fastest way to deal with the actual file is. Currently I am file-mapping the .mts file into memory to work on it, although I am not sure if it would be faster to (for example) read 100 MB of the file into memory in chunks and deal with it that way. These files require a lot of bit-shifting and such to read flags, so I am wondering that when I reference some of the memory if it is faster to read 4 bytes at once as an integer or 1 byte as a character. I thought I read somewhere that x86 processors are optimized to a 4-byte granularity, but I'm not sure if this is true... Thanks!

  Read the article

• How much memory is reserved when i declare a string?

  - by Bhagya

  What exactly happens, in terms of memory, when i declare something like: char arr[4]; How many bytes are reserved for arr? How is null string accommodated when I 'strcpy' a string of length 4 in arr? I was writing a socket program, and when I tried to suffix NULL at arr[4] (i.e. the 5th memory location), I ended up replacing the values of some other variables of the program (overflow) and got into a big time mess. Any descriptions of how compilers (gcc is what I used) manage memory?

  Read the article

• How to protect against GHC7 compiled programs taking all memory?

  - by Petr Pudlák

  When playing with various algorithms in Haskell it often happens to me that I create a program with a memory leak, as it often happens with lazy evaluation. The program taking all the memory isn't really fun, I often have difficulty killing it if I realize it too late. When using GHC6 I simply had export GHCRTS='-M384m' in my .bashrc. But in GHC7 they added a security measure that unless a program is compiled with -rtsopts, it simply fails when it is given any RTS option either on a command line argument or in GHCRTS. Unfortunately, almost no Haskell programs are compiled with this flag, so setting this variable makes everything to fail (as I discovered in After upgrading to GHC7, all programs suddenly fail saying "Most RTS options are disabled. Link with -rtsopts to enable them."). Any ideas how to make any use of GHCRTS with GHC7, or another convenient way how to prevent my programs taking all memory?

  Read the article

• using arrays to get best memory alignment and cache use, is it necessary?

  - by Alberto Toglia

  I'm all about performance these days cause I'm developing my first game engine. I'm no c++ expert but after some research I discovered the importance of the cache and the memory alignment. Basically what I found is that it is recommended to have memory well aligned specially if you need to access them together, for example in a loop. Now, In my project I'm doing my Game Object Manager, and I was thinking to have an array of GameObjects references. meaning I would have the actual memory of my objects one after the other. static const size_t MaxNumberGameObjects = 20; GameObject mGameObjects[MaxNumberGameObjects]; But, as I will be having a list of components per object -Component based design- (Mesh, RigidBody, Transformation, etc), will I be gaining something with the array at all? Anyway, I have seen some people just using a simple std::map for storing game objects. So what do you guys think? Am I better off using a pure component model?

  Read the article

• Under Windows CE, how can I check which RAM based DLLs are loaded in virtual memory space?

  - by Michal Drozdowicz

  I have a problem with loading a DLL under Windows Mobile 5.0. I'm pretty confident that this is caused by running out of the application virtual memory (the 32 MB slot of the process, as explained in Windows CE .NET Advanced Memory Management). I'm looking for a way to actually make sure that this is the issue and investigate whether my efforts bring expected results. Do you know of a way to check the contents of the virtual memory application slot? Any applications that can help me with this task?

  Read the article

• Why and How to avoid Event Handler memory leaks ?

  - by gillyb

  Hey There, I just came to realize, by reading some questions and answers on StackOverflow, that adding event handlers using += in C# (or i guess, other .net languages) can cause common memory leaks... I have used event handlers like this in the past many times, and never realized that they can cause, or have caused, memory leaks in my applications. How does this work (meaning, why does this actually cause a memory leak) ? How can I fix this problem ? Is using -= to the same event handler enough ? Are there common design patterns or best practices for handling situations like this ? Example : How am I supposed to handle an application that has many different threads, using many different event handlers to raise several events on the UI ? Are there any good and simple ways to monitor this efficiently in an already built big application ? Thanks in advance!

  Read the article

• Shared memory of same DLL in different 32 bit processes is sometimes different in a terminal session

  - by KBrusing

  We have an 32 bit application consisting of some processes. They communicate with shared memory of a DLL used by every process. Shared memory is build with global variables in C++ by "#pragma data_seg ("Shared")". When running this application sometime during starting a new process in addition to an existing (first) process we observe that the shared memory of both processes is not the same. All new started processes cannot communicate with the first process. After stopping all of our processes and restarting the application (with some processes) everything works fine. But sometime or other after successfully starting and finishing new processes the problem occurs again. Running on all other Windows versions or terminal sessions on Windows server 2003 our application never got this problem. Is there any new "feature" on Windows server 2008 that might disturb the hamony of our application?

  Read the article

• How to manage memory for ios with large csv files?

  - by Pell000

  I'm new to ios development, and I'm running into issues relating to memory management and my approaches to dealing with large datasets. Right now, I am loading the csv files and storing the relevant data as objects in memory at app initialization. Some of the csv files are larger than 1MB, and in total, my app uses about 180MB of memory. This is obviously way too high of a number (unless the info I found is wrong and this is an acceptable number, then please let me know). I feel as though there is a fundamental flaw in my approach: is there a way I can avoid storing the csv files in the project itself? Or, is there a kind of "lazy" loading I can do so that I can simply look up info in the csv file, as opposed to loading all of the data from it at once? Any help would do. I think that I need a new perspective in how to manage this more efficiently.

  Read the article

• SQL Server 2005 jobs running twice in a row - using LiteSpeed

  - by Malnizzle

  Howdy! I have a SQL server (2005) backing up to a network share, who has a group of maintenance plans setup through LiteSpeed to backup different DBs. They were just set up to run two sub plans on different schedules for full/diff backups and did that just fine for a couple of months. Then I added "Clean Up" task to the subplans. Ever since that point, the backup creates another bak right after the first bak job is completed. I removed the clean up item from the subplan, and it still creates two baks when ran. Both the SQL Activity Monitor and the machine's windows application log show just one job being executed. I did this same thing to a couple of other servers backing up to the same location, and they are behaving correctly. Thoughts?

  Read the article

• How to interpret output from Linux 'top' command?

  - by Ali

  Following a discussion made HERE about how PHP-FPM consuming memory, I just found a problem in reading the memory in top command. Here is a screenshot of my top just after restarting PHP-FPM. Everything is normal: about 20 PHP-FPM processes, each consuming 5.5MB memory (0.3% of total). Here is the aged server right before restart of PHP-FPM (one day after the previous restart). Here, we still have about 25 PHP-FPM with double memory usage (10MB indicating 0.5% of total). Thus, the total memory used should be 600-700 MB. Then, why 1.6GB memory has been used?

  Read the article

• How to warehouse data that is not needed from MS SQL server

  - by I__

  I have been asked to truncate a large table in MS SQL Server 2008. The data is not needed but might be needed once every two years. It will NEVER have to be changed, only viewed. The question is, since I don't need the data on a day-to-day basis, what do I do with it to protect and back it up? Please keep in mind that I will need to have it accessible maybe once every two years, and it is FINE for us if the recovery process takes a few hours. The entire table is about 3 million rows and I need to truncate it to about 1 million rows.

  Read the article

• 'Memory read error',Sever hardware error?

  - by wss8848

  hello I got a error about my server which is running CentOS5.5. MCE 20 HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor CPU 1 BANK 8 TSC 6ab9ff9745f62 [at 2394 Mhz 9 days 1:50:52 uptime (unreliable)] MISC cf36ad0100081186 ADDR 203376500 MCG status: MCi status: MCi_MISC register valid MCi_ADDR register valid MCA: MEMORY CONTROLLER RD_CHANNELunspecified_ERR Transaction: Memory read error STATUS 8c0000400001009f MCGSTATUS 0 what is the matter? is memory card error or memory controller error?

  Read the article

• Column locking in innodb?

  - by mingyeow

  I know this sounds weird, but apparently one of my columns is locked. select * from table where type_id = 1 and updated_at < '2010-03-14' limit 1; select * from table where type_id = 3 and updated_at < '2010-03-14' limit 10; the first one would not finish running, while the second one completes smoothly. the only difference is the type_id Thanks in advance for your help - i have an urgent data job to finish, and this problem is driving me crazy

  Read the article

• Maintenance Plan Reporting - Append To File - Clean Up?

  - by Adam J.R. Erickson

  Background: (SQL Server 2005, Standard Ed.) I have a maintenance plan running backups, taking a full backup 1/day, and t-log every 15 minutes. I have it set to create a text file report of each run, but that creates A LOT of files on the file server. These are hard to sort through, which makes them less useful. Question: There is an option in "Reporting and Logging" settings for appending all logs together, but how do you clean this out? If you're appending to the same log file every time, how should you make sure this file doesn't grow indefinitely? Is there a build-in function to clean out portions of appended logs like there is for cleaning out individual old log files?

  Read the article

• How do I restore a database on a remote SQL server 2005 from a local backup?

  - by MatsT

  I have been given access to (parts of) a remote SQL Server 2005 with SQL Server authentication in order to be able to make changes to a database without involving other people who is not working on the project. The database have been created on my local machine. Is there any way to restore the remote database from a backup file on my local computer? I do not currently have access to the filesystem on the remote server. EDIT: To clarify, the access I have is that i can log in to the server via the SQL Server Management Studio. I have one connection to my local database server and one connection to the remote server. What I basically want to do is copy the database from one connection to the other.

  Read the article

• How do I restore a database on a remote SQL server 2005 from a local backup?

  - by MatsT

  I have been given access to (parts of) a remote SQL Server 2005 with SQL Server authentication in order to be able to make changes to a database without involving other people who is not working on the project. The database have been created on my local machine. Is there any way to restore the remote database from a backup file on my local computer? I do not currently have access to the filesystem on the remote server. EDIT: To clarify, the access I have is that i can log in to the server via the SQL Server Management Studio. I have one connection to my local database server and one connection to the remote server. What I basically want to do is copy the database from one connection to the other.

  Read the article

• Apache consuming large memory with Apache Mobile Filter

  - by VarunGupta

  I installed and configured Apache Mobile Filter module for Apache to redirect users to mobile version of our website, depending on their user agents. I configured the module to use WURFL. But as soon as I start Apache it hogs a large amount of memory (without any in-coming web requests): Resident Memory: 300 to 400 MB Virtual Memory: 300 to 650 MB Without this module, Apache was consuming much lesser memory (4 to 10 MB). What could be the reason here?

  Read the article

• ft_stopword_file not picked up

  - by Alex Holsgrove

  I have a VPS server with a company called Webfusion. I want to remove some or all of the FULLTEXT stopwords because some specific words needs to be searchable with my DB content. I opened /etc/mysql/my.cnf and added the line ft_stopword_file="". I restarted the mysql service, ran a repair table and then tried my MATCH query with no success. I ran SHOW VARIABLES LIKE 'ft_%' and it simply shows (built-in) next to the stopword file. I am running WAMP on my workstation, and whilst I realise this isn't configured the same as a commercial VPS, the above method worked just fine. Couple someone please offer some guidance?

  Read the article

• Multiple columns in a single index versus multiple indexes

  - by Tim Coker

  The short version of my question is what's the difference between three indexes each indexing a single column and one index indexing three columns. Background follows. I'm primarily a programmer but have to do DBA work because we don't have a DBA. I'm evaluating our indexes versus the queries run against a particular table. The table as 3 columns that I'm often filtering against or getting the max value of. Most of the time the queries look like select max(col_a) from table where col_b = 'avalue' or select col_c from table where col_b = 'avalue' and col_a = 'anothervalue' All columns are independently indexed. My question is would I see any difference if I had an index that indexed col_b and col_a together since they can appear in a where clause together?

  Read the article

• What are the best tools for modeling a pre-existing SQL database structure?

  - by Ejoso

  I have a MS SQL database that has been running strong for 10+ years. I'd like to diagram the database structure, without spending hours laying it all out in Visio or something similar... I've seen nice models diagrammed before, but I have no idea how they were created. From what I've seen - those models were created in advance of the database itself to assist in clarifying the relationships... but my database already exists! Anyone have any suggestions for tools that would work, or methods I could employ to tease out a nice clean document describing my database structure? Thanks in advance!

  Read the article

< Previous Page | 114 115 116 117 118 119 120 121 122 123 124 125  | Next Page >