Search Results

Search found 4547 results on 182 pages for 'haskell io'.

Page 48/182 | < Previous Page | 44 45 46 47 48 49 50 51 52 53 54 55  | Next Page >

  • What is the fastest way for reading huge files in Delphi?

    - by dummzeuch
    My program needs to read chunks from a huge binary file with random access. I have got a list of offsets and lengths which may have several thousand entries. The user selects an entry and the program seeks to the offset and reads length bytes. The program internally uses a TMemoryStream to store and process the chunks read from the file. Reading the data is done via a TFileStream like this: FileStream.Position := Offset; MemoryStream.CopyFrom(FileStream, Size); This works fine but unfortunately it becomes increasingly slower as the files get larger. The file size starts at a few megabytes but frequently reaches several tens of gigabytes. The chunks read are around 100 kbytes in size. The file's content is only read by my program. It is the only program accessing the file at the time. Also the files are stored locally so this is not a network issue. I am using Delphi 2007 on a Windows XP box. What can I do to speed up this file access?

    Read the article

  • How to store a scaleable sized extensible event log?

    - by firoso
    Hello everyone! I've been contemplating writing a simple "event log" that takes a paramater list and stores event messages in a log file, trouble is, I forsee this file growing to be rather large (assume 1M entries or more) the question is, how can I implement this system without pulling teeth, I know that SQL would be a possible way to go. XML would be ideal but not really practical for scaleability if i'm not going nuts. Example Log Entry -----Time Date-------- ---------Sender----------------------- ---------Tags---------- --Message---------- 12/24/2008 24:00:00 $DOMAIN\SYSTEM\Application$ :Trivial: :Notification: It's Christmas in 1s

    Read the article

  • How to append text into text file dynamically

    - by niraj deshmukh
    [12] key1=val1 key2=val2 key3=val3 key4=val4 key5=val5 [13] key1=val1 key2=val2 key3=val3 key4=val4 key5=xyz [14] key1=val1 key2=val2 key3=val3 key4=val4 key5=val5 I want to update key5=val5 where [13]. try { br = new BufferedReader(new FileReader(oldFileName)); bw = new BufferedWriter(new FileWriter(tmpFileName)); String line; while ((line = br.readLine()) != null) { System.out.println(line); if (line.contains("[13]")) { while (line.contains("key5")) { if (line.contains("key5")) { line = line.replace("key5", "key5= Val5"); bw.write(line+"\n"); } } } } } catch (Exception e) { return; } finally { try { if(br != null) br.close(); } catch (IOException e) { // } try { if(bw != null) bw.close(); } catch (IOException e) { // } }

    Read the article

  • MFC: Reading entire file to buffer...

    - by deostroll
    I've meddled with some code but I am unable to read the entire file properly...a lot of junk gets appended to the output. How do I fix this? // wmfParser.cpp : Defines the entry point for the console application. // #include "stdafx.h" #include "wmfParser.h" #include <cstring> #ifdef _DEBUG #define new DEBUG_NEW #endif // The one and only application object CWinApp theApp; using namespace std; int _tmain(int argc, TCHAR* argv[], TCHAR* envp[]) { int nRetCode = 0; // initialize MFC and print and error on failure if (!AfxWinInit(::GetModuleHandle(NULL), NULL, ::GetCommandLine(), 0)) { // TODO: change error code to suit your needs _tprintf(_T("Fatal Error: MFC initialization failed\n")); nRetCode = 1; } else { // TODO: code your application's behavior here. CFile file; CFileException exp; if( !file.Open( _T("c:\\sample.txt"), CFile::modeRead, &exp ) ){ exp.ReportError(); cout<<'\n'; cout<<"Aborting..."; system("pause"); return 0; } ULONGLONG dwLength = file.GetLength(); cout<<"Length of file to read = " << dwLength << '\n'; /* BYTE* buffer; buffer=(BYTE*)calloc(dwLength, sizeof(BYTE)); file.Read(buffer, 25); char* str = (char*)buffer; cout<<"length of string : " << strlen(str) << '\n'; cout<<"string from file: " << str << '\n'; */ char str[100]; file.Read(str, sizeof(str)); cout << "Data : " << str <<'\n'; file.Close(); cout<<"File was closed\n"; //AfxMessageBox(_T("This is a test message box")); system("pause"); } return nRetCode; }

    Read the article

  • Search a string in a file and write the matched lines to another file in Java

    - by Geeta
    For searching a string in a file and writing the lines with matched string to another file it takes 15 - 20 mins for a single zip file of 70MB(compressed state). Is there any ways to minimise it. my source code: getting Zip file entries zipFile = new ZipFile(source_file_name); entries = zipFile.entries(); while (entries.hasMoreElements()) { ZipEntry entry = (ZipEntry)entries.nextElement(); if (entry.isDirectory()) { continue; } searchString(Thread.currentThread(),entry.getName(), new BufferedInputStream (zipFile.getInputStream(entry)), Out_File, search_string, stats); } zipFile.close(); Searching String public void searchString(Thread CThread, String Source_File, BufferedInputStream in, File outfile, String search, String stats) throws IOException { int count = 0; int countw = 0; int countl = 0; String s; String[] str; BufferedReader br2 = new BufferedReader(new InputStreamReader(in)); System.out.println(CThread.currentThread()); while ((s = br2.readLine()) != null) { str = s.split(search); count = str.length - 1; countw += count; //word count if (s.contains(search)) { countl++; //line count WriteFile(CThread,s, outfile.toString(), search); } } br2.close(); in.close(); } -------------------------------------------------------------------------------- public void WriteFile(Thread CThread,String line, String out, String search) throws IOException { BufferedWriter bufferedWriter = null; System.out.println("writre thread"+CThread.currentThread()); bufferedWriter = new BufferedWriter(new FileWriter(out, true)); bufferedWriter.write(line); bufferedWriter.newLine(); bufferedWriter.flush(); } Please help me. Its really taking 40 mins for 10 files using threads and 15 - 20 mins for a single file of 70MB after being compressed. Any ways to minimise the time.

    Read the article

  • Read whole ASCII file into C++ std::string

    - by Arrieta
    Hello, I need to read a whole file into memory and place it in a C++ std::string. If I were to read it into a char, the answer would be very simple: std::ifstream t; int lenght; t.open("file.txt", "r"); // open input file t.seekg(0, std::ios::end); // go to the end length = t.tellg(); // report location (this is the lenght) t.seekg(0, std::ios::beg); // go back to the beginning buffer = new char[length]; // allocate memory for a buffer of appropriate dimension t.read(buffer, length); // read the whole file into the buffer t.close(); // close file handle // ... do stuff with buffer here ... Now, I want to do the exact same thing, but using a std::string instead of a char. I want to avoid loops, i. e., I don't want to: std::ifstream t; t.open("file.txt", "r"); std::string buffer; std::string line; while(t){ std::getline(t, line); // ... append line to buffer and go on } t.close() any ideas?

    Read the article

  • Error copying file from app bundle

    - by Michael Chen
    I used the FireFox add-on SQLite Manager, created a database, which saved to my desktop as "DB.sqlite". I copied the file into my supporting files for the project. But when I run the app, immediately I get the error "Assertion failure in -[AppDelegate copyDatabaseIfNeeded], /Users/Mac/Desktop/Note/Note/AppDelegate.m:32 2014-08-19 23:38:02.830 Note[28309:60b] Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Failed to create writable database file with message 'The operation couldn’t be completed. (Cocoa error 4.)'.' First throw call stack: "... Here is the App Delegate Code where the error takes place -(void) copyDatabaseIfNeeded { NSFileManager *fileManager = [NSFileManager defaultManager]; NSError *error; NSString *dbPath = [self getDBPath]; BOOL success = [fileManager fileExistsAtPath:dbPath]; if (!success) { NSString *defaultDBPath = [[ [NSBundle mainBundle ] resourcePath] stringByAppendingPathComponent:@"DB.sqlite"]; success = [fileManager copyItemAtPath:defaultDBPath toPath:dbPath error:&error]; if (!success) NSAssert1(0, @"Failed to create writable database file with message '%@'.", [error localizedDescription]); } } I am very new to Sqlite, so I maybe I didn't create a database correctly in the FireFox Sqlite manager, or maybe I didn't "properly" copy the .sqlite file in? (I did check the target membership in the sqlite and it correctly has my project selected. Also, the .sqlite file names all match up perfectly.)

    Read the article

  • Most efficient way to write over file after reading

    - by Ryan McClure
    I'm reading in some data from a file, manipulating it, and then overwriting it to the same file. Until now, I've been doing it like so: open (my $inFile, $file) or die "Could not open $file: $!"; $retString .= join ('', <$inFile>); ... close ($inFile); open (my $outFile, $file) or die "Could not open $file: $!"; print $outFile, $retString; close ($inFile); However I realized I can just use the truncate function and open the file for read/write: open (my $inFile, '+<', $file) or die "Could not open $file: $!"; $retString .= join ('', <$inFile>); ... truncate $inFile, 0; print $inFile $retString; close ($inFile); I don't see any examples of this anywhere. It seems to work well, but am I doing it correctly? Is there a better way to do this?

    Read the article

  • Embarrassingly parallel workflow creates too many output files

    - by Hooked
    On a Linux cluster I run many (N > 10^6) independent computations. Each computation takes only a few minutes and the output is a handful of lines. When N was small I was able to store each result in a separate file to be parsed later. With large N however, I find that I am wasting storage space (for the file creation) and simple commands like ls require extra care due to internal limits of bash: -bash: /bin/ls: Argument list too long. Each computation is required to run through a qsub scheduling algorithm so I am unable to create a master program which simply aggregates the output data to a single file. The simple solution of appending to a single fails when two programs finish at the same time and interleave their output. I have no admin access to the cluster, so installing a system-wide database is not an option. How can I collate the output data from embarrassingly parallel computation before it gets unmanageable?

    Read the article

  • Homemade fstat to get file size, always return 0 length.

    - by Fred
    Hello, I am trying to use my own function to get the file size from a file. I'll use this to allocate memory for a data structure to hold the information on the file. The file size function looks like this: long fileSize(FILE *fp){ long start; fflush(fp); rewind(fp); start = ftell(fp); return (fseek(fp, 0L, SEEK_END) - start); } Any ideas what I'm doing wrong here?

    Read the article

  • Is there a concise way to create an InputSupplier for an InputStream in Google Guava?

    - by Fabian Steeg
    There are a few factory methods in Google Guava to create InputSuppliers, e.g. from a byte[]: ByteStreams.newInputStreamSupplier(bytes); Or from a File: Files.newInputStreamSupplier(file); Is there a similar way to to create an InputSupplier for a given InputStream? That is, a way that's more concise than an anonymous class: new InputSupplier<InputStream>() { public InputStream getInput() throws IOException { return inputStream; } }; Background: I'd like to use InputStreams with e.g. Files.copy(...) or ByteStreams.equal(...).

    Read the article

  • c++ File input/output

    - by Myx
    Hi: I am trying to read from a file using fgets and sscanf. In my file, I have characters on each line of the while which I wish to put into a vector. So far, I have the following: FILE *fp; fp = fopen(filename, "r"); if(!fp) { fprintf(stderr, "Unable to open file %s\n", filename); return 0; } // Read file int line_count = 0; char buffer[1024]; while(fgets(buffer, 1023, fp)) { // Increment line counter line_count++; char *bufferp = buffer; ... while(*bufferp != '\n') { char *tmp; if(sscanf(bufferp, "%c", tmp) != 1) { fprintf(stderr, "Syntax error reading axiom on " "line %d in file %s\n", line_count, filename); return 0; } axiom.push_back(tmp); printf("put %s in axiom vector\n", axiom[axiom.size()-1]); // increment buffer pointer bufferp++; } } my axiom vector is defined as vector<char *> axiom;. When I run my program, I get a seg fault. It happens when I do the sscanf. Any suggestions on what I'm doing wrong?

    Read the article

  • Java I/O: How to append to an already existing text file.

    - by Joe
    Hi I am having no problem writing to or appending to a file, the only problem is that as soon as I quit the program and then run it again, it creates a new file overwriting my original file. This is a problem, as I am using the text file to keep a running tally. Is there a way to get an already created text file as an object and then append to it? Thanks in advance.

    Read the article

  • Batch backup a harddrive without modifying access times C#

    - by johnathan-doena
    I'm trying to write a simple program that will backup my flash drive. I want it to work automatically and silently in the background, and I also want it to be as quick as possible. The thing is, resetting all the access times is useless to me, and something I want to avoid. I know I can read the access times and set them back, but I bet it will fail one day in the future. It would be much simpler to read the files without ever changing it. Also, what is the fastest way to do this? What differences would there be between, say, a flash drive and an external hard drive. I am writing this in C#, as it is the simplest way to do it and it will probably last more generations of Windows..

    Read the article

  • fortran error I/O

    - by jpcgandre
    I get this error when compiling: forrtl: severe (256): unformatted I/O to unit open for formatted transfers, unit 27, file C:\Abaqus_JOBS\w.txt The error occurs in the beginning of the analysis. At the start, the file w.txt is created but is empty. The error may be related to the fact that I want to read from an empty file. My code is: OPEN(27, FILE = "C:/Abaqus_JOBS/w.txt", status = "UNKNOWN") READ(27, *, iostat=stat) w IF (stat .NE. 0) CALL del_file(27, stat) SUBROUTINE del_file(uFile, stat) IMPLICIT NONE INTEGER uFile, stat C If the unit is not open, stat will be non-zero CLOSE(unit=uFile, status='delete', iostat=stat) END SUBROUTINE Ref: Close multiple files If you agree with my opion about the cause of the error, is there a way to solve it? Thanks

    Read the article

  • best way to output a full precision double into a text file

    - by flevine100
    Hi, I need to use an existing text file to store some very precise values. When read back in, the numbers essentially need to be exactly equivalent to the ones that were originally written. Now, a normal person would use a binary file... for a number of reasons, that's not possible in this case. So... do any of you have a good way of encoding a double as a string of characters (aside from increasing the precision). My first thought was to cast the double to a char[] and write out the chars. I don't think that's going to work because some of the characters are not visible, produce sounds, and even terminate strings ('\0'... I'm talkin to you!) Thoughts?

    Read the article

  • Explained shell statement

    - by Mats Stijlaart
    The following statement will remove line numbers in a txt file: cat withLineNumbers.txt | sed 's/^.......//' >> withoutLineNumbers.txt The input file is created with the following statement (this one i understand): nl -ba input.txt >> withLineNumbers.txt I know the functionality of cat and i know the output is written to the 'withoutLineNumbers.txt' file. But the part of '| sed 's/^.......//'' is not really clear to me. Thanks for your time.

    Read the article

  • iphone file download not working

    - by Anonymous
    Hi, In my app I 'm first connecting to a web service, which in return sends a url for a file. I use the url to download the file and then display it on the new view. I get the correct URL but not able to download file from that location. I have another test app which will download file from the same location and it works like a charm. following is my code for webservice-file download. This is a snippet of the code where i 'm parsing the web service xml and then pass the result to NSData for file download. Any suggestions where am i going wrong -- I 'm referring to the following tutorials. Web Service PDF Viewer if ([elementName isEqualToString:@"PRHPdfResultsResult"]) { NSLog(soapResults); UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Report downloaded from:" message:soapResults delegate:self cancelButtonTitle:@"OK" otherButtonTitles:nil]; NSData *pdfData = [[NSData alloc] initWithContentsOfURL:[NSURL URLWithString:soapResults]]; //Store the Data locally as PDF File NSString *resourceDocPath = [[NSString alloc] initWithString:[[[[NSBundle mainBundle] resourcePath] stringByDeletingLastPathComponent] stringByAppendingPathComponent:@"Documents"]]; NSString *filePath = [resourceDocPath stringByAppendingPathComponent:@"myPDF.pdf"]; [pdfData writeToFile:filePath atomically:YES]; [alert show]; [alert release]; [soapResults setString:@""]; elementFound = FALSE; }

    Read the article

  • Foreach File in a Folder in Flash?

    - by msandbot
    Hey, I have an image slideshow program working right now and it takes in a folder of a hard coded in number of images. I would like to change this so that it can take in a folder and will display all of them no matter the number. Is there a way to do this in flash? I'm thinking something like the foreach loop in perl or other scripting language. It is possible to store then number of images in a text file but I also don't know how to read that in flash either. I'm working in actionscript 3. Any help would be greatly appreciated. Thanks -Mike

    Read the article

  • Efficient file buffering & scanning methods for large files in python

    - by eblume
    The description of the problem I am having is a bit complicated, and I will err on the side of providing more complete information. For the impatient, here is the briefest way I can summarize it: What is the fastest (least execution time) way to split a text file in to ALL (overlapping) substrings of size N (bound N, eg 36) while throwing out newline characters. I am writing a module which parses files in the FASTA ascii-based genome format. These files comprise what is known as the 'hg18' human reference genome, which you can download from the UCSC genome browser (go slugs!) if you like. As you will notice, the genome files are composed of chr[1..22].fa and chr[XY].fa, as well as a set of other small files which are not used in this module. Several modules already exist for parsing FASTA files, such as BioPython's SeqIO. (Sorry, I'd post a link, but I don't have the points to do so yet.) Unfortunately, every module I've been able to find doesn't do the specific operation I am trying to do. My module needs to split the genome data ('CAGTACGTCAGACTATACGGAGCTA' could be a line, for instance) in to every single overlapping N-length substring. Let me give an example using a very small file (the actual chromosome files are between 355 and 20 million characters long) and N=8 import cStringIO example_file = cStringIO.StringIO("""\ header CAGTcag TFgcACF """) for read in parse(example_file): ... print read ... CAGTCAGTF AGTCAGTFG GTCAGTFGC TCAGTFGCA CAGTFGCAC AGTFGCACF The function that I found had the absolute best performance from the methods I could think of is this: def parse(file): size = 8 # of course in my code this is a function argument file.readline() # skip past the header buffer = '' for line in file: buffer += line.rstrip().upper() while len(buffer) = size: yield buffer[:size] buffer = buffer[1:] This works, but unfortunately it still takes about 1.5 hours (see note below) to parse the human genome this way. Perhaps this is the very best I am going to see with this method (a complete code refactor might be in order, but I'd like to avoid it as this approach has some very specific advantages in other areas of the code), but I thought I would turn this over to the community. Thanks! Note, this time includes a lot of extra calculation, such as computing the opposing strand read and doing hashtable lookups on a hash of approximately 5G in size. Post-answer conclusion: It turns out that using fileobj.read() and then manipulating the resulting string (string.replace(), etc.) took relatively little time and memory compared to the remainder of the program, and so I used that approach. Thanks everyone!

    Read the article

  • C++ file input/output search

    - by Brian J
    Hi I took the following code from a program I'm writing to check a user generated string against a dictionary as well as other validation. My problem is that although my dictionary file is referenced correctly,the program gives the default "no dictionary found".I can't see clearly what I'm doing in error here,if anyone has any tips or pointers it would be appreciated, Thanks. //variables for checkWordInFile #define gC_FOUND 99 #define gC_NOT_FOUND -99 // static bool certifyThat(bool condition, const char* error) { if(!condition) printf("%s", error); return !condition; } //method to validate a user generated password following password guidelines. void validatePass() { FILE *fptr; char password[MAX+1]; int iChar,iUpper,iLower,iSymbol,iNumber,iTotal,iResult,iCount; //shows user password guidelines printf("\n\n\t\tPassword rules: "); printf("\n\n\t\t 1. Passwords must be at least 9 characters long and less than 15 characters. "); printf("\n\n\t\t 2. Passwords must have at least 2 numbers in them."); printf("\n\n\t\t 3. Passwords must have at least 2 uppercase letters and 2 lowercase letters in them."); printf("\n\n\t\t 4. Passwords must have at least 1 symbol in them (eg ?, $, £, %)."); printf("\n\n\t\t 5. Passwords may not have small, common words in them eg hat, pow or ate."); //gets user password input get_user_password: printf("\n\n\t\tEnter your password following password rules: "); scanf("%s", &password); iChar = countLetters(password,&iUpper,&iLower,&iSymbol,&iNumber,&iTotal); iUpper = countLetters(password,&iUpper,&iLower,&iSymbol,&iNumber,&iTotal); iLower =countLetters(password,&iUpper,&iLower,&iSymbol,&iNumber,&iTotal); iSymbol =countLetters(password,&iUpper,&iLower,&iSymbol,&iNumber,&iTotal); iNumber = countLetters(password,&iUpper,&iLower,&iSymbol,&iNumber,&iTotal); iTotal = countLetters(password,&iUpper,&iLower,&iSymbol,&iNumber,&iTotal); if(certifyThat(iUpper >= 2, "Not enough uppercase letters!!!\n") || certifyThat(iLower >= 2, "Not enough lowercase letters!!!\n") || certifyThat(iSymbol >= 1, "Not enough symbols!!!\n") || certifyThat(iNumber >= 2, "Not enough numbers!!!\n") || certifyThat(iTotal >= 9, "Not enough characters!!!\n") || certifyThat(iTotal <= 15, "Too many characters!!!\n")) goto get_user_password; iResult = checkWordInFile("dictionary.txt", password); if(certifyThat(iResult != gC_FOUND, "Password contains small common 3 letter word/s.")) goto get_user_password; iResult = checkWordInFile("passHistory.txt",password); if(certifyThat(iResult != gC_FOUND, "Password contains previously used password.")) goto get_user_password; printf("\n\n\n Your new password is verified "); printf(password); //writing password to passHistroy file. fptr = fopen("passHistory.txt", "w"); // create or open the file for( iCount = 0; iCount < 8; iCount++) { fprintf(fptr, "%s\n", password[iCount]); } fclose(fptr); printf("\n\n\n"); system("pause"); }//end validatePass method int checkWordInFile(char * fileName,char * theWord){ FILE * fptr; char fileString[MAX + 1]; int iFound = -99; //open the file fptr = fopen(fileName, "r"); if (fptr == NULL) { printf("\nNo dictionary file\n"); printf("\n\n\n"); system("pause"); return (0); // just exit the program } /* read the contents of the file */ while( fgets(fileString, MAX, fptr) ) { if( 0 == strcmp(theWord, fileString) ) { iFound = -99; } } fclose(fptr); return(0); }//end of checkwORDiNFile

    Read the article

< Previous Page | 44 45 46 47 48 49 50 51 52 53 54 55  | Next Page >