Search Results

Search found 12 results on 1 pages for 'tfilestream'.

Page 1/1 | 1 

  • What is the fastest way for reading huge files in Delphi?

    - by dummzeuch
    My program needs to read chunks from a huge binary file with random access. I have got a list of offsets and lengths which may have several thousand entries. The user selects an entry and the program seeks to the offset and reads length bytes. The program internally uses a TMemoryStream to store and process the chunks read from the file. Reading the data is done via a TFileStream like this: FileStream.Position := Offset; MemoryStream.CopyFrom(FileStream, Size); This works fine but unfortunately it becomes increasingly slower as the files get larger. The file size starts at a few megabytes but frequently reaches several tens of gigabytes. The chunks read are around 100 kbytes in size. The file's content is only read by my program. It is the only program accessing the file at the time. Also the files are stored locally so this is not a network issue. I am using Delphi 2007 on a Windows XP box. What can I do to speed up this file access?

    Read the article

  • How to use a TFileStream to read 2D matrices into dynamic array?

    - by Robert Frank
    I need to read a large (2000x2000) matrix of binary data from a file into a dynamic array with Delphi 2010. I don't know the dimensions until run-time. I've never read raw data like this, and don't know IEEE so I'm posting this to see if I'm on track. I plan to use a TFileStream to read one row at a time. I need to be able to read as many of these formats as possible: 16-bit two's complement binary integer 32-bit two's complement binary integer 64-bit two's complement binary integer IEEE single precision floating-point For 32-bit two's complement, I'm thinking something like the code below. Changing to Int64 and Int16 should be straight forward. How can I read the IEEE? Am I on the right track? Any suggestions on this code, or how to elegantly extend it for all 4 data types above? Since my post-processing will be the same after reading this data, I guess I'll have to copy the matrix into a common format when done. I have no problem just having four procedures (one for each data type) like the one below, but perhaps there's an elegant way to use RTTI or buffers and then move()'s so that the same code works for all 4 datatypes? Thanks! type TRowData = array of Int32; procedure ReadMatrix; var Matrix: array of TRowData; NumberOfRows: Cardinal; NumberOfCols: Cardinal; CurRow: Integer; begin NumberOfRows := 20; // not known until run time NumberOfCols := 100; // not known until run time SetLength(Matrix, NumberOfRows); for CurRow := 0 to NumberOfRows do begin SetLength(Matrix[CurRow], NumberOfCols); FileStream.ReadBuffer(Matrix[CurRow], NumberOfCols * SizeOf(Int32)) ); end; end;

    Read the article

  • Delphi7 - How can i copy a file that is being written to

    - by Simon
    I have an application that logs information to a daily text file every second on a master PC. A Slave PC on the network using the same application would like to copy this text file to its local drive. I can see there is going to be file access issues. These files should be no larger than 30-40MB each. the network will be 100MB ethernet. I can see there is potential for the copying process to take longer than 1 second meaning the logging PC will need to open the file for writing while it is being read. What is the best method for the file writing(logging) and file copying procedures? I know there is the standard Windows CopyFile() procedure, however this has given me file access problems. There is also TFileStream using the fmShareDenyNone flag, but this also very occasionally gives me an access problem too (like 1 per week). What is this the best way of accomplishing this task? My current File Logging: procedure FSWriteline(Filename,Header,s : String); var LogFile : TFileStream; line : String; begin if not FileExists(filename) then begin LogFile := TFileStream.Create(FileName, fmCreate or fmShareDenyNone); try LogFile.Seek(0,soFromEnd); line := Header + #13#10; LogFile.Write(line[1],Length(line)); line := s + #13#10; LogFile.Write(line[1],Length(line)); finally logfile.Free; end; end else begin line := s + #13#10; Logfile:=tfilestream.Create(Filename,fmOpenWrite or fmShareDenyNone); try logfile.Seek(0,soFromEnd); Logfile.Write(line[1], length(line)); finally Logfile.free; end; end; end; My file copy procedure: procedure DoCopy(infile, Outfile : String); begin ForceDirectories(ExtractFilePath(outfile)); //ensure folder exists if FileAge(inFile) = FileAge(OutFile) then Exit; //they are the same modified time try { Open existing destination } fo := TFileStream.Create(Outfile, fmOpenReadWrite or fmShareDenyNone); fo.Position := 0; except { otherwise Create destination } fo := TFileStream.Create(OutFile, fmCreate or fmShareDenyNone); end; try { open source } fi := TFileStream.Create(InFile, fmOpenRead or fmShareDenyNone); try cnt:= 0; fi.Position := cnt; max := fi.Size; {start copying } Repeat dod := BLOCKSIZE; // Block size if cnt+dod>max then dod := max-cnt; if dod>0 then did := fo.CopyFrom(fi, dod); cnt:=cnt+did; Percent := Round(Cnt/Max*100); until (dod=0) finally fi.free; end; finally fo.free; end; end;

    Read the article

  • Delphi - Capture stdout and stderr output from statically linked MSVC++ compiled DLL

    - by Alan G.
    I have been trying to capture stdout and stderr output from a DLL compiled in MSVC++ that my Delphi app statically links to, but so far have been unsuccessful. procedure Test; var fs: TFileStream; begin fs := TFileStream.Create('C:\temp\output.log', fmCreate or fmShareDenyWrite); SetStdHandle(STD_OUTPUT_HANDLE, fs.Handle); SetStdHandle(STD_ERROR_HANDLE, fs.Handle); dllFunc(0); // Writes to stdout in MSVC++ console app, but not here // fs.Length is always zero fs.Free; end; Thought I was on the right track, but it does not work. Is SetStdHandle() enough? Is TFileStream the right thing to use here? Am I using TFileStream properly for SetStdHandle()? Is it possible that the DLL sets its stdout/stderr handles when the app loads? If so, where is the best place to use SetStdHandle() or equivalent? Any help would be appreciated.

    Read the article

  • delphi - how to get the progress of socket.SendStream ?

    - by ass
    hi all ! I use dephi7's client/server socket components to send a file with tfilestream from client to server. I'am able to get the progress of file received at server side, but at client side, how do i get the progress of the file sent ? this is how I send the file: fstream:=tfilestream.Create(opendialog1.FileName,fmOpenRead); clientsocket1.Socket.SendStream(fstream); Thanks and appreciate for any help.

    Read the article

  • Is Valid IMAGE_DOS_SIGNATURE

    - by iira
    I want to check a file has a valid IMAGE_DOS_SIGNATURE (MZ) function isMZ(FileName : String) : boolean; var Signature: Word; fexe: TFileStream; begin result:=false; try fexe := TFileStream.Create(FileName, fmOpenRead or fmShareDenyNone); fexe.ReadBuffer(Signature, SizeOf(Signature)); if Signature = $5A4D { 'MZ' } then result:=true; finally fexe.free; end; end; I know I can use some code in Windows unit to check the IMAGE_DOS_SIGNATURE. The problem is I want the fastest way to check IMAGE_DOS_SIGNATURE (for a big file). I need your some suggestion about my code or maybe a new code? Thanks

    Read the article

  • Indy IdSMTP and attachments in Thunderbird

    - by Lobuno
    Hello! Using the latest snapshot of Indy tiburon on D2010. A very simple project like: var stream: TFileStream; (s is TidSMTP and m is TidMessage) begin s.Connect; Stream := TFileStream.Create('c:\Test.zip', fmOpenRead or fmShareExclusive); try with TIdAttachmentMemory.Create(m.MessageParts, Stream) do begin ContentType := 'application/x-zip-compressed'; Name := ExtractFilePath('C:\'); //' FileName := 'Test.zip'; end; finally FreeAndNil(Stream); end; s.Send(m); s.Disconnect(); end; Everything works Ok in Outlook, The bat!, OE, yahoo, etc... but in Thunderbird the attachment is not shown. Looking at the source of the message in Thunderbird, the attachment is there. The only difference I can find between messages send by indy and other clients is that Indy messages have this order: Content-Type: multipart/mixed; boundary="Z\=_7oeC98yIhktvxiwiDTVyhv9R9gwkwT1" MIME-Version: 1.0 while any other clients have the order: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="Z\=_7oeC98yIhktvxiwiDTVyhv9R9gwkwT1" Don't know if THAT is the source of the problem, but if so: is this a bug on Thunderbird or is this a problem with indy which "malforms" the headers of the messages? Is this order a problem? Does that matter anyway?

    Read the article

  • Delphi - Read File To StringList, then delete and write back to file.

    - by Jkraw90
    I'm currently working on a program to generate the hashes of files, in Delphi 2010. As part of this I have a option to create User Presets, e.g. pre-defined choice of hashing algo's which the user can create/save/delete. I have the create and load code working fine. It uses a ComboBox and loads from a file "fhpre.ini", inside this file is the users presets stored in format of:- PresetName PresetCode (a 12 digit string using 0 for don't hash and 1 for do) On application loading it loads the data from this file into the ComboBox and an Array with the ItemIndex of ComboBox matching the corrisponding correct string of 0's and 1's in the Array. Now I need to implement a feature to have the user delete a preset from the list. So far my code is as follows, procedure TForm1.Panel23Click(Sender : TObject); var fil : textfile; contents : TStringList; x,i : integer; filline : ansistring; filestream : TFileStream; begin //Start Procedure //Load data into StringList contents := TStringList.Create; fileStream := TFileStream.Create((GetAppData+'\RFA\fhpre.ini'), fmShareDenyNone); Contents.LoadFromStream(fileStream); fileStream.Destroy(); //Search for relevant Preset i := 0; if ComboBox4.Text <> Contents[i] then begin Repeat i := i + 1; Until ComboBox4.Text = Contents[i]; end; contents.Delete(i); //Delete Relevant Preset Name contents.Delete(i); //Delete Preset Digit String //Write StringList back to file. AssignFile(fil,(GetAppData+'\RFA\fhpre.ini')); ReWrite(fil); for i := 0 to Contents.Count -1 do WriteLn(Contents[i]); CloseFile(fil); Contents.Free; end; However if this is run, I get a 105 error when it gets to the WriteLn section. I'm aware that the code isn't great, for example doesn't have checks for presets with same name, but that will come, I want to get the base code working first then can tweak and add extra checks etc. Any help would be appreciated.

    Read the article

  • Delphi: Alternative to using Assign/ReadLn for text file reading

    - by Ian Boyd
    i want to process a text file line by line. In the olden days i loaded the file into a StringList: slFile := TStringList.Create(); slFile.LoadFromFile(filename); for i := 0 to slFile.Count-1 do begin oneLine := slFile.Strings[i]; //process the line end; Problem with that is once the file gets to be a few hundred megabytes, i have to allocate a huge chunk of memory; when really i only need enough memory to hold one line at a time. (Plus, you can't really indicate progress when you the system is locked up loading the file in step 1). The i tried using the native, and recommended, file I/O routines provided by Delphi: var f: TextFile; begin Assign(filename, f); while ReadLn(f, oneLine) do begin //process the line end; Problem withAssign is that there is no option to read the file without locking (i.e. fmShareDenyNone). The former stringlist example doesn't support no-lock either, unless you change it to LoadFromStream: slFile := TStringList.Create; stream := TFileStream.Create(filename, fmOpenRead or fmShareDenyNone); slFile.LoadFromStream(stream); stream.Free; for i := 0 to slFile.Count-1 do begin oneLine := slFile.Strings[i]; //process the line end; So now even though i've gained no locks being held, i'm back to loading the entire file into memory. Is there some alternative to Assign/ReadLn, where i can read a file line-by-line, without taking a sharing lock? i'd rather not get directly into Win32 CreateFile/ReadFile, and having to deal with allocating buffers and detecting CR, LF, CRLF's. i thought about memory mapped files, but there's the difficulty if the entire file doesn't fit (map) into virtual memory, and having to maps views (pieces) of the file at a time. Starts to get ugly. i just want Assign with fmShareDenyNone!

    Read the article

  • Delphi: Alternative to using Reset/ReadLn for text file reading

    - by Ian Boyd
    i want to process a text file line by line. In the olden days i loaded the file into a StringList: slFile := TStringList.Create(); slFile.LoadFromFile(filename); for i := 0 to slFile.Count-1 do begin oneLine := slFile.Strings[i]; //process the line end; Problem with that is once the file gets to be a few hundred megabytes, i have to allocate a huge chunk of memory; when really i only need enough memory to hold one line at a time. (Plus, you can't really indicate progress when you the system is locked up loading the file in step 1). The i tried using the native, and recommended, file I/O routines provided by Delphi: var f: TextFile; begin Reset(f, filename); while ReadLn(f, oneLine) do begin //process the line end; Problem withAssign is that there is no option to read the file without locking (i.e. fmShareDenyNone). The former stringlist example doesn't support no-lock either, unless you change it to LoadFromStream: slFile := TStringList.Create; stream := TFileStream.Create(filename, fmOpenRead or fmShareDenyNone); slFile.LoadFromStream(stream); stream.Free; for i := 0 to slFile.Count-1 do begin oneLine := slFile.Strings[i]; //process the line end; So now even though i've gained no locks being held, i'm back to loading the entire file into memory. Is there some alternative to Assign/ReadLn, where i can read a file line-by-line, without taking a sharing lock? i'd rather not get directly into Win32 CreateFile/ReadFile, and having to deal with allocating buffers and detecting CR, LF, CRLF's. i thought about memory mapped files, but there's the difficulty if the entire file doesn't fit (map) into virtual memory, and having to maps views (pieces) of the file at a time. Starts to get ugly. i just want Reset with fmShareDenyNone!

    Read the article

  • How to convert Big Endian and how to flip the highest bit?

    - by Robert Frank
    I am using a TStream to read binary data (thanks to this post: http://stackoverflow.com/questions/2878180/how-to-use-a-tfilestream-to-read-2d-matrices-into-dynamic-array). My next problem is that the data is Big Endian. From my reading, the Swap() method is seemingly deprecated. How would I swap the types below? 16-bit two's complement binary integer 32-bit two's complement binary integer 64-bit two's complement binary integer IEEE single precision floating-point - Are IEEE affected by Big Endian? And, finally, since the data is unsigned, the creators of this dataset have stored the unsigned values as signed integers (excluding the IEEE). They instruct that one need only add an offset (2^15, 2^31, and 2^63) to recover the unsigned data. But, they note that flipping the most significant bit is the fastest way to do that. How does one efficiently flip the most significant bit of a 16, 32, or 64-bit integer? So, if the data on disk (16-bit) is "85 FB" - the desired result after reading the data and swapping and bit flipping would be 1531. Is there a way to accomplish the swapping and bit flipping with generics so it fits into the generic answer at the link above? Yes, kids, THIS is how scientific astronomical data is stored by NASA, ESO, and all professional astronomers. This FITS standard is considered by some to be one of the most successful standards ever created in its proliferation and flexibility!

    Read the article

  • Delphi - Using DeviceIoControl passing IOCTL_DISK_GET_LENGTH_INFO to get flash media physical size (Not Partition)

    - by SuicideClutchX2
    Alright this is the result of a couple of other questions. It appears I was doing something wrong with the suggestions and at this point have come up with an error when using the suggested API to get the media size. Those new to my problem I am working at the physical disk level, not within the confines of a partition or file system. Here is the pastebin code for the main unit (Delphi 2009) - http://clutchx2.pastebin.com/iMnq8kSx Here is the application source and executable with a form built to output the status of whats going on - http://www.mediafire.com/?js8e6ci8zrjq0de Its probably easier to use the download, unless your just looking for problems within the code. I will also paste the code here. unit Main; interface uses Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms, Dialogs, StdCtrls; type TfrmMain = class(TForm) edtDrive: TEdit; lblDrive: TLabel; btnMethod1: TButton; btnMethod2: TButton; lblSpace: TLabel; edtSpace: TEdit; lblFail: TLabel; edtFail: TEdit; lblError: TLabel; edtError: TEdit; procedure btnMethod1Click(Sender: TObject); private { Private declarations } public { Public declarations } end; TDiskExtent = record DiskNumber: Cardinal; StartingOffset: Int64; ExtentLength: Int64; end; DISK_EXTENT = TDiskExtent; PDiskExtent = ^TDiskExtent; TVolumeDiskExtents = record NumberOfDiskExtents: Cardinal; Extents: array[0..0] of TDiskExtent; end; VOLUME_DISK_EXTENTS = TVolumeDiskExtents; PVolumeDiskExtents = ^TVolumeDiskExtents; var frmMain: TfrmMain; const FILE_DEVICE_DISK = $00000007; METHOD_BUFFERED = 0; FILE_ANY_ACCESS = 0; IOCTL_DISK_BASE = FILE_DEVICE_DISK; IOCTL_VOLUME_BASE = DWORD('V'); IOCTL_DISK_GET_LENGTH_INFO = $80070017; IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS = ((IOCTL_VOLUME_BASE shl 16) or (FILE_ANY_ACCESS shl 14) or (0 shl 2) or METHOD_BUFFERED); implementation {$R *.dfm} function GetLD(Drive: Char): Cardinal; var Buffer : String; begin Buffer := Format('\\.\%s:',[Drive]); Result := CreateFile(PChar(Buffer),GENERIC_READ Or GENERIC_WRITE,FILE_SHARE_READ,nil,OPEN_EXISTING,0,0); If Result = INVALID_HANDLE_VALUE Then begin Result := CreateFile(PChar(Buffer),GENERIC_READ,FILE_SHARE_READ,nil,OPEN_EXISTING,0,0); end; end; function GetPD(Drive: Byte): Cardinal; var Buffer : String; begin If Drive = 0 Then begin Result := INVALID_HANDLE_VALUE; Exit; end; Buffer := Format('\\.\PHYSICALDRIVE%d',[Drive]); Result := CreateFile(PChar(Buffer),GENERIC_READ Or GENERIC_WRITE,FILE_SHARE_READ,nil,OPEN_EXISTING,0,0); If Result = INVALID_HANDLE_VALUE Then begin Result := CreateFile(PChar(Buffer),GENERIC_READ,FILE_SHARE_READ,nil,OPEN_EXISTING,0,0); end; end; function GetPhysicalDiskNumber(Drive: Char): Byte; var LD : DWORD; DiskExtents : PVolumeDiskExtents; DiskExtent : TDiskExtent; BytesReturned : Cardinal; begin Result := 0; LD := GetLD(Drive); If LD = INVALID_HANDLE_VALUE Then Exit; Try DiskExtents := AllocMem(Max_Path); DeviceIOControl(LD,IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS,nil,0,DiskExtents,Max_Path,BytesReturned,nil); If DiskExtents^.NumberOfDiskExtents > 0 Then begin DiskExtent := DiskExtents^.Extents[0]; Result := DiskExtent.DiskNumber; end; Finally CloseHandle(LD); end; end; procedure TfrmMain.btnMethod1Click(Sender: TObject); var PD : DWORD; CardSize: Int64; BytesReturned: DWORD; CallSuccess: Boolean; begin PD := GetPD(GetPhysicalDiskNumber(edtDrive.Text[1])); If PD = INVALID_HANDLE_VALUE Then Begin ShowMessage('Invalid Physical Disk Handle'); Exit; End; CallSuccess := DeviceIoControl(PD, IOCTL_DISK_GET_LENGTH_INFO, nil, 0, @CardSize, SizeOf(CardSize), BytesReturned, nil); if not CallSuccess then begin edtError.Text := IntToStr(GetLastError()); edtFail.Text := 'True'; end else edtFail.Text := 'False'; CloseHandle(PD); end; end. I placed a second method button on the form so I can write a different set of code into the app if I feel like it. Only minimal error handling and safeguards are there is nothing that wasn't necessary for debugging this via source. I tried this on a Sony Memory Stick using a PSP as the reader because I cant find the adapter for using a duo in my machine. The target is an MS and half of my users use a PSP for a reader half dont. However this should work fine on SD cards and that is a secondary target for my work as well. I tried this on a usb memory card reader and several SD cards. Now that I have fixed my attempt I get an error returned. 50 ERROR_NOT_SUPPORTED The request is not supported. I have found an application that uses this API as well as alot of related functions for what I am trying todo. I am getting ready to look into it the application is called DriveImage and its source is here - http://sourceforge.net/projects/diskimage/ The only thing I have really noticed from that application is there use of TFileStream and using that to get a handle on the physical disk.

    Read the article

1