Search Results

Search found 12367 results on 495 pages for 'disk io'.

Page 72/495 | < Previous Page | 68 69 70 71 72 73 74 75 76 77 78 79  | Next Page >

  • c# Named Pipe Asynchronous Peeking

    - by KJ Tsanaktsidis
    Hey all, I need to find a way to be notified when a System.IO.Pipe.NamedPipeServerStream opened in asynchronous mode has more data available for reading on it- a WaitHandle would be ideal. I cannot simply use BeginRead() to obtain such a handle because it's possible that i might be signaled by another thread which wants to write to the pipe- so I have to release the lock on the pipe and wait for the write to be complete, and NamedPipeServerStream doesnt have a CancelAsync method. I also tried calling BeginRead(), then calling the win32 function CancelIO on the pipe if the thread gets signaled, but I don't think this is an ideal solution because if CancelIO is called just as data is arriving and being processed, it will be dropped- I still wish to keep this data, but process it at a later time, after the write. I suspect the win32 function PeekNamedPipe might be useful but i'd like to avoid having to continuously poll for new data with it. In the likley event that the above text is a bit unclear, here's roughly what i'd like to be able to do... NamedPipeServerStream pipe; ManualResetEvent WriteFlag; //initialise pipe lock (pipe) { //I wish this method existed WaitHandle NewDataHandle = pipe.GetDataAvailableWaithandle(); Waithandle[] BreakConditions = new Waithandle[2]; BreakConditions[0] = NewDataHandle; BreakConditions[1] = WriteFlag; int breakcode = WaitHandle.WaitAny(BreakConditions); switch (breakcode) { case 0: //do a read on the pipe break; case 1: //break so that we release the lock on the pipe break; } }

    Read the article

  • Java: Efficiency of the readLine method of the BufferedReader and possible alternatives

    - by Luhar
    We are working to reduce the latency and increase the performance of a process written in Java that consumes data (xml strings) from a socket via the readLine() method of the BufferedReader class. The data is delimited by the end of line separater (\n), and each line can be of a variable length (6KBits - 32KBits). Our code looks like: Socket sock = connection; InputStream in = sock.getInputStream(); BufferedReader inputReader = new BufferedReader(new InputStreamReader(in)); ... do { String input = inputReader.readLine(); // Executor call to parse the input thread in a seperate thread }while(true) So I have a couple of questions: Will the inputReader.readLine() method return as soon as it hits the \n character or will it wait till the buffer is full? Is there a faster of picking up data from the socket than using a BufferedReader? What happens when the size of the input string is smaller than the size of the Socket's receive buffer? What happens when the size of the input string is bigger than the size of the Socket's receive buffer? I am getting to grips (slowly) with Java's IO libraries, so any pointers are much appreciated. Thank you!

    Read the article

  • When downloading a file using FileStream, why does page error message refers to aspx page name, not

    - by StuperUser
    After building a filepath (path, below) in a string (I am aware of Path in System.IO, but am using someone else's code and do not have the opportunity to refactor it to use Path). I am using a FileStream to deliver the file to the user (see below): FileStream myStream = new FileStream(path, FileMode.Open, FileAccess.Read); long fileSize = myStream.Length; byte[] Buffer = new byte[(int)fileSize + 1]; myStream.Read(Buffer, 0, (int)myStream.Length); myStream.Close(); Response.ContentType = "application/csv"; Response.AddHeader("content-disposition", "attachment; filename=" + filename); Response.BinaryWrite(Buffer); Response.Flush(); Response.End(); I have seen from: http://stackoverflow.com/questions/736301/asp-net-how-to-stream-file-to-user reasons to avoid use of Response.End() and Response.Close(). I have also seen several articles about different ways to transmit files and have diagnosed and found a solution to the problem (https and http headers) with a colleague. However, the error message that was being displayed was not about access to the file at path, but the aspx file. Edit: Error message is: Internet Explorer cannot download MyPage.aspx from server.domain.tld Internet Explorer was not able to open this Internet site. The requested site is either unavailable or cannot be found. Please try again later. (page name and address anonymised) Why is this? Is it due to the contents of the file coming from the HTTP response .Flush() method rather than a file being accessed at its address?

    Read the article

  • Failure remediation strategy for File I/O

    - by Brett
    I'm doing buffered IO into a file, both read and write. I'm using fopen(), fseeko(), standard ANSI C file I/O functions. In all cases, I'm writing to a standard local file on a disk. How often do these file I/O operations fail, and what should the strategy be for failures? I'm not exactly looking for stats, but I'm looking for a general purpose statement on how far I should go to handle error conditions. For instance, I think everyone recognizes that malloc() could and probably will fail someday on some user's machine and the developer should check for a NULL being returned, but there is no great remediation strategy since it probably means the system is out of memory. At least, this seems to be the approach taken with malloc() on desktop systems, embedded systems are different. Likewise, is it worth reattempting a file I/O operation, or should I just consider a failure to be basically unrecoverable, etc. I would appreciate some code samples demonstrating proper usage, or a library guide reference that indicates how this is to be handled. Any other data is, of course, welcome.

    Read the article

  • Why is calling close() after fopen() not closing?

    - by Richard Morgan
    I ran across the following code in one of our in-house dlls and I am trying to understand the behavior it was showing: long GetFD(long* fd, const char* fileName, const char* mode) { string fileMode; if (strlen(mode) == 0 || tolower(mode[0]) == 'w' || tolower(mode[0]) == 'o') fileMode = string("w"); else if (tolower(mode[0]) == 'a') fileMode = string("a"); else if (tolower(mode[0]) == 'r') fileMode = string("r"); else return -1; FILE* ofp; ofp = fopen(fileName, fileMode.c_str()); if (! ofp) return -1; *fd = (long)_fileno(ofp); if (*fd < 0) return -1; return 0; } long CloseFD(long fd) { close((int)fd); return 0; } After repeated calling of GetFD with the appropriate CloseFD, the whole dll would no longer be able to do any file IO. I wrote a tester program and found that I could GetFD 509 times, but the 510th time would error. Using Process Explorer, the number of Handles did not increase. So it seems that the dll is reaching the limit for the number of open files; setting _setmaxstdio(2048) does increase the amount of times we can call GetFD. Obviously, the close() is working quite right. After a bit of searching, I replaced the fopen() call with: long GetFD(long* fd, const char* fileName, const char* mode) { *fd = (long)open(fileName, 2); if (*fd < 0) return -1; return 0; } Now, repeatedly calling GetFD/CloseFD works. What is going on here?

    Read the article

  • Parsing URL error

    - by user577875
    It didn't seem like there was a post about this, so here goes. I've been working on a simple app to grab my time table from my school, and get it on my phone. Currently I'm working on the port on android but I've hit an issue. I get the error: java.io.IOException: -1 error loading URL urladress. Code: public void updateTimeTable(){ //Get UID and Birthday SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(getBaseContext()); String uid = prefs.getString("uid", "000000"); String fods = prefs.getString("fodsdag", "000000"); //Set URL String url = "http://unv1.aalborg-stu.dk/cgi-bin/elevskema.pl?ugen=0&unavn=" + uid + "&fodsdag=" + fods; try { Document doc = Jsoup.connect(url).get(); Elements td = doc.getElementsByTag("td"); //ArrayList<String> tdArray = new ArrayList<String>(); // for (Element tds : td) { // String tdText = tds.text(); // tdArray.add(tdText); //} //String[] data = tdArray.toArray(new String[tdArray.size()]); } catch (IOException e ){ Log.e("Parser", "shite", e); } Context context = getApplicationContext(); CharSequence text = url; int duration = Toast.LENGTH_SHORT; Toast toast = Toast.makeText(context, text, duration); toast.show(); } I've commented some lines out to identify where the issue is, and it seems it's at the actual parsing. Anywho, screenshot of the error I get: Screenshot I got about 4 days worth of Java experience so forgive me if it's something silly. Best Regards

    Read the article

  • Package upgrade on Ubuntu raid server and grub setup issue

    - by RecNes
    I have remote Ubuntu 10.10 server running on raid system. I did package upgrade yesterday night for security reasons. During the upgrade, grub installation screen appeared and asked me which partition I wanted to install grub. Options are sda,sdb,md1 and md2. I decide to install them on both sda and sdb partitions. I wondering, was I make true decision? If machine get reboot is it can be boot up safely? You can find fdisk output and fstab mount points below: Fstab: proc /proc proc defaults 0 0 none /dev/pts devpts gid=5,mode=620 0 0 /dev/md0 none swap sw 0 0 /dev/md1 /boot ext3 defaults 0 0 /dev/md2 / ext3 defaults 0 0 Fdisk: Disk /dev/sda: 750.2 GB, 750156374016 bytes 255 heads, 63 sectors/track, 91201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00029bb5 Device Boot Start End Blocks Id System /dev/sda1 1 262 2102562 fd Linux raid autodetect /dev/sda2 263 295 265072+ fd Linux raid autodetect /dev/sda3 296 91201 730202445 fd Linux raid autodetect Disk /dev/md0: 2152 MB, 2152923136 bytes 2 heads, 4 sectors/track, 525616 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/md0 doesn't contain a valid partition table Disk /dev/md1: 271 MB, 271319040 bytes 2 heads, 4 sectors/track, 66240 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/md1 doesn't contain a valid partition table Disk /dev/md2: 747.7 GB, 747727224832 bytes 2 heads, 4 sectors/track, 182550592 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/md2 doesn't contain a valid partition table Disk /dev/sdb: 750.2 GB, 750156374016 bytes 255 heads, 63 sectors/track, 91201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00088969 Device Boot Start End Blocks Id System /dev/sdb1 1 262 2102562 fd Linux raid autodetect /dev/sdb2 263 295 265072+ fd Linux raid autodetect /dev/sdb3 296 91201 730202445 fd Linux raid autodetect

    Read the article

  • Should I use "Raid 5 + spare" or "Raid 6"?

    - by Trevor Boyd Smith
    What is "Raid 5 + Spare" (excerpt from User Manual, Sect 4.17.2, P.54): RAID5+Spare: RAID 5+Spare is a RAID 5 array in which one disk is used as spare to rebuild the system as soon as a disk fails (Fig. 79). At least four disks are required. If one physical disk fails, the data remains available because it is read from the parity blocks. Data from a failed disk is rebuilt onto the hot spare disk. When a failed disk is replaced, the replacement becomes the new hot spare. No data is lost in the case of a single disk failure, but if a second disk fails before the system can rebuild data to the hot spare, all data in the array will be lost. What is "Raid 6" (excerpt from User Manual, Sect 4.17.2, P.54): RAID6: In RAID 6, data is striped across all disks (minimum of four) and a two parity blocks for each data block (p and q in Fig. 80) is written on the same stripe. If one physical disk fails, the data from the failed disk can be rebuilt onto a replacement disk. This Raid mode can support up to two disk failures with no data loss. RAID 6 provides for faster rebuilding of data from a failed disk. Both "Raid 5 + spare" and "Raid 6" are SO similar ... I can't tell the difference. When would "Raid 5 + Spare" be optimal? And when would "Raid 6" be optimal"? The manual dumbs down the different raid with 5 star ratings. "Raid 5 + Spare" only gets 4 stars but "Raid 6" gets 5 stars. If I were to blindly trust the manual I would conclude that "Raid 6" is always better. Is "Raid 6" always better?

    Read the article

  • Atomic int writes on file

    - by Waneck
    Hello! I'm writing an application that will have to be able to handle many concurrent accesses to it, either by threads as by processes. So no mutex'es or locks should be applied to this. To make the use of locks go down to a minimum, I'm designing for the file to be "append-only", so all data is first appended to disk, and then the address pointing to the info it has updated, is changed to refer to the new one. So I will need to implement a small lock system only to change this one int so it refers to the new address. How is the best way to do it? I was thinking about maybe putting a flag before the address, that when it's set, the readers will use a spin lock until it's released. But I'm afraid that it isn't at all atomic, is it? e.g. a reader reads the flag, and it is unset on the same time, a writer writes the flag and changes the value of the int the reader may read an inconsistent value! I'm looking for locking techniques but all I find is either for thread locking techniques, or to lock an entire file, not fields. Is it not possible to do this? How do append-only databases handle this? Thanks! Cauê

    Read the article

  • best practices question: How to save a collection of images and a java object in a single file? File

    - by Richard
    Hi all, I am making a java program that has a collection of flash-card like objects. I store the objects in a jtree composed of defaultmutabletreenodes. Each node has a user object attached to it with has a few string/native data type parameters. However, i also want each of these objects to have an image (typical formats, jpg, png etc). I would like to be able to store all of this information, including the images and the tree data to the disk in a single file so the file can be transferred between users and the entire tree, including the images and parameters for each object, can be reconstructed. I had not approached a problem like this before so I was not sure what the best practices were. I found XLMEncoder (http://java.sun.com/j2se/1.4.2/docs/api/java/beans/XMLEncoder.html) to be a very effective way of storing my tree and the native data type information. However I couldn't figure out how to save the image data itself inside of the XML file, and I'm not sure it is possible since the data is binary (so restricted characters would be invalid). My next thought was to associate a hash string instead of an image within each user object, and then gzip together all of the images, with the hash strings as the names and the XMLencoded tree in the same compmressed file. That seemed really contrived though. Does anyone know a good approach for this type of issue? THanks! Thanks!

    Read the article

  • Memory mapping of files and system cache behavior in WinXP

    - by Canopus
    Our application is memory intensive and deals with reading a large number of disk files. The total load can be more than 3 GB. There is a custom memory manager that uses memory mapped files to achieve reading of such a huge data. The files are mapped into the process memory space only when needed and with this the process memory is well under control. But what is observed is, with memory mapping, the system cache keeps on increasing until it occupies the available physical memory. This leads to the slowing down of the entire system. My question is how to prevent system cache from hogging the physical memory? I attempted to remove the file buffering (by using FILE_FLAG_NO_BUFFERING ), but with this, the read operations take considerable amount of time and slows down the application performance. How to achieve the scalability without sacrificing much on performance. What are the common techniques used in such cases? I dont have a good understanding of the WinXP OS caching behavior. Any good links explaining the same would also be helpful.

    Read the article

  • unbuffered I/O in Linux

    - by stuck
    I'm writing lots and lots of data that will not be read again for weeks - as my program runs the amount of free memory on the machine (displayed with 'free' or 'top') drops very quickly, the amount of memory my app uses does not increase - neither does the amount of memory used by other processes. This leads me to believe the memory is being consumed by the filesystems cache - since I do not intend to read this data for a long time I'm hoping to bypass the systems buffers, such that my data is written directly to disk. I dont have dreams of improving perf or being a super ninja, my hope is to give a hint to the filesystem that I'm not going to be coming back for this memory any time soon, so dont spend time optimizing for those cases. On Windows I've faced similar problems and fixed the problem using FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH - the machines memory was not consumed by my app and the machine was more usable in general. I'm hoping to duplicate the improvements I've seen but on Linux. On Windows there is the restriction of writing in sector sized pieces, I'm happy with this restriction for the amount of gain I've measured. is there a similar way to do this in Linux?

    Read the article

  • Android , Read in binary data and write it to file

    - by Shpongle
    Hi all , Im trying to read in image file from a server , with the code below . It keeps going into the exception. I know the correct number of bytes are being sent as I print them out when received. Im sending the image file from python like so #open the image file and read it into an object imgfile = open (marked_image, 'rb') obj = imgfile.read() #get the no of bytes in the image and convert it to a string bytes = str(len(obj)) #send the number of bytes self.conn.send( bytes + '\n') if self.conn.sendall(obj) == None: imgfile.flush() imgfile.close() print 'Image Sent' else: print 'Error' Here is the android part , this is where I'm having the problem. Any suggestions on the best way to go about receiving the image and writing it to a file ? //read the number of bytes in the image String noOfBytes = in.readLine(); Toast.makeText(this, noOfBytes, 5).show(); byte bytes [] = new byte [Integer.parseInt(noOfBytes)]; //create a file to store the retrieved image File photo = new File(Environment.getExternalStorageDirectory(), "PostKey.jpg"); DataInputStream dis = new DataInputStream(link.getInputStream()); try{ os =new FileOutputStream(photo); byte buf[]=new byte[1024]; int len; while((len=dis.read(buf))>0) os.write(buf,0,len); Toast.makeText(this, "File recieved", 5).show(); os.close(); dis.close(); }catch(IOException e){ Toast.makeText(this, "An IO Error Occured", 5).show(); } EDIT: I still cant seem to get it working. I have been at it since and the result of all my efforts have either resulted in a file that is not the full size or else the app crashing. I know the file is not corrupt before sending server side. As far as I can tell its definitely sending too as the send all method in python sends all or throws an exception in the event of an error and so far it has never thrown an exception. So the client side is messed up . I have to send the file from the server so I cant use the suggestion suggested by Brian .

    Read the article

  • VirtualBox sound problem under Ubuntu

    - by VoY
    As I recently upgraded to karmic I started to see the following stuff in the logs when I run VirtualBox: Oct 30 18:14:34 apocalypse pulseaudio[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532)io[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532) Oct 30 18:14:34 apocalypse pulseaudio[2813]: alsa-source.c: Resume failed, couldn't restore original fragment settings. (Old: 65536/65536, New 1073676288/65532) After a while the logs grow to large sizes and fill up all of my /var partition. In VirtualBox there is an option to choose between pulseaudio and alsa for sound, but it seems to have no effect. I am using virtualbox-3.0 packages, not the ose version. My system is up to date.

    Read the article

  • Parsing xml file that comes in as one object per line

    - by Casey
    I haven't been here in so long, I forgot my prior account! Anyways, I am working on parsing an xml document that comes in ugly. It is for banking statements. Each line is a <statement>all tags</statement>. Now, what I need to do is read this file in, and parse the XML document at the same time, while formatting it more human readable too. Point beeing, Original input looks like this: <statement><accountHeader><fiAddress></fiAddress><accountNumber></accountNumber><startDate>20140101</startDate><endDate>20140228</endDate><statementGroup>1</statementGroup><sortOption>0</sortOption><memberBranchCode>1</memberBranchCode><memberName></memberName><jointOwner1Name></jointOwner1Name><jointOwner2Name></jointOwner2Name></summary></statement> <statement><accountHeader><fiAddress></fiAddress><accountNumber></accountNumber><startDate>20140101</startDate><endDate>20140228</endDate><statementGroup>1</statementGroup><sortOption>0</sortOption><memberBranchCode>1</memberBranchCode><memberName></memberName><jointOwner1Name></jointOwner1Name><jointOwner2Name></jointOwner2Name></summary></statement> <statement><accountHeader><fiAddress></fiAddress><accountNumber></accountNumber><startDate>20140101</startDate><endDate>20140228</endDate><statementGroup>1</statementGroup><sortOption>0</sortOption><memberBranchCode>1</memberBranchCode><memberName></memberName><jointOwner1Name></jointOwner1Name><jointOwner2Name></jointOwner2Name></summary></statement> I need the final output to be as follows: <statement> <name></name> <address></address> </statement> This is fine and dandy. I am using the following "very slow considering 5.1 million lines, 254k data file, and about 60k statements takes around 8 minutes". foreach(String item in lines) { XElement xElement = XElement.Parse(item); sr.WriteLine(xElement.ToString().Trim()); } Then when the file is formatted this is what sucks. I need to check every single tag in transaction elements, and if a tag is missing that could be there, I have to fill it in. Our designer software will default prior values in if a tag is possible, and the current objects does not have. It defaults in the value of a prior one that was not Null. "I know, and they swear up and down it is not a bug... ok?" So, that is also taking about 5 to 10 minutes. I need to break all this down, and find a faster method for working with the initial XML. This is a preprocess action, and cannot take that long if not necessary. It just seems redundant. Is there a better way to parse the XML, or is this the best I can do? I parse the XML, write to a temp file, and then read that file in, to the output file inserting the missing tags. 2 IO runs for one process. Yuck.

    Read the article

  • File sizing issue in DOS/FAT

    - by Heather
    I've been tasked with writing a data collection program for a Unitech HT630, which runs a proprietary DOS operating system that can run executables compiled for 16-bit MS DOS with some restrictions. I'm using the Digital Mars C/C++ compiler, which is working well thus far. One of the application requirements is that the data file must be human-readable plain text, meaning the file can be imported into Excel or opened by Notepad. I'm using a variable length record format much like CSV that I've successfully implemented using the C standard library file I/O functions. When saving a record, I have to calculate whether the updated record is larger or smaller than the version of the record currently in the data file. If larger, I first shift all records immediately after the current record forward by the size difference calculated before saving the updated record. EOF is extended automatically by the OS to accommodate the extra data. If smaller, I shift all records backwards by my calculated offset. This is working well, however I have found no way to modify the EOF marker or file size to ignore the data after the end of the last record. Most of the time records will grow in size because the data collection program will be filling some of the empty fields with data when saving a record. Records will only shrink in size when a correction is made on an existing entry, or on a normal record save if the descriptive data in the record is longer than what the program reads in memory. In the situation of a shrinking record, after the last record in the file I'm left with whatever data was sitting there before the shift. I have been writing an EOF delimiter into the file after a "shrinking record save" to signal where the end of my records are and space-filling the remaining data, but then I no longer have a clean file until a "growing record save" extends the size of the file over the space-filled area. The truncate() function in unistd.h does not work (I'm now thinking this is for *nix flavors only?). One proposed solution I've seen involves creating a second file and writing all the data you wish to save into that file, and then deleting the original. Since I only have 4MB worth of disk space to use, this works if the file size is less than 2MB minus the size of my program executable and configuration files, but would fail otherwise. It is very likely that when this goes into production, users would end up with a file exceeding 2MB in size. I've looked at Ralph Brown's Interrupt List and the interrupt reference in IBM PC Assembly Language and Programming and I can't seem to find anything to update the file size or similar. Is reducing a file's size without creating a second file even possible in DOS?

    Read the article

  • Recommendations for a C++ polymorphic, seekable, binary I/O interface

    - by Trevor Robinson
    I've been using std::istream and ostream as a polymorphic interface for random-access binary I/O in C++, but it seems suboptimal in numerous ways: 64-bit seeks are non-portable and error-prone due to streampos/streamoff limitations; currently using boost/iostreams/positioning.hpp as a workaround, but it requires vigilance Missing operations such as truncating or extending a file (ala POSIX ftruncate) Inconsistency between concrete implementations; e.g. stringstream has independent get/put positions whereas filestream does not Inconsistency between platform implementations; e.g. behavior of seeking pass the end of a file or usage of failbit/badbit on errors Don't need all the formatting facilities of stream or possibly even the buffering of streambuf streambuf error reporting (i.e. exceptions vs. returning an error indicator) is supposedly implementation-dependent in practice I like the simplified interface provided by the Boost.Iostreams Device concept, but it's provided as function templates rather than a polymorphic class. (There is a device class, but it's not polymorphic and is just an implementation helper class not necessarily used by the supplied device implementations.) I'm primarily using large disk files, but I really want polymorphism so I can easily substitute alternate implementations (e.g. use stringstream instead of fstream for unit tests) without all the complexity and compile-time coupling of deep template instantiation. Does anyone have any recommendations of a standard approach to this? It seems like a common situation, so I don't want to invent my own interfaces unnecessarily. As an example, something like java.nio.FileChannel seems ideal. My best solution so far is to put a thin polymorphic layer on top of Boost.Iostreams devices. For example: class my_istream { public: virtual std::streampos seek(stream_offset off, std::ios_base::seekdir way) = 0; virtual std::streamsize read(char* s, std::streamsize n) = 0; virtual void close() = 0; }; template <class T> class boost_istream : public my_istream { public: boost_istream(const T& device) : m_device(device) { } virtual std::streampos seek(stream_offset off, std::ios_base::seekdir way) { return boost::iostreams::seek(m_device, off, way); } virtual std::streamsize read(char* s, std::streamsize n) { return boost::iostreams::read(m_device, s, n); } virtual void close() { boost::iostreams::close(m_device); } private: T m_device; };

    Read the article

  • java ioexception error=24 too many files open

    - by MattS
    I'm writing a genetic algorithm that needs to read/write lots of files. The fitness test for the GA is invoking a program called gradif, which takes a file as input and produces a file as output. Everything is working except when I make the population size and/or the total number of generations of the genetic algorithm too large. Then, after so many generations, I start getting this: java.io.FileNotFoundException: testfiles/GradifOut29 (Too many open files). (I get it repeatedly for many different files, the index 29 was just the one that came up first last time I ran it). It's strange because I'm not getting the error after the first or second generation, but after a significant amount of generations, which would suggest that each generation opens up more files that it doesn't close. But as far as I can tell I'm closing all of the files. The way the code is set up is the main() function is in the Population class, and the Population class contains an array of Individuals. Here's my code: Initial creation of input files (they're random access so that I could reuse the same file across multiple generations) files = new RandomAccessFile[popSize]; for(int i=0; i<popSize; i++){ files[i] = new RandomAccessFile("testfiles/GradifIn"+i, "rw"); } At the end of the entire program: for(int i=0; i<individuals.length; i++){ files[i].close(); } Inside the Individual's fitness test: FileInputStream fin = new FileInputStream("testfiles/GradifIn"+index); FileOutputStream fout = new FileOutputStream("testfiles/GradifOut"+index); Process process = Runtime.getRuntime().exec ("./gradif"); OutputStream stdin = process.getOutputStream(); InputStream stdout = process.getInputStream(); Then, later.... try{ fin.close(); fout.close(); stdin.close(); stdout.close(); process.getErrorStream().close(); }catch (IOException ioe){ ioe.printStackTrace(); } Then, afterwards, I append an 'END' to the files to make parsing them easier. FileWriter writer = new FileWriter("testfiles/GradifOut"+index, true); writer.write("END"); try{ writer.close(); }catch(IOException ioe){ ioe.printStackTrace(); } My redirection of stdin and stdout for gradif are from this answer. I tried using the try{close()}catch{} syntax to see if there was a problem with closing any of the files (there wasn't), and I got that from this answer. It should also be noted that the Individuals' fitness tests run concurrently. UPDATE: I've actually been able to narrow it down to the exec() call. In my most recent run, I first ran in to trouble at generation 733 (with a population size of 100). Why are the earlier generations fine? I don't understand why, if there's no leaking, the algorithm should be able to pass earlier generations but fail on later generations. And if there is leaking, then where is it coming from? UPDATE2: In trying to figure out what's going on here, I would like to be able to see (preferably in real-time) how many files the JVM has open at any given point. Is there an easy way to do that?

    Read the article

  • Java File I/O problems

    - by dwwilson66
    This is my first time working with file i/o in java, and it's not working. The section of the program where I parse individual lines and output a semicolon delimited line works like a charm when I hardcode a file and display on screen. Whne I try to write to a file public static OutputStream... errors out as an illegal start to expression, and I've been unable to get the program to step through an entire directory of files instead of one at a time. Where I'm not clear: I'm note setting an output filename anywhere...whare am I supposed to do that? The path variable won't pass. What's the proper format for a path? Can anyone see what I need to debug here? import java.io.*; public class FileRead { public static void main(String args[]) { try { // Open the file(s) // single file works OK FileInputStream fstream = new FileInputStream("testfile.txt"); Path startingDir = R:\Data\cs\RoboHelp\CorrLib\Output\Production\WebHelp; PrintFiles pf = new PrintFiles(); Files.walkFileTree(startingDir, pf); // Get the object of DataInputStream DataInputStream in = new DataInputStream(fstream); BufferedReader br = new BufferedReader(new InputStreamReader(in)); String inputLine; String desc = ""; String docNo = ""; String replLtr = ""; String specCond = ""; String states = ""; String howGen = ""; String whenGen = ""; String owner = ""; String lastChange = ""; //Read File Line By Line while ((inputLine = br.readLine()) != null) { int testVal=0; int stringMax = inputLine.length(); // if(inputLine.startsWith("Description")) {desc = inputLine.substring(13,inputLine.length());} else if(inputLine.startsWith("Reference Number")) {docNo = inputLine.substring(20,inputLine.length());} else if(inputLine.startsWith("Replaces Letter")) {replLtr = inputLine.substring(17,inputLine.length());} else if(inputLine.startsWith("Special Conditions")) {specCond = inputLine.substring(21,inputLine.length());} else if(inputLine.startsWith("States Applicable")) {states = inputLine.substring(19,inputLine.length());} else if(inputLine.startsWith("How Generated")) {howGen = inputLine.substring(15,inputLine.length());} else if(inputLine.startsWith("When Generated")) {whenGen = inputLine.substring(16,inputLine.length());} else if(inputLine.startsWith("Owner")) {owner = inputLine.substring(7,inputLine.length());} else if(inputLine.startsWith("Last Change Date")) {lastChange = inputLine.substring(17,inputLine.length());} } //close while loop // Print the content on the console String outStr1 = (desc + ";" + docNo + ";" + replLtr + ";" + specCond + ";" + states); String outStr2 = (";" + howGen + ";" + whenGen + ";" + owner + ";" + lastChange); String outString = (outStr1 + outStr2); System.out.print(inputLine + "\n" + outString); String lineItem = (outStr1+outStr2); // try (OutputStream out = new BufferedOutputStream (logfile.newOutputStream(CREATE, APPEND))) { out.write(lineItem, 0, lineItem.length); } catch (IOException x) { System.err.println(x); } public static OutputStream newOutputStream() throws IOException { // append to an existing file, create file if it doesn't initially exist out = Files.newOutputStream(c:, CREATE, APPEND); } //Close the input stream in.close(); } catch (Exception e) { //Catch exception if any System.err.println("Error: " + e.getMessage()); } } }

    Read the article

  • How bad is it to use a virtual file system with VMWare? [closed]

    - by user30997
    IT is running a series of VMs that we'd like to see optimized further: if the VMs' are Windows XP, storing their NTFS images out to the virtual disk (ext3) provided by Linux/VMWare, how much of a hit are we taking - as opposed to having a partition of the host hard drive formatted NTFS to eliminate the translation layer and the extra level of operating system IO preparation?

    Read the article

  • Why does OS X insist on spinning up all external drives when loading a file from the local drive?

    - by Phillip Oldham
    Why does OS X insist on spinning up all the attached external drives (firewire, usb) when loading a file from the local (internal) drive? It's driving me insane that I have to wait for 3 attached drives (1 back-up, 2 media) to spin up -- a total of 20s -- to access a file that is located only on my local/internal drive. There is no obvious need to access the other drives; nothing is being read from them and nothing need be written. Examples: Quicktime X opening a file from the local HDD. Starting Caffeine, an app which doesn't access any other files at all. Can I tell OS X to only spin those drives up when actually accessing them?

    Read the article

  • CloneZilla PXE Boot Without NFS

    - by John
    I am trying to setup CloneZilla to be bootable via PXE without using NFS. I do not have NFS running on our PXE server and would like to keep it that way. However, most of the information that I have found online indicates that you need to setup NFS in order to PXE boot CloneZilla. I believe that I am pretty close in getting it to work, but am not sure where to go next. Listed below are the different PXE menu option configurations that I have used so far. LABEL Clonezilla Live MENU LABEL Clonezilla Live KERNEL utilities/clonezilla/vmlinuz APPEND initrd=utilities/clonezilla/initrd.img boot=live live-config noswap nolocales edd=on nomodeset ocs_live_run="ocs-live-general" ocs_live_extra_param="" ocs_live_keymap="" ocs_live_batch="no" o$ I have also tried the following append lines, without success: APPEND initrd=utilities/clonezilla/initrd.img boot=live union=aufs noswap noprompt vga=788 fetch=tftp://10.130.155.23/filesystem.squashfs APPEND initrd=utilities/clonezilla/initrd.img boot=live union=aufs noswap noprompt vga=normal nomodeset nosplash fetch=tftp://10.130.155.23/filesystem.squashfs Each of them have resulted in a no go with the following error: "Unable to find a live file system on the network". It looks like it gets to the point of trying to load the filesystem.squashfs file, hangs, and then throws the error. Any help would be greatly appreciated.

    Read the article

  • what is acceptable datastore latency on VMware ESXi host?

    - by BeowulfNode42
    Looking at our performance figures on our existing VMware ESXi 4.1 host at the Datastore/Real-time performance data Write Latency Avg 14 ms Max 41 ms Read Latency Avg 4.5 ms Max 12 ms People don't seem to be complaining too much about it being slow with those numbers. But how much higher could they get before people found it to be a problem? We are reviewing our head office systems due to running low on storage space, and are tossing up between buying a 2nd VM host with DAS or buying some sort of NAS for SMB file shares in the near term and maybe running VMs from it in the longer term. Currently we have just under 40 staff at head office with 9 smaller branches spread across the country. Head office is runnning in an MS RDS session based environment with linux ERP and mail systems. In total 22 VMs on a single host with DAS made from a RAID 10 made of 6x 15k SAS disks.

    Read the article

  • MD3200 - 3 to 4x less throughput than MD1220. Am I missing something here?

    - by Igor Polishchuk
    I have two R710 servers with similar configuration. One in my office has MD1220 attached. Another one in the datacenter of my hosting services vendor has MD3200. I'm getting significantly worse throughput from MD3200 at my vendors setup. I'm mostly interested in sequential writes, and I'm getting these results in bonnie++ and dd tests: Seq. writes on MD1220 in my office: 1.1 GB/s - bonnie++, 1.3GB/s - dd Seq. writes on MD3200 at my vendor's: 240MB/s - bonnie++, 310MB/s - dd Unfortunately, I could not test the exactly the same configurations, but the two I have should be comparable. If anything, my good performing environment is cheaper than the bad performing. I expect at least similar throughput from these two setups. My vendor cannot really help me. Hopefully, somebody more familiar with the DAS performance can look at it and tell if I'm missing something here and my expectations are too high. To summarize, the question here is it reasonable to expect about 100MB/s of sequential write throughput per each couple of drives in RAID10 on MD3200? Is there any trick to enable such performance in MD3200 with dual controller as opposed to simple MD1220 with a single H800 adapter? More details about the configurations: A good one in my office: Dell R710 2CPU X5650 @ 2.67GHz 12 cores 96GB DDR3, OS: RHEL 5.5, kernel 2.6.18-194.26.1.el5 x86_64 20x300GB 2.5" SAS 10K in a single RAID10 1MB chunk size on MD1220 + Dell H800 I/O controller with 1GB cache in the host Not so good one at my vendor's: Dell R710 2CPU L5520 @ 2.27GHz 8 cores 144GB DDR3, OS: RHEL 5.5, kernel 2.6.18-194.11.4.el5 x86_64 20x146GB 2.5" SAS 15K in a single RAID10 512KB chunk size, Dell MD3200, 2 I/O controllers in array with 1GB cache each Additional information. I've also ran the same tests on the same vendor's host, but the storage was: two raids of 14x146GB 15K RPM drives RAID 10, striped together on the OS level on MD3000+MD1000. The performance was about 25% worse than on MD3200 despite having more drives. When I ran similar tests on the internal storage of my vendor's host (2x146GB 15K RPM drives RAID1, Perc 6i) I've got about 128MB/s seq. writes. Just two internal drives gave me about a half of 20 drives' throughput on MD3200. The random I/O performance of the MD3200 setup is ok, it gives me at least 1300 IOPS. I'm mostly have problems with sequentioal I/O throughput. Thank you for looking into it. Regards Igor

    Read the article

< Previous Page | 68 69 70 71 72 73 74 75 76 77 78 79  | Next Page >