Search Results

Search found 5919 results on 237 pages for 'io priority'.

Page 15/237 | < Previous Page | 11 12 13 14 15 16 17 18 19 20 21 22  | Next Page >

• Working with Sub-Optimal Disk Configurations (Making the best of what you’ve got)

  - by Jonathan Kehayias

  This is the first post in a what will be a series of posts on working with a sub-optimal disk configuration to squeeze as much performance out of it as possible.  You might ask what a Sub-Optimal Disk Configuration?  In this case it is a Dell Powervault MD3000 with 15 Seagate Barracuda ES.2 SAS 1 TB 7.2K RPM disks (Model Number ST31000640SS).  This equates to just under 14TB of raw storage that can configured into a number of RAID configurations.  In this case, the disk array...(read more)

  Read the article

• Generic and type safe I/O model in any language

  - by Eduardo León

  I am looking for an I/O model, in any programming language, that is generic and type safe. By genericity, I mean there should not be separate functions for performing the same operations on different devices (read_file, read_socket, read_terminal). Instead, a single read operation works on all read-able devices, a single write operation works on all write-able devices, and so on. By type safety, I mean operations that do not make sense should not even be expressible in first place. Using the read operation on a non-read-able device ought to cause a type error at compile time, similarly for using the write operation on a non-write-able device, and so on. Is there any generic and type safe I/O model?

  Read the article

• How does I/O work for large graph databases?

  - by tjb1982

  I should preface this by saying that I'm mostly a front end web developer, trained as a musician, but over the past few years I've been getting more and more into computer science. So one idea I have as a fun toy project to learn about data structures and C programming was to design and implement my own very simple database that would manage an adjacency list of posts. I don't want SQL (maybe I'll do my own query language? I'm just having fun). It should support ACID. It should be capable of storing 1TB let's say. So with that, I was trying to think of how a database even stores data, without regard to data structures necessarily. I'm working on linux, and I've read that in that world "everything is a file," including hardware (like /dev/*), so I think that that obviously has to apply to a database, too, and it clearly does--whether it's MySQL or PostgreSQL or Neo4j, the database itself is a collection of files you can see in the filesystem. That said, there would come a point in scale where loading the entire database into primary memory just wouldn't work, so it doesn't make sense to design it with that mindset (I assume). However, reading from secondary memory would be much slower and regardless some portion of the database has to be in primary memory in order for you to be able to do anything with it. I read this post: Why use a database instead of just saving your data to disk? And I found it difficult to understand how other databases, like SQLite or Neo4j, read and write from secondary memory and are still very fast (faster, it would seem, than simply writing files to the filesystem as the above question suggests). It seems the key is indexing. But even indexes need to be stored in secondary memory. They are inherently smaller than the database itself, but indexes in a very large database might be prohibitively large, too. So my question is how is I/O generally done with large databases like the one I described above that would be at least 1TB storing a big adjacency list? If indexing is more or less the answer, how exactly does indexing work--what data structures should be involved?

  Read the article

• responsibility for storage

  - by Stefano Borini

  A colleague and I were brainstorming about where to put the responsibility of an object to store itself on the disk in our own file format. There are basically two choices: object.store(file) fileformatWriter.store(object) The first one gives the responsibility of serialization on the disk to the object itself. This is similar to the approach used by python pickle. The second groups the representation responsibility on a file format writer object. The data object is just a plain data container (eventually with additional methods not relevant for storage). We agreed on the second methodology, because it centralizes the writing logic from generic data. We also have cases of objects implementing complex logic that need to store info while the logic is in progress. For these cases, the fileformatwriter object can be passed and used as a delegate, calling storage operations on it. With the first pattern, the complex logic object would instead accept the raw file, and implement the writing logic itself. The first method, however, has the advantage that the object knows how to write and read itself from any file containing it, which may also be convenient. I would like to hear your opinion before starting a rather complex refactoring.

  Read the article

• How are Java ByteBuffer's limit and position variable's updated?

  - by Dummy Derp

  There are two scenarios: writing and reading Writing: Whenever I write something to the ByteBuffer by calling its put(byte[]) method the position variable is incremented as: current position + size of byte[] and limit stays at the max. If, however, I put the data in a view buffer then I will have to, manually, calculate and update the position Before I call the write(ByteBuffer) method of the channel to write something, I will have to flip() the Bytebuffer so that position points to zero and limit points to the last byte that was written to the ByteBuffer. Reading: Whenever I call the read(ByteBuffer) method of a channel to read something, the position variable stays at 0 and the limit variable of the ByteBuffer points to the last byte that was read. So, if the ByteBuffer is smaller than the file being read, the limit variable is pushed to max This means that the ByteBuffer is already flipped and I can proceed to extracting the values from the ByteBuffer. Please, correct me where I am wrong :)

  Read the article

• Quick Tip - Speed a Slow Restore from the Transaction Log

  - by KKline

  Here's a quick tip for you: During some restore operations on Microsoft SQL Server, the transaction log redo step might be taking an unusually long time. Depending somewhat on the version and edition of SQL Server you've installed, you may be able to increase performance by tinkering with the readahead performance for the redo operations. To do this, you should use the MAXTRANSFERSIZE parameter of the RESTORE statement. For example, if you set MAXTRANSFERSIZE=1048576, it'll use 1MB buffers. If you...(read more)

  Read the article

• Design pattern for an automated mechanical test bench

  - by JJS

  Background I have a test fixture with a number of communication/data acquisition devices on it that is used as an end of line test for a product. Because of all the various sensors used in the bench and the need to run the test procedure in near real-time, I'm having a hard time structuring the program to be more friendly to modify later on. For example, a National Instruments USB data acquisition device is used to control an analog output (load) and monitor an analog input (current), a digital scale with a serial data interface measures position, an air pressure gauge with a different serial data interface, and the product is interfaced through a proprietary DLL that handles its own serial communication. The hard part The "real-time" aspect of the program is my biggest tripping point. For example, I need to time how long the product needs to go from position 0 to position 10,000 to the tenth of a second. While it's traveling, I need to ramp up an output of the NI DAQ when it reaches position 6,000 and ramp it down when it reaches position 8,000. This sort of control looks easy from browsing NI's LabVIEW docs but I'm stuck with C# for now. All external communication is done by polling which makes for lots of annoying loops. I've slapped together a loose Producer Consumer model where the Producer thread loops through reading the sensors and sets the outputs. The Consumer thread executes functions containing timed loops that poll the Producer for current data and execute movement commands as required. The UI thread polls both threads for updating some gauges indicating current test progress. Unsure where to start Is there a more appropriate pattern for this type of application? Are there any good resources for writing control loops in software (non-LabVIEW) that interface with external sensors and whatnot?

  Read the article

• Quick Tip - Speed a Slow Restore from the Transaction Log

  - by KKline

  Here's a quick tip for you: During some restore operations on Microsoft SQL Server, the transaction log redo step might be taking an unusually long time. Depending somewhat on the version and edition of SQL Server you've installed, you may be able to increase performance by tinkering with the readahead performance for the redo operations. To do this, you should use the MAXTRANSFERSIZE parameter of the RESTORE statement. For example, if you set MAXTRANSFERSIZE=1048576, it'll use 1MB buffers. If you...(read more)

  Read the article

• iPhone in-app purchasing for Ecommerce [closed]

  - by Kyle B.

  This may not be the appropriate location for this, but would like to ask in the hopes an iOS developer with familiarity on the rules and regulations could comment. I would like to develop an iOS app that performs Ecommerce transactions. If I roll my own payment processor, and checkout process: 1) Is this allowed by Apple's rules, and 2) Would I be required to remit 30% of the transaction sale to Apple?

  Read the article

• How many disk should I use to meed the capacity and IOPS need?

  - by facebook-100005613813158

  An application needs 1.6 TB of storage capacity and performs 1000 IOPS. How many disks are required to meet the application requirement and offer acceptable response time? The disk specifications are as follows: - Drive capacity = 100 GB - 15K RPM - Each disk can perform 50 IOPS 4 candidate , 10,12,16,20, which one is the most likely answer? in my opinion,16 disks can only meet the capacity need ,but cannot meet the IOPS need, so ,the right answer should be 20 disks?right?

  Read the article

• At what point is asynchronous reading of disk I/O more efficient than synchronous?

  - by blesh

  Assuming there is some bit of code that reads files for multiple consumers, and the files are of any arbitrary size: At what size does it become more efficient to read the file asynchronously? Or to put it another way, how small must a file be for it to be faster just to read it synchronously? I've noticed (and perhaps I'm incorrect) that when reading very small files, it takes longer to read them asynchronously than synchronously (in particular with .NET). I'm assuming this has to do with set up time for things like I/O Completion Ports, threads, etc. Is there any rule of thumb to help out here? Or is it dependent on the system and the environment?

  Read the article

• Ubuntu 13.XX unable to mount USB HDD. Tried everything. I/O error boot sector/file system

  - by XaviGG

  I know that there are many posts related but none of them helped me. I will jump to the last test because it is the one that should work, but it does not. An external HDD with single partition slow NTFS formatted in Windows, empty and clean. Checked for errors, it tells that not errors where found. Moving to Ubutnu 13.04... Gparted throws the first error when trying to read the disk: Input/output error It appears as unknown the content of the disk. Unable to create partition table or format it, getting the same error when trying. If I try to mount it in the terminal it tells me the same, specifying that also there is an I/O error reading the boot sector. I have this problem since I upgraded (always with fresh install) to 13.04. I thought it will be solved by the 13.10 but it has the same behavior. I tried with two different HDD (HD and SSHD) that work perfectly in Windows 7. In 13.04 at least I got a trying of mounting where the icon of the drive started showing and disappearing until finally it disappeared. But now it doesn't even try. Possible causes: -The HDD was my old main HDD, so it had WIN,RECOVERY,SYSTEM,UBU,SWAP partitions. Maybe the way or place where the partition table is defined is not the best for an external HDD but I don't know a lot in that topic. I would appreciate a lot if someone can give me a guideline to convert one of these HDD in a working external HDD. No files to recover, nothing to care about. Just format completely the disk and be able to use it for storing backups without having to move the files first to the windows partition, load windows and then copy them to the external HDD. Because I want to use a file comparator for the backups. Thanks a lot Edit 1: I found an option in Windows to convert it to a dynamic HDD that warns me that I wont be able to run O.S. after changing. I suppose that is what I need because in the current mode I cannot safely extract it. But it tells me an error that it couldn't change the mode.

  Read the article

• Reading input all together or in steps?

  - by nischayn22

  For many programming quizzes we are given a bunch of input lines and we have to process each input , do some computation and output the result. My question is what is the best way to optimize the runtime of the solution ? Read all input, store it (in array or something) ,compute result for all of them, finally output it all together. or 2. Read one input, compute the result, output the result and so on for each input given.

  Read the article

• Structuring an input file

  - by Ricardo

  I am in the process of structuring a small program to perform some hydraulic analysis of pipe flow. As I am envisioning this, the program will read an input file, store the input parameters in a suitable way, operate on them and finally output results. I am struggling with how to structure the input file in a sane way; that is, in a way that a human can write it easily and a machine can parse it easily. A sample input file made available to me for a similar program is just a stream of comma-separated numbers that don't make much sense on their own, so that's the scenario I am trying to avoid. Though I am giving the details of my particular problem, I am more interested in general input-file structuring strategies. Is a stream of comma-separated values my best bet? Would I be better off using some sort of key:value structure? I don't know much about this, so any help will probably put me in a better track than I am now.

  Read the article

• At what point asynchronous reading of disk I/O is more efficient than synchronous?

  - by blesh

  Assuming there is some bit of code that reads files for multiple consumers, and the files are of any arbitrary size: At what size does it become more efficient to read the file asynchronously? Or to put it another way, how small must a file be for it to be faster just to read it synchronously? I've noticed (and perhaps I'm incorrect) that when reading very small files, it takes longer to read them asynchronously than synchronously (in particular with .NET). I'm assuming this has to do with set up time for things like I/O Completion Ports, threads, etc. Is there any rule of thumb to help out here? Or is it dependent on the system and the environment?

  Read the article

• What is wrong with this code for reading binary files? [on hold]

  - by qed

  What is wrong with this code for reading binary files? It compiles OK, but will not print out the file as planned, in fact, it prints nothing at all. #include <iostream> #include <fstream> int main(int argc, const char *argv[]) { if (argc < 2) { ::std::cerr << "usage: " << argv[0] << " <filename>\n"; return 1; } ::std::basic_ifstream<unsigned char> in(argv[1], ::std::ios::binary); unsigned char uc; while (in.get(uc)) { printf("%02X ", uc); } // TODO: error handling, in case the file could not be opened or read return 0; }

  Read the article

• Java - HtmlUnit - Unable to save HTML to file (in some cases)

  - by Walter White

  Hi all, I am having intermittent issues saving the response HTML in HtmlUnit. Caused by: java.io.IOException: Unable to save file:C:\ccview\PP50773_4.0_walter\TSC_hca\Applications\HCA_J2EE\HCA\target\HtmlUnitTests\single\1\com\pnc\tsc\hca\ui\test\SiteCrawler\crawlSiteAsProvider\10.SiteCrawler.crawl.html at com.pnc.tsc.hca.ui.util.GetUtil.save(GetUtil.java:128) at com.pnc.tsc.hca.ui.util.GetUtil.add(GetUtil.java:75) at com.pnc.tsc.hca.ui.util.GetUtil.click(GetUtil.java:49) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawl(SiteCrawler.java:87) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawl(SiteCrawler.java:61) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawl(SiteCrawler.java:63) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawl(SiteCrawler.java:63) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawl(SiteCrawler.java:63) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawl(SiteCrawler.java:54) at com.pnc.tsc.hca.ui.test.SiteCrawler.crawlSiteAsProvider(SiteCrawler.java:50) ... 15 more Caused by: java.lang.RuntimeException: java.io.IOException: The system cannot find the path specified at com.gargoylesoftware.htmlunit.html.XmlSerializer.getAttributesFor(XmlSerializer.java:165) at com.gargoylesoftware.htmlunit.html.XmlSerializer.printOpeningTag(XmlSerializer.java:126) at com.gargoylesoftware.htmlunit.html.XmlSerializer.printXml(XmlSerializer.java:83) at com.gargoylesoftware.htmlunit.html.XmlSerializer.printXml(XmlSerializer.java:93) at com.gargoylesoftware.htmlunit.html.XmlSerializer.printXml(XmlSerializer.java:93) at com.gargoylesoftware.htmlunit.html.XmlSerializer.asXml(XmlSerializer.java:73) at com.gargoylesoftware.htmlunit.html.XmlSerializer.save(XmlSerializer.java:55) at com.gargoylesoftware.htmlunit.html.HtmlPage.save(HtmlPage.java:2259) at com.pnc.tsc.hca.ui.util.GetUtil.save(GetUtil.java:126) ... 24 more Caused by: java.io.IOException: The system cannot find the path specified at java.io.WinNTFileSystem.createFileExclusively(Native Method) at java.io.File.createNewFile(File.java:883) at com.gargoylesoftware.htmlunit.html.XmlSerializer.createFile(XmlSerializer.java:216) at com.gargoylesoftware.htmlunit.html.XmlSerializer.getAttributesFor(XmlSerializer.java:160) ... 32 more Now, the parent directory exists and some other files have already been written to the directory. Looking at the filename, I don't see anything that would stand out as a red flag indicating the filename is bad. What can I do to correct this error? Thanks, Walter

  Read the article

• Understanding Node.js and concept of non-blocking I/O

  - by Saif Bechan

  Recently I became interested in using Node.js to tackle some of the parts of my web-application. I love the part that its full JavaScript and its very light weight so no use anymore to call an JavaScript-PHP call but a lighter JavaScript-JavaScript call. I however do not understand all the concepts explained. Basic concepts Now in the presentation for Node.js Ryan Dahl talks about non-blocking IO and why this is the way we need to create our programs. I can understand the theoretical concept. You just don't wait for a response, you go ahead and do other things. You make a callback for the response, and when the response arrives millions of clock-cycles later, you can fire that. If you have not already I recommend to watch this presentation. It is very easy to follow and pretty detailed. There are some nice concepts explained on how to write your code in a good manner. There are also some examples given and I am going to work with the basic example given. Examples The way we do thing now: puts("Enter your name: "); var name = gets(); puts("Name: " + name); Now the problem with this is that the code is halted at line 1. It blocks your code. The way we need to do things according to node puts("Enter your name: "); gets(function (name) { puts("Name: " + name); }); Now with this your program does not halt, because the input is a function within the output. So the programs continues to work without halting. Questions Now the basic question I have is how does this work in real-life situations. I am talking here for the use in web-applications. The application I am writing does I/O, bit is still does it in am blocking matter. I think that most of the time, if not all, you need to block, because you have to wait on what the response is you have to work with. When you need to get some information from the database, most of the time this data needs to be verified before you can further with the code. Example 1 If you take a login for example. You have to wait for the database to response to return, because you can not do anything else. I can't see a way around this without blocking. Example 2 Going back to the basic example. The use just request something from a database which does not need any verification. You still have to block because you don't have anything to do more. I can not come up with a single example where you want to do other things while you wait for the response to return. Possible answers I have read that this frees up recourses. When you program like this it takes less CPU or memory usage. So this non-blocking IO is ONLY meant to free up recourses and does not have any other practical use. Not that this is not a huge plus, freeing up recourses is always good. Yet I fail to see this as a good solution. because in both of the above examples, the program has to wait for the response of the user. Whether this is inside a function, or just inline, in my opinion there is a program that wait for input. Resources I looked at I have looked at some recourses before I posted this question. They talk a lot about the theoretical concept, which is quite clear. Yet i fail to see some real-life examples where this is makes a huge difference. Stackoverflow: What is in simple words blocking IO and non-blocking IO? Blocking IO vs non-blocking IO; looking for good articles tidy code for asynchronous IO Other recources: Wikipedia: Asynchronous I/O Introduction to non-blocking I/O The C10K problem

  Read the article

• FileInfo.MoveTo() vs File.Move()

  - by Eric

  Is there any difference between these two methods of moving a file? System.IO.FileInfo f = new System.IO.FileInfo(@"c:\foo.txt"); f.MoveTo(@"c:\bar.txt"); //vs System.IO.File.Move(@"c:\foo.txt", @"c:\bar.txt");

  Read the article

• please give me a solution

  - by user327832

  here is the code i have written so far but ended up giving me error import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.io.InputStream; public class Main { public static void main(String[] args) throws Exception { File file = new File("c:\\filea.txt"); InputStream is = new FileInputStream(file); long length = file.length(); System.out.println (length); bytes[] bytes = new bytes[(int) length]; try { int offset = 0; int numRead = 0; while (numRead >= 0) { numRead = is.read(bytes); } } catch (IOException e) { System.out.println ("Could not completely read file " + file.getName()); } is.close(); Object[] see = new Object[(int) length]; see[1] = bytes; System.out.println ((String[])see[1]); } }

  Read the article

• C# File IO with Streams - Best Memory Buffer Size

  - by AJ

  Hi, I am writing a small IO library to assist with a larger (hobby) project. A part of this library performs various functions on a file, which is read / written via the FileStream object. On each StreamReader.Read(...) pass, I fire off an event which will be used in the main app to display progress information. The processing that goes on in the loop is vaired, but is not too time consuming (it could just be a simple file copy, for example, or may involve encryption...). My main question is: What is the best memory buffer size to use? Thinking about physical disk layouts, I could pick 2k, which would cover a CD sector size and is a nice multiple of a 512 byte hard disk sector. Higher up the abstraction tree, you could go for a larger buffer which could read an entire FAT cluster at a time. I realise with today's PC's, I could go for a more memory hungry option (a couple of MiB, for example), but then I increase the time between UI updates and the user perceives a less responsive app. As an aside, I'm eventually hoping to provide a similar interface to files hosted on FTP / HTTP servers (over a local network / fastish DSL). What would be the best memory buffer size for those (again, a "best-case" tradeoff between perceived responsiveness vs. performance). Thanks in advance for any ideas, Adam

  Read the article

• java.io.IOException: Premature EOF

  - by jouzef19

  Hi: I am trying to download an xml file using Stream and things was fine , until the xml size became bigger than 9 MB , so i've got this error java.io.IOException: Premature EOF this is the code BufferedInputStream bfi = null; try { bfi = new BufferedInputStream(new URL("The URL").openStream()); String name = "name.xml"; FileOutputStream fb = new FileOutputStream(name); BufferedOutputStream bout = new BufferedOutputStream(fb, 1024); byte[] data = new byte[1024]; int x = 0; while ((x = bfi.read(data, 0, 1024)) >= 0) { bout.write(data, 0, x); } this.deletePhishTankDatabase(this.recreateFileName()); ptda.insertDownloadTime(hour, day, month, year); bout.close(); bfi.close(); } catch (IOException ex) { Logger.getLogger(PhishTankDataBase.class.getName()).log(Level.SEVERE, null, ex); } finally { try { bfi.close(); } catch (IOException ex) { Logger.getLogger(PhishTankDataBase.class.getName()).log(Level.SEVERE, null, ex); } } } else { System.out.println("You can't do anything"); return; } thanks in advanced

  Read the article

• Hibernate JDBCConnectionException: Communications link failure and java.io.EOFException: Can not read response from server

  - by Marc

  I get a quite well-known using MySql jdbc driver : JDBCConnectionException: Communications link failure, java.io.EOFException: Can not read response from server. This is caused by the wait_timeout parameter in my.cnf. So I decided to use c3p0 pool connection along with Hibernate. Here is what I added to hibernate.cfg.xml : <property name="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property> <property name="c3p0.min_size">10</property> <property name="c3p0.max_size">100</property> <property name="c3p0.timeout">1000</property> <property name="c3p0.preferredTestQuery">SELECT 1</property> <property name="c3p0.acquire_increment">1</property> <property name="c3p0.idle_test_period">2</property> <property name="c3p0.max_statements">50</property> idle_test_period is volontarily low for test purposes. Looking at the mysql logs I can see the "SELECT 1" request which is regularly sent to the mysql server so it works. Unfortunately I still get this EOF exception within my app if I wait longer than 'wait_timout' seconds (set to 10 for test purposes). I'm using Hibernate 4.1.1 and mysql-jdbc-connector 5.1.18. So what am I doing wrong? Thanks, Marc.

  Read the article

• File IO with Streams - Best Memory Buffer Size

  - by AJ

  I am writing a small IO library to assist with a larger (hobby) project. A part of this library performs various functions on a file, which is read / written via the FileStream object. On each StreamReader.Read(...) pass, I fire off an event which will be used in the main app to display progress information. The processing that goes on in the loop is vaired, but is not too time consuming (it could just be a simple file copy, for example, or may involve encryption...). My main question is: What is the best memory buffer size to use? Thinking about physical disk layouts, I could pick 2k, which would cover a CD sector size and is a nice multiple of a 512 byte hard disk sector. Higher up the abstraction tree, you could go for a larger buffer which could read an entire FAT cluster at a time. I realise with today's PC's, I could go for a more memory hungry option (a couple of MiB, for example), but then I increase the time between UI updates and the user perceives a less responsive app. As an aside, I'm eventually hoping to provide a similar interface to files hosted on FTP / HTTP servers (over a local network / fastish DSL). What would be the best memory buffer size for those (again, a "best-case" tradeoff between perceived responsiveness vs. performance).

  Read the article

• Rapid Opening and Closing System.IO.StreamWriter in C#

  - by ccomet

  Suppose you have a file that you are programmatically logging information into with regards to a process. Kinda like your typical debug Console.WriteLine, but due to the nature of the code you're testing, you don't have a console to write onto so you have to write it somewhere like a file. My current program uses System.IO.StreamWriter for this task. My question is about the approach to using the StreamWriter. Is it better to open just one StreamWriter instance, do all of the writes, and close it when the entire process is done? Or is it a better idea to open a new StreamWriter instance to write a line into the file, then immediately close it, and do this for every time something needs to be written in? In the latter approach, this would probably be facilitated by a method that would do just that for a given message, rather than bloating the main process code with excessive amounts of lines. But having a method to aid in that implementation doesn't necessarily make it the better choice. Are there significant advantages to picking one approach or the other? Or are they functionally equivalent, leaving the choice on the shoulders of the programmer?

  Read the article

< Previous Page | 11 12 13 14 15 16 17 18 19 20 21 22  | Next Page >