Search Results

Search found 4688 results on 188 pages for 'io redirection'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 36/188 | < Previous Page | 32 33 34 35 36 37 38 39 40 41 42 43  | Next Page >

  • mmap() for large file I/O?

    - by Boatzart
    I'm creating a utility in C++ to be run on Linux which can convert videos to a proprietary format. The video frames are very large (up to 16 megapixels), and we need to be able to seek directly to exact frame numbers, so our file format uses libz to compress each frame individually, and append the compressed data onto a file. Once all frames are finished being written, a journal which includes meta data for each frame (including their file offsets and sizes) is written to the end of the file. I'm currently using ifstream and ofstream to do the file i/o, but I am looking to optimize as much as possible. I've heard that mmap() can increase performance in a lot of cases, and I'm wondering if mine is one of them. Our files will be in the tens to hundreds of gigabytes, and although writing will always be done sequentially, random access reads should be done in constant time. Any thoughts as to whether I should investigate this further, and if so does anyone have any tips for things to look out for? Thanks!

    Read the article

  • ASP.net file operations delay

    - by mtranda
    Ok, so here's the problem: I'm reading the stream from a FileUpload control, reading in chunks of n bytes and writing the array in a loop until I reach the stream's end. Now the reason I do this is because I need to check several things while the upload is still going on (rather than doing a Save(); which does the whole thing in one go). Here's the problem: when doing this from the local machine, I can see the file just fine as it's uploading and its size increases (had to add a Sleep(); clause in the loop to actually get to see the file being written). However, when I upload the file from a remote machine, I don't get to see it until the the file has completed uploading. Also, I've added another call to write the progress to a text file as the progress is going on, and I get the same thing. Local: the file updates as the upload goes on, remote: the token file only appears after the upload's done (which is somewhat useless since I need it while the upload's still happening). Is there some sort of security setting in (or ASP.net) that maybe saves files in a temporary location for remote machines as opposed to the local machine and then moves them to the specified destination? I would liken this with ASP.net displaying error messages when browsing from the local machine (even on the public hostname) as opposed to the generic compilation error page/generic exception page that is shown when browsing from a remote machine (and customErrors are not off) Any clues on this? Thanks in advance.

    Read the article

  • How to loop an executable command in the terminal in Linux?

    - by user1452373
    Let me first describe my situation, I am working on a Linux platform and have a collection of .bmp files that add one to the picture number from filename0022.bmp up to filename0680.bmp. So a total of 658 pictures. I want to be able to run each of these pictures through a .exe file that operates on the picture then kicks out the file to a file specified by the user, it also has some threshold arguments: lower, upper. So the typical call for the executable is: ./filter inputfile outputfile lower upper Is there a way that I can loop this call over all the files just from the terminal or by creating some kind of bash script? My problem is similar to this: Execute a command over multiple files with a batch file but this time I am working in a Linux command line terminal. Thank you for your help, Luke H

    Read the article

  • Performance of fopen vs stat

    - by Alex Marshall
    Hello, I'm writing several C programs for an embedded system where every bit of performance we can squeeze out will matter. Part of that is accessing log files. When determining if a file exists, is there any performance difference between using open / fopen, and stat ? I've been using stat on the assumption that it only has to do a quick check against the file system, whereas fopen would have to actually gain access to a file and manipulate internal data structures before returning. Is there any merit to this ?

    Read the article

  • Interface for reading variable length files with header and footer.

    - by John S
    I could use some hints or tips for a decent interface for reading file of special characteristics. The files in question has a header (~120 bytes), a body (1 byte - 3gb) and a footer (4 bytes). The header contains information about the body and the footer is only a simple CRC32-value of the body. I use Java so my idea was to extend the "InputStream" class and add a constructor such as "public MyInStream( InputStream in)" where I immediately read the header and the direct the overridden read()'s the body. Problem is, I can't give the user of the class the CRC32-value until the whole body has been read. Because the file can be 3gb large, putting it all in memory is a be an idea. Reading it all in to a temporary file is going to be a performance hit if there are many small files. I don't know how large the file is because the InputStream doesn't have to be a file, it could be a socket. Looking at it again, maybe extending InputStream is a bad idea. Thank you for reading the confused thoughts of a tired programmer. :)

    Read the article

  • display records which exist in file2 but not in file1

    - by Phoenix
    log file1 contains records of customers(name,id,date) who visited yesterday log file2 contains records of customers(name,id,date) who visited today How would you display customers who visited yesterday but not today? Constraint is: Don't use auxiliary data structure because file contains millions of records. [So, no hashes] Is there a way to do this using Unix commands ??

    Read the article

  • Atomic int writes on file

    - by Waneck
    Hello! I'm writing an application that will have to be able to handle many concurrent accesses to it, either by threads as by processes. So no mutex'es or locks should be applied to this. To make the use of locks go down to a minimum, I'm designing for the file to be "append-only", so all data is first appended to disk, and then the address pointing to the info it has updated, is changed to refer to the new one. So I will need to implement a small lock system only to change this one int so it refers to the new address. How is the best way to do it? I was thinking about maybe putting a flag before the address, that when it's set, the readers will use a spin lock until it's released. But I'm afraid that it isn't at all atomic, is it? e.g. a reader reads the flag, and it is unset on the same time, a writer writes the flag and changes the value of the int the reader may read an inconsistent value! I'm looking for locking techniques but all I find is either for thread locking techniques, or to lock an entire file, not fields. Is it not possible to do this? How do append-only databases handle this? Thanks! Cauê

    Read the article

  • how to read an arraylist from a txt file in java?

    - by lox
    how to read an arraylist from a txt file in java? my arraylist is the form of: public class Account { String username; String password; } i managed to put some "Accounts" in the a txt file, but now i don't know how to read them. this is how my arraylist look in the txt file: username1 password1 | username2 password2 | etc this is a part of the code i came up with, but it doesn't work. it looks logic to me though... :) . public static void RdAc(String args[]) { ArrayList<Account> peoplelist = new ArrayList<Account>(50); int i,i2,i3; String[] theword = null; try { FileReader fr = new FileReader("myfile.txt"); BufferedReader br = new BufferedReader(fr); String line = ""; while ((line = br.readLine()) != null) { String[] theline = line.split(" | "); for (i = 0; i < theline.length; i++) { theword = theline[i].split(" "); } for(i3=0;i3<theline.length;i3++) { Account people = new Account(); for (i2 = 0; i2 < theword.length; i2++) { people.username = theword[i2]; people.password = theword[i2+1]; peoplelist.add(people); } } } } catch (IOException ex) { System.out.println("Could not read from file"); } }

    Read the article

  • IOCP multiple socket completionports in same container

    - by Ohmages
    For the past couple of days I have been thinking about how to solve one of my problems I am facing, and I have tried to research the topic but don't really know what I can do. I have 2 sockets in the same struct that both have the same completionport. Problem is, they both use different protocols. Is there a way that I can find out which socket got triggered? Their called game_socket, and client_socket Example code would be something like... while (true) { error = GetQueuedCompletionStatus(CompletionPort, &BytesTransfered, (PULONG_PTR)&Key, &lpOverlapped, 0); srvc = CONTAINING_RECORD ( lpOverlapped, client , ol ); if ( error == TRUE ) { cout << endl << "SOCKET: [" << srvc->client_socket << "] TRIGGERED - WORKER THREAD" << endl; cout << endl << "BytesTransfered: [" << BytesTransfered << "]" << endl; if ( srvc->game_client triggered ) { // .. this code } else { // .. this code } Any ideas our help would be appreciated :)

    Read the article

  • How can i zip files in Java and not include files paths

    - by Ignacio
    For example, i want to zip a file stored in /Users/me/Desktop/image.jpg I maded this method: public static Boolean generateZipFile(ArrayList<String> sourcesFilenames, String destinationDir, String zipFilename){ // Create a buffer for reading the files byte[] buf = new byte[1024]; try { // VER SI HAY QUE CREAR EL ROOT PATH boolean result = (new File(destinationDir)).mkdirs(); String zipFullFilename = destinationDir + "/" + zipFilename ; System.out.println(result); // Create the ZIP file ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFullFilename)); // Compress the files for (String filename: sourcesFilenames) { FileInputStream in = new FileInputStream(filename); // Add ZIP entry to output stream. out.putNextEntry(new ZipEntry(filename)); // Transfer bytes from the file to the ZIP file int len; while ((len = in.read(buf)) > 0) { out.write(buf, 0, len); } // Complete the entry out.closeEntry(); in.close(); } // Complete the ZIP file out.close(); return true; } catch (IOException e) { return false; } } But when i extract the file, the unzipped files have the full path. I don't want the full path of each file in the zip i only want the filename. How can i made this?

    Read the article

  • Python: slow read & write for millions of small files

    - by Jami
    I am building directory tree which has tons of subdirectories and files. The total directory count is somewhere along 256^32 subdirectories with 256 files in each end which are only a few bytes long. I did this so I would have fast access to these files (since i'm not searching and i'm just directly accessing then via a known file path) I have a python script that builds this filesystem and reads & writes those files. The problem is that when I reach more than 1Gb of total filesize, the read and write methods become extremely slow. Here's the function I have that reads the contents of a file (the file contains an integer string), adds a certain number to it, then writes it back to the original file. def addInFile(path, scoreToAdd): num = scoreToAdd try: shutil.copyfile(path, '/tmp/tmp.txt') fp = open('/tmp/tmp.txt', 'r') num += int(fp.readlines()[0]) fp.close() except: pass fp = open('/tmp/tmp.txt', 'w') fp.write(str(num)) fp.close() shutil.copyfile('/tmp/tmp.txt', path) I previously tried performing linux console commands but it was slower. I copy the file to a temporary file first then access/modify it then copy it back because i found this was faster than directly accessing the file. I think the cause of the slowdown is because there're tons of files. performing this function 1000 times sometimes reach 1 minute now, but before (when there were only a few files, 1000 calls was performed for only less than 1 second) How do you suggest I fix this?

    Read the article

  • Loop over a file and write the next line if a condition is met

    - by 111078384259264152964
    Having a hard time fixing this or finding any good hints about it. I'm trying to loop over one file, modify each line slightly, and then loop over a different file. If the line in the second file starts with the line from the first then the following line in the second file should be written to a third file. !/usr/bin/env python with open('ids.txt', 'rU') as f: with open('seqres.txt', 'rU') as g: for id in f: id=id.lower()[0:4]+'_'+id[4] with open(id + '.fasta', 'w') as h: for line in g: if line.startswith(''+ id): h.write(g.next()) All the correct files appear, but they are empty. Yes, I am sure the if has true cases. :-) "seqres.txt" has lines with an ID number in a certain format, each followed by a line with data. The "ids.txt" has lines with the ID numbers of interest in a different format. I want each line of data with an interesting ID number in its own file. Thanks a million to anyone with a little advice!

    Read the article

  • How can I read/write data from a file?

    - by samy
    I'm writing a simple chrome extension. I need to create the ability to add sites URLs to a list, or read from the list. I use the list to open the sites in the new tabs. I'm looking for a way to have a data file I can write to, and read from. I was thinking on XML. I read there is a problem changing the content of files with Javascript. Is XML the right choice for this kinda thing? I should add that there is no web server, and the app will run locally, so maybe the problem websites are having are not same as this. Before I wrote this question, I tried one thing, and started to feel insecure because it didn't work. I made a XML file called Site.xml: <?xml version="1.0" encoding="utf-8" ?> <Sites> <site> <url> http://www.sulamacademy.com/AddMsgForum.asp?FType=273171&SBLang=0&WSUAccess=0&LocSBID=20375 </url> </site> <site> <url> http://www.wow.co.il </url> </site> <site> <url> http://www.Google.co.il </url> </site> I made this script to read the data from him, and put in on the html file. function LoadXML() { var ajaxObj = new XMLHttpRequest(); ajaxObj.open('GET', 'Sites.xml', false); ajaxObj.send(); var myXML = ajaxObj.responseXML; document.write('<table border="2">'); var prs = myXML.getElementsByTagName("site"); for (i = 0; i < prs.length; i++) { document.write("<tr><td>"); document.write(prs[i].getElementsByName("url")[0].childNode[0].nodeValue); document.write("</td></tr>"); } document.write("</table"); }

    Read the article

  • Ruby Thread with "watchdog"

    - by Sergio Campamá
    I'm implementing a ruby server for handling sockets being created from GPRS modules. The thing is that when the module powers down, there's no indication that the socket closed. I'm doing threads to handle multiple sockets with the same server. What I'm asking is this: Is there a way to use a timer inside a thread, reset it after every socket input, and that if it hits the timeout, closes the thread? Where can I find more information about this? EDIT: Code example that doesn't detect the socket closing require 'socket' server = TCPServer.open(41000) loop do Thread.start(server.accept) do |client| puts "Client connected" begin loop do line = client.readline open('log.txt', 'a') { |f| f.puts line.strip } end rescue puts "Client disconnected" end end end

    Read the article

  • Cannot write to SD card -- canWrite is returning false

    - by Fizz
    Sorry for the ambiguous title but I'm doing the following to write a simple string to a file: try { File root = Environment.getExternalStorageDirectory(); if (root.canWrite()){ System.out.println("Can write."); File def_file = new File(root, "default.txt"); FileWriter fw = new FileWriter(def_file); BufferedWriter out = new BufferedWriter(fw); String defbuf = "default"; out.write(defbuf); out.flush(); out.close(); } else System.out.println("Can't write."); }catch (IOException e) { e.printStackTrace(); } But root.canWrite() seems to be returning false everytime. I am not running this off of an emulator, I have my android Eris plugged into my computer via USB and running the app off of my phone via Eclipse. Is there a way of giving my app permission so this doesn't happen? Also, this code seems to be create the file default.txt but what if it already exists, will it ignore the creation and just open it to write or do I have to catch something like FileAlreadyExists(if such an exception exists) which then just opens it and writes? Thanks for any help guys.

    Read the article

  • UVA Online Judge 3n+1 : Right answer is Wrong answer

    - by Samuraisoulification
    Ive been toying with this problem for more than a week now, I have optimized it a lot, I seem to be getting the right answer, since it's the same as when I compare it to other's answers that got accepted, but I keep getting wrong answer. Im not sure what's going on! Anyone have any advice? I think it's a problem with the input or the output, cause Im not exactly sure how this judge thing works. So if anyone could pinpoint the problem, and also give me any advice on my code, Id be very appreciative!!! #include <iostream> #include <cstdlib> #include <stdio.h> #include <vector> using namespace std; class Node{ // node for each number that has teh cycles and number private: int number; int cycles; bool cycleset; // so it knows whether to re-set the cycle public: Node(int num){ number = num; cycles = 0; cycleset = false; } int getnumber(){ return number; } int getcycles(){ return cycles; } void setnumber(int num){ number = num; } void setcycles(int num){ cycles = num; cycleset = true; } bool cycled(){ return cycleset; } }; class Cycler{ private: vector<Node> cycleArray; int biggest; int cycleReal(unsigned int number){ // actually cycles through the number int cycles = 1; if (number != 1) { if (number < 1000000) { // makes sure it's in vector bounds if (!cycleArray[number].cycled()) { // sees if it's been cycled if (number % 2 == 0) { cycles += this->cycleReal((number / 2)); } else { cycles += this->cycleReal((3 * number) + 1); } } else { // if cycled get the number of cycles and don't re-calculate, ends recursion cycles = cycleArray[number].getcycles(); } } else { // continues recursing if it's too big for the vector if (number % 2 == 0) { cycles += this->cycleReal((number / 2)); } else { cycles += this->cycleReal((3 * number) + 1); } } } if(number < 1000000){ // sets cycles table for the number in the vector if (!cycleArray[number].cycled()) { cycleArray[number].setcycles(cycles); } } return cycles; } public: Cycler(){ biggest = 0; for(int i = 0; i < 1000000; i++){ // initialize the vector, set the numbers Node temp(i); cycleArray.push_back(temp); } } int cycle(int start, int end){ // cycles thorugh the inputted numbers. int size = 0; for(int i = start; i < end ; i++){ size = this->cycleReal(i); if(size > biggest){ biggest = size; } } int temp = biggest; biggest = 0; return temp; } int getBiggest(){ return biggest; } }; int main() { Cycler testCycler; int i, j; while(cin>>i>>j){ //read in untill \n int biggest = 0; if(i > j){ biggest = testCycler.cycle(j, i); }else{ biggest = testCycler.cycle(i, j); } cout << i << " " << j << " " << biggest << endl; } return 0; }

    Read the article

  • Execute a Application On The Server Using JavaScript

    - by Nathan Campos
    I have an application on my server that is called leaf.exe, that haves two arguments needed to run, they are: inputfile and outputfile, that will be like this example: pnote.exe input.pnt output.txt They are all on the same directory as my home page file(the executable and the input file). But I need that a JavaScript could run the application like that, then I want to know how could I do this.

    Read the article

  • C# - Problem while listing directories - DirectoryNotFoundException

    - by HoNgOuRu
    I'm getting a "DirectoryNotFoundException" error, here is the code: string directorio = "D:\MUSICA\La Trampa - El Mísero Espiral De Encanto"; DirectoryInfo dir = new DirectoryInfo(directorio); DirectoryInfo[] dirs = dir.GetDirectories(); <------------This is the line I'm having this problem. I believe it's caused when it tries to parse the tilde part of that string "Mísero". the directory "D:\MUSICA\La Trampa - El Mísero Espiral De Encanto" exists because I can see it and also have some files in it. Is there any way to send this string in correct way? Thanks

    Read the article

  • Online Image Slideshow Question. File Access Problems.

    - by msandbot
    Hi, I have a flash .swf file that I embed on my webpage. On my server I have the .swf file and multiple image folders. I would like to load every file in one of those folders into the flash slideshow. How should I go about doing this? I tried used Air but it doesn't work on my system as an application so I doubt it will work online. Eventually I plan on making a menu where you can select different folders to display and since they are of different sizes, a foreach loop would be optimal. Keeping a txt file with the number of images is also possible if theres a way to read that in, but I would prefer the more dynamic approach. I am working towards using php for the website if that helps find a solution. Thanks, -Mike Also my slideshow works great online currently but i have to hardcode in the number of files.

    Read the article

  • Second Thread Holding Up Entire Program in C# Windows Form Application

    - by Brandon
    In my windows form application, I'm trying to test the user's ability to access a remote machine's shared folder. The way I'm doing this (and I'm sure that there are better ways...but I don't know of them) is to check for the existence of a specific directory on the remote machine (I'm doing this because of firewall/other security restrictions that I'm confronted with in my organization). If the user has rights to access the shared folder, then it returns in no time at all, but if they don't, it hangs forever. To solve this, I threw the check into another thread and wait only 1000 milliseconds before determining that the share can't be hit by the user. However, when I do this, it still hangs as if it was never run in the same thread. What is making it hang and how do I fix it? I would think that the fact that it is in a separate thread would allow me to just let the thread finish on it's own in the background. Here is my code: bool canHitInstallPath = false; Thread thread = new Thread(new ThreadStart(() => { canHitInstallPath = Directory.Exists(compInfo.InstallPath); })); thread.Start(); thread.Join(1000); if (canHitInstallPath == false) { throw new Exception("Cannot hit folder: " + compInfo.InstallPath); }

    Read the article

  • shell_exec() Doesn't Show The Output

    - by Nathan Campos
    I'm doing a PHP site that uses a shell_exec() function like this: $file = "upload/" . $_FILES["file"]["name"]; $output = shell_exec("leaf $file"); echo "<pre>$output</pre>"; Where leaf is a program that is located in the same directory of my script, but when I tried to run this script on the server, I just got nothing. What is wrong?

    Read the article

  • C# "Could not find a part of the path" - Creating Local File

    - by Pyronaut
    I am trying to write to a folder that is located on my C:\ drive. I keep getting the error of : Could not find a part of the path .. etc My filepath looks basically like this : C:\WebRoot\ManagedFiles\folder\thumbs\5c27a312-343e-4bdf-b294-0d599330c42d\Image\lighthouse.jpg And I am writing to it like so : using (MemoryStream memoryStream = new MemoryStream()) { thumbImage.Save(memoryStream, ImageFormat.Jpeg); using (FileStream diskCacheStream = new FileStream(cachePath, FileMode.CreateNew)) { memoryStream.WriteTo(diskCacheStream); } memoryStream.WriteTo(context.Response.OutputStream); } Don't worry too much about the memory stream. It is just outputting it (After I save it). Since I am creating a file, I am a bit perplexed as to why it cannot find the file (Shouldn't it just write to where I tell it to, regardless?). The strange thing is, It has no issue when I'm testing it above using File.Exists. Obviously that is returning false, But it means that atleast my Filepath is semi legit. Any help is much appreciated.

    Read the article

  • java.util.zip - ZipInputStream v.s. ZipFile

    - by lucho
    Hello, community! I have some general questions regarding the java.util.zip library. What we basically do is an import and an export of many small components. Previously these components were imported and exported using a single big file, e.g.: <component-type-a id="1"/> <component-type-a id="2"/> <component-type-a id="N"/> <component-type-b id="1"/> <component-type-b id="2"/> <component-type-b id="N"/> Please note that the order of the components during import is relevant. Now every component should occupy its own file which should be externally versioned, QA-ed, bla, bla. We decided that the output of our export should be a zip file (with all these files in) and the input of our import should be a similar zip file. We do not want to explode the zip in our system. We do not want opening separate streams for each of the small files. My current questions: Q1. May the ZipInputStream guarantee that the zip entries (the little files) will be read in the same order in which they were inserted by our export that uses ZipOutputStream? I assume reading is something like: ZipInputStream zis = new ZipInputStream(new BufferedInputStream(fis)); ZipEntry entry; while((entry = zis.getNextEntry()) != null) { //read from zis until available } I know that the central zip directory is put at the end of the zip file but nevertheless the file entries inside have sequential order. I also know that relying on the order is an ugly idea but I just want to have all the facts in mind. Q2. If I use ZipFile (which I prefer) what is the performance impact of calling getInputStream() hundreds of times? Will it be much slower than the ZipInputStream solution? The zip is opened only once and ZipFile is backed by RandomAccessFile - is this correct? I assume reading is something like: ZipFile zipfile = new ZipFile(argv[0]); Enumeration e = zipfile.entries();//TODO: assure the order of the entries while(e.hasMoreElements()) { entry = (ZipEntry) e.nextElement(); is = zipfile.getInputStream(entry)); } Q3. Are the input streams retrieved from the same ZipFile thread safe (e.g. may I read different entries in different threads simultaneously)? Any performance penalties? Thanks for your answers!

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 32 33 34 35 36 37 38 39 40 41 42 43  | Next Page >