Search Results

Search found 7538 results on 302 pages for 'parallel processing'.

Page 30/302 | < Previous Page | 26 27 28 29 30 31 32 33 34 35 36 37  | Next Page >

• processing a file full of unix time strings to human readble

  - by skymook

  I am processing a file full of unix time strings. I want to convert them all to human readable. The file looks like so: 1153335401 1153448586 1153476729 1153494310 1153603662 1153640211 Here is the script: #! /bin/bash FILE="test.txt" cat $FILE | while read line; do perl -e 'print scalar(gmtime($line)), "\n"' done This is not working. The output I get is Thu Jan 1 00:00:00 1970 for every line. I think the line breaks are being picked up and that is why it is not working. Any ideas? I'm using Mac OSX is that makes any difference.

  Read the article

• Processing SMTP bounces with .net

  - by justSteve

  Am looking for examples specific to .net/mvc and servers native WinServer08 where problem being addressed is processing a bounced smtp msg so as to bind to an estore transaction and updating account/profile properties. Reading the related questions i find an interesting reference to [VERP]2. Under the heading 'Software that supports VERP i find that IIS is not on the list. Does that mean i need to find a library to integrate into my store's assembly? What resources do I have to pull together to make sure that the webapp is informed when mail bounces? fwiw - i'm working with a very low volume site.

  Read the article

• Realtime processing and callbacks with Python and C++

  - by Doughy

  I need to write code to do some realtime processing that is fairly computationally complex. I would like to create some Python classes to manage all my scripting, and leave the intensive parts of the algorithm coded in C++ so that they can run as fast as possible. I would like to instantiate the objects in Python, and have the C++ algorithms chime back into the script with callbacks in python. Something like: myObject = MyObject() myObject.setCallback(myCallback) myObject.run() def myCallback(val): """Do something with the value passed back to the python script.""" pass Will this be possible? How can I run a callback in python from a loop that is running in a C++ module? Anyone have a link or a tutorial to help me do this correctly?

  Read the article

• Python text file processing speed issues

  - by Anonymouslemming

  Hi all, I'm having a problem with processing a largeish file in Python. All I'm doing is f = gzip.open(pathToLog, 'r') for line in f: counter = counter + 1 if (counter % 1000000 == 0): print counter f.close This takes around 10m25s just to open the file, read the lines and increment this counter. In perl, dealing with the same file and doing quite a bit more (some regular expression stuff), the whole process takes around 1m17s. Perl Code: open(LOG, "/bin/zcat $logfile |") or die "Cannot read $logfile: $!\n"; while (<LOG>) { if (m/.*\[svc-\w+\].*login result: Successful\.$/) { $_ =~ s/some regex here/$1,$2,$3,$4/; push @an_array, $_ } } close LOG; Can anyone advise what I can do to make the Python solution run at a similar speed to the Perl solution? I've tried just uncompressing the file and dealing with it using open instead of gzip.open, but that made a very small difference to the overall time.

  Read the article

• Rails controller processing as HTML instead of XML

  - by Andy

  I've recently upgraded from Ruby 1.8.6 and Rails 2.3.4 to Ruby 1.9 and Rails 3.0.3. I have the following controller: class ChartController < ApplicationController before_filter :login_required respond_to :html, :xml def load_progress chart.add( :series, "Memorized", y_memorized ) chart.add( :series, "Learning", y_learning ) chart.add( :series, "Mins / Day", y_time ) chart.add( :user_data, :secondary_y_interval, time_axis_interval ) respond_to do |fmt| fmt.xml { render :xml => chart.to_xml } end # Also tried # respond_with chart end end However, when I call the 'load_progress method' I get the following: Started GET "/load_progress.xml" for 127.0.0. Processing by ChartController#load_progress as HTML Completed 406 Not Acceptable in 251ms I have also tried changing the respond_to block to respond_with chart But I get the same response. I've read all the new Rails documentation on the new respond_with format but I can't seem to elicit an XML response. Am desperately hoping someone has some ideas.

  Read the article

• Securing database keys for client-side processing

  - by danp

  I have a tree of information which is sent to the client in a JSON object. In that object, I don't want to have raw IDs which are coming from the database. I thought of making a hash of the id and a field in the object (title, for example) or a salt, but I'm worried that this might have a serious effect on processing overhead. SELECT * FROM `things` where md5(concat(id,'some salt')) = md5('1some salt'); Is there a standard practice for obscuring IDs in this kind of situation?

  Read the article

• MessageBox not shown when opened processing WM_CLOSE from taskbar thumbnail close button

  - by Katana

  Trying to put up a "Do you want to save"-dialog when trying to close window with close-button in taskbar thumbnail in windows 7(with aero peek active). Using MessageBox() when processing WM_CLOSE does not work. MessageBox won't show until you move mouse cursor outside thumbnail so aero peek is disabled. Lots of applications have this buggy behaviour so it's probably a design flaw in Windows 7, but for some programs it works (Word, Notepad, Visual Studio, ...), so I'm wondering what trick they are using(or what it takes to "exit" aero peek-mode programmatically). The small "Sound Recorder" application that comes with Windows 7 has the same problem (if you have recorded something without saving and try to close it using thumbnail close-button)...

  Read the article

• Is it possible to have asynchronous processing

  - by prashant2361

  Hi, I have a requirement where I need to send continuous updates to my clients. Client is browser in this case. We have some data which updates every sec, so once client connects to our server, we maintain a persistent connection and keep pushing data to the client. I am looking for suggestions of this implementation at the server end. Basically what I need is this: 1. client connects to server. I maintain the socket and metadata about the socket. metadata contains what updates need to be send to this client 2. server process now waits for new client connections 3. One other process will have the list of all the sockets opened and will go through each of them and send the updates if required. Can we do something like this in Apache module: 1. Apache process gets the new connection. It maintains the state for the connection. It keeps the state in some global memory and returns back to root process to signify that it is done so that it can accept the new connection 2. the Apache process though has returned the status to root process but it is also executing in parallel where it going through its global store and sending updates to the client, if any. So can a Apache process do these things: 1. Have more than one connection associated with it 2. Asynchronously waiting for new connection and at the same time processing the previous connections? Regards Prashant

  Read the article

• How to SET TIMING ON for parallel upgrades to 12c?

  - by Mike Dietrich

  Have you asked yourself how to get timings in an Oracle Database 12c upgrade for all statements? When you run the parallel upgrade via catctl.pl, the parallel upgrade Perl driving script in Oracle Database 12c, you may also want to get timings written in your logfile during execution. As catctl.pl does not offer an option yet the best way to achieve this is to edit the catupses.sql script in $ORACLE/rdbms/admin as this script will get called all time over and over again throughout all steps of theupgrade run. Just add these lines marked in RED to catupses.sql and start your upgrade: Rem =============================================Rem Call Common session settingsRem =============================================@@catpses.sql Rem =============================================Rem  Set Timing On during the UpgradeRem =============================================SET TIMING ON; Rem =============================================Rem Turn off PL/SQL event used by APPSRem =============================================ALTER SESSION SET EVENTS='10933 trace name context off'; -Mike PS: This may become the default in a future patch set

  Read the article

• How can I use a CanoScan N640Pex scanner over a USB/parallel cable?

  - by detly

  I have an old CanoScan N640Pex flat bed scanner and a USB-to-parallel port cable through which I can connect it to my PC. Unfortunately neither Simple Scan nor XSane detect the scanner. I'm running Ubuntu 12.04 (plus updates and backports) with kernel 3.2.0-31-generic. dmesg tells me this when I plus the cable in: [256411.641910] usb 7-1: new full-speed USB device number 10 using uhci_hcd [256411.872392] usblp0: USB Bidirectional printer dev 10 if 0 alt 1 proto 2 vid 0x067B pid 0x2305 [256411.872417] usbcore: registered new interface driver usblp lsusb shows this device for my cable: Bus 007 Device 010: ID 067b:2305 Prolific Technology, Inc. PL2305 Parallel Port The device node created is /dev/usb/lp0 crw-rw---- 1 root lp 180, 0 Sep 9 17:46 /dev/usb/lp0 There is no extra information from any of these commands when I attach the scanner to the cable and power it on, though. I suspect I might need to change something in /etc/sane.d/canon_pp.conf, but I have no idea what to put for the ieee1284 line, since there is pretty much zero documentation for that parameter. So how can I get it to work?

  Read the article

• Intel lance Parallel Studio XE 2013 et Cluster Studio XE 2013, ses suites d'outils pour booster les applications parallèles

  Intel lance Parallel Studio XE 2013 et Cluster Studio XE 2013 Ses suites d'outils pour booster les applications parallèles De l'analyse des données analytiques en temps réel aux traitements d'un volume important de données scientifiques, le parallélisme occupe une part de plus en plus importante dans le monde du développement. Des outils tels que ceux qu'Intel vient d'annoncer permettent d'optimiser et d'analyser les applications parallèles, réputées pour leurs grandes complexités. Il s'agit de Parallel Studio XE 2013 et Cluster Studio XE 2013, pour les langages C/C++ et Fortran sous Windows et Linux. [IMG]http://idelways.developpez.com/news/images/intel...

  Read the article

• How to get around batch file processing limit

  - by Patrick Cuff

  I have a Windows batch file that processes all the files in a given directory. I have 206,783 files I need to process: for %%f in (*.xml) do call :PROCESS %%f goto :STOP :PROCESS :: do something with the file program.exe %1 > %1.new set /a COUNTER=%COUNTER%+1 goto :EOF :STOP @echo %COUNTER% files processed When I run the batch file, the following output is written: 65535 files processed As part of the processing, an output file is created for each file procesed, with a .new extension. When I do a dir *.new it reports 65,535 files exist. So, it appears my command environment has a hard limit on the number of files it can recognize, and that limit is 64K - 1. Is there a way to extend the command environment to manage more than 64K - 1 files? If not, would a VBScript or JavaScript be able to process all 206,783 files? I'm running on Windows 2003 server, Enterprise Edition, 32-bit. UPDATE It looks like the root cause of my issue was with the built-in Windows "extract" command for ZIP files. The files I have to process were copied from another system via a ZIP file. My server doesn't have a ZIP utility installed, just the native Windows commands. I right-clicked on the ZIP file, and did an "Extract all...", which apparently just extracted the first 65,535 files. I downloaded and installed 7-zip onto my server, unzipped all the files, and my batch script worked as intended.

  Read the article

• Unable to load huge XML document (incorrectly suppose it's due to the XSLT processing)

  - by krisvandenbergh

  I'm trying to match certain elements using XSLT. My input document is very large and the source XML fails to load after processing the following code (consider especially the first line). <xsl:template match="XMI/XMI.content/Model_Management.Model/Foundation.Core.Namespace.ownedElement/Model_Management.Package/Foundation.Core.Namespace.ownedElement"> <rdf:RDF> <rdf:Description rdf:about=""> <xsl:for-each select="Foundation.Core.Class"> <xsl:for-each select="Foundation.Core.ModelElement.name"> <owl:Class rdf:ID="@Foundation.Core.ModelElement.name" /> </xsl:for-each> </xsl:for-each> </rdf:Description> </rdf:RDF> </xsl:template> Apparently the XSLT fails to load after "Model_Management.Model". The PHP code is as follows: if ($xml->loadXML($source_xml) == false) { die('Failed to load source XML: ' . $http_file); } It then fails to perform loadXML and immediately dies. I think there are two options now. 1) I should set a maximum executing time. Frankly, I don't know how that I do this for the built-in PHP 5 XSLT processor. 2) Think about another way to match. What would be the best way to deal with this? The input document can be found at http://krisvandenbergh.be/uml_pricing.xml Any help would be appreciated! Thanks.

  Read the article

• Apache module, is it possible to have asynchronous processing

  - by prashant2361

  Hi, I have a requirement where I need to send continous updates to my clients. Client is browser in this case. We have some data which updates every sec, so once client connects to our server, we maintain a persistent connection and keep pushing data to the client. I am looking for suggestions of this implementation at the server end. Basically what I need is this: 1. client connects to server. I maintain the socket and metadata about the socket. metadata contains what updates need to be send to this client 2. server process now waits for new client connections 3. One other process will have the list of all the sockets opened and will go through each of them and send the updates if required. Can we do something like this in apache module: 1. apache process gets the new connection. It maintains the state for the connection. It keeps the state in some global memory and returns back to root process to signify that it is done so that it can accept the new connection 2. the apache process though has returned the status to root process but it is also executing parallely where it going through its global store and sending updates to the client, if any. So can a apache process do these things: 1. Have more than one connection associated with it 2. Asynchronously waiting for new connection and at the same time processing the previous connections? Regards Prashant

  Read the article

• Objective C code to handle large amount of data processing in iPhone

  - by user167662

  I had the following code that takes in 14 mb or more of image data encoded in base4 string and converts them to jpeg before writing to a file in iphone. It crashes my program giving the following error : Program received signal: “0”. warning: check_safe_call: could not restore current frame I tweak my program and it can process a few more images before the error appear again. My coding is as follows: // parameters is an array where the fourth element contains a list of images in base64 >encoded string NSMutableArray *imageStrList = (NSMutableArray*) [parameters objectAtIndex:5]; while (imageStrList.count != 0) { NSString *imgString = [imageStrList objectAtIndex:0]; // Create a file name using my own Utility class NSString *fileName = [Utility generateFileNName]; NSData *restoredImg = [NSData decodeWebSafeBase64ForString:imgString]; UIImage *img = [UIImage imageWithData: restoredImg]; NSData *imgJPEG = UIImageJPEGRepresentation(img, 0.4f); [imgJPEG writeToFile:fileName atomically:YES]; [imageStrList removeObjectAtIndex:0]; } I tried playing around with UIImageJPEGRepresentation and found out that the lower the value, the more image it can processed but this should not be the way. I am wondering if there is anyway to free up memory of the imageStrList immediately after processing each image so that it can be used by the next one in the line.

  Read the article

• Program structure in long running data processing python script

  - by fmark

  For my current job I am writing some long-running (think hours to days) scripts that do CPU intensive data-processing. The program flow is very simple - it proceeds into the main loop, completes the main loop, saves output and terminates: The basic structure of my programs tends to be like so: <import statements> <constant declarations> <misc function declarations> def main(): for blah in blahs(): <lots of local variables> <lots of tightly coupled computation> for something in somethings(): <lots more local variables> <lots more computation> <etc., etc.> <save results> if __name__ == "__main__": main() This gets unmanageable quickly, so I want to refactor it into something more manageable. I want to make this more maintainable, without sacrificing execution speed. Each chuck of code relies on a large number of variables however, so refactoring parts of the computation out to functions would make parameters list grow out of hand very quickly. Should I put this sort of code into a python class, and change the local variables into class variables? It doesn't make a great deal of sense tp me conceptually to turn the program into a class, as the class would never be reused, and only one instance would ever be created per instance. What is the best practice structure for this kind of program? I am using python but the question is relatively language-agnostic, assuming a modern object-oriented language features.

  Read the article

• Processing more than one button click at Android Widget

  - by dive

  Hi, all. I saw this topic and implement IntentService as describes, but what if I want more that one button? How can I distinguish button from each other? I'm trying to setFlags, but cannot read it at onHandleIntent() method: public static class UpdateService extends IntentService { ... @Override public void onHandleIntent(Intent intent) { ComponentName me = new ComponentName(this, ExampleProvider.class); AppWidgetManager manager = AppWidgetManager.getInstance(this); manager.updateAppWidget(me, buildUpdate(this)); } private RemoteViews buildUpdate(Context context) { RemoteViews updateViews = new RemoteViews(context.getPackageName(), R.layout.main_layout); Intent i = new Intent(this, ExampleProvider.class); PendingIntent pi = PendingIntent.getBroadcast(context, 0, i, 0); updateViews.setOnClickPendingIntent(R.id.button_refresh, pi); i = new Intent(this, ExampleProvider.class); pi = PendingIntent.getBroadcast(context, 0, i, 0); updateViews.setOnClickPendingIntent(R.id.button_about, pi); return updateViews; } } At this little piece of code I have two PendingIntent linked with setOnClickPendingIntent, can I distinguish this intent for different actions and processing? Thanks for help

  Read the article

• Processing velocity-vectors during collision as neatly as possible

  - by DevEight

  Hello. I'm trying to create a good way to handle all possible collisions between two objects. Typically one will be moving and hitting the other, and should then "bounce" away. What I've done so far (I'm creating a typical game where you have a board and bounce a ball at bricks) is to check if the rectangles intersect and if they do, invert the Y-velocity. This is a really ugly and temporary solution that won't work in the long haul and since this is kind of processing is very common in games I'd really like to find a great way of doing this for future projects aswell. Any links or helpful info is appreciated. Below is what my collision-handling function looks like right now. protected void collision() { #region Boundaries if (bal.position.X + bal.velocity.X >= viewportRect.Width || bal.position.X + bal.velocity.X <= 0) { bal.velocity.X *= -1; } if (bal.position.Y + bal.velocity.Y <= 0) { bal.velocity.Y *= -1; } #endregion bal.rect = new Rectangle((int)bal.position.X+(int)bal.velocity.X-bal.sprite.Width/2, (int)bal.position.Y-bal.sprite.Height/2+(int)bal.velocity.Y, bal.sprite.Width, bal.sprite.Height); player.rect = new Rectangle((int)player.position.X-player.sprite.Width/2, (int)player.position.Y-player.sprite.Height/2, player.sprite.Width, player.sprite.Height); if (bal.rect.Intersects(player.rect)) { bal.position.Y = player.position.Y - player.sprite.Height / 2 - bal.sprite.Height / 2; if (player.position.X != player.prevPos.X) { bal.velocity.X -= (player.prevPos.X - player.position.X) / 2; } bal.velocity.Y *= -1; } foreach (Brick b in brickArray.list) { b.rect.X = Convert.ToInt32(b.position.X-b.sprite.Width/2); b.rect.Y = Convert.ToInt32(b.position.Y-b.sprite.Height/2); if (bal.rect.Intersects(b.rect)) { b.recieveHit(); bal.velocity.Y *= -1; } } brickArray.removeDead(); }

  Read the article

• Java, Massive message processing with queue manager (trading)

  - by Ronny

  Hello, I would like to design a simple application (without j2ee and jms) that can process massive amount of messages (like in trading systems) I have created a service that can receive messages and place them in a queue to so that the system won't stuck when overloaded. Then I created a service (QueueService) that wraps the queue and has a pop method that pops out a message from the queue and if there is no messages returns null, this method is marked as "synchronized" for the next step. I have created a class that knows how process the message (MessageHandler) and another class that can "listen" for messages in a new thread (MessageListener). The thread has a "while(true)" and all the time tries to pop a message. If a message was returned, the thread calls the MessageHandler class and when it's done, he will ask for another message. Now, I have configured the application to open 10 MessageListener to allow multi message processing. I have now 10 threads that all time are in a loop. Is that a good design?? Can anyone reference me to some books or sites how to handle such scenario?? Thanks, Ronny

  Read the article

• Processing forms that generate many rows in DB

  - by Zack

  I'm wondering what the best approach to take here is. I've got a form that people use to register for a class and a lot of times the manager of a company will register multiple people for the class at the same time. Presently, they'd have to go through the registration process multiple times and resubmit the form once for every person they want to register. What I want to do is give the user a form that has a single <input/> for one person to register with, along with all the other fields they'll need to fill out (Email, phone number, etc); if they want to add more people, they'll be able to press a button and a new <input/> will be generated. This part I know how to do, but I'm including it to best describe what I'm aiming to do. The part I don't know how to approach is processing that data the form submits, I need some way of making a new row in the Registrant table for every <input/> that's added and include the same contact information (phone, email, etc) as the first row with that row. For the record, I'm using the Django framework for my back-end code. What's the best approach here? Should it just POST the form x times for x people, or is there a less "brute force" way of handling this?

  Read the article

• How are you taking advantage of Multicore?

  - by tgamblin

  As someone in the world of HPC who came from the world of enterprise web development, I'm always curious to see how developers back in the "real world" are taking advantage of parallel computing. This is much more relevant now that all chips are going multicore, and it'll be even more relevant when there are thousands of cores on a chip instead of just a few. My questions are: How does this affect your software roadmap? I'm particularly interested in real stories about how multicore is affecting different software domains, so specify what kind of development you do in your answer (e.g. server side, client-side apps, scientific computing, etc). What are you doing with your existing code to take advantage of multicore machines, and what challenges have you faced? Are you using OpenMP, Erlang, Haskell, CUDA, TBB, UPC or something else? What do you plan to do as concurrency levels continue to increase, and how will you deal with hundreds or thousands of cores? If your domain doesn't easily benefit from parallel computation, then explaining why is interesting, too. Finally, I've framed this as a multicore question, but feel free to talk about other types of parallel computing. If you're porting part of your app to use MapReduce, or if MPI on large clusters is the paradigm for you, then definitely mention that, too. Update: If you do answer #5, mention whether you think things will change if there get to be more cores (100, 1000, etc) than you can feed with available memory bandwidth (seeing as how bandwidth is getting smaller and smaller per core). Can you still use the remaining cores for your application?

  Read the article

• How to decide between using PLINQ and LINQ at runtime?

  - by Hamish Grubijan

  Or decide between a parallel and a sequential operation in general. It is hard to know without testing whether parallel or sequential implementation is best due to overhead. Obviously it will take some time to train "the decider" which method to use. I would say that this method cannot be perfect, so it is probabilistic in nature. The x,y,z do influence "the decider". I think a very naive implementation would be to give both 1/2 chance at the beginning and then start favoring them according to past performance. This disregards x,y,z, however. I suspect that this question would be better answered by academics than practitioners. Anyhow, please share your heuristic, your experience if any, your tips on this. Sample code: public interface IComputer { decimal Compute(decimal x, decimal y, decimal z); } public class SequentialComputer : IComputer { public decimal Compute( ... // sequential implementation } public class ParallelComputer : IComputer { public decimal Compute( ... // parallel implementation } public class HybridComputer : IComputer { private SequentialComputer sc; private ParallelComputer pc; private TheDecider td; // Helps to decide between the two. public HybridComputer() { sc = new SequentialComputer(); pc = new ParallelComputer(); td = TheDecider(); } public decimal Compute(decimal x, decimal y, decimal z) { decimal result; decimal time; if (td.PickOneOfTwo() == 0) { // Time this and save result into time. result = sc.Compute(...); } else { // Time this and save result into time. result = pc.Compute(); } td.Train(time); return result; } }

  Read the article

• rsync to multiple destinations using same filelist?

  - by Dylan B.

  I'm wondering if it's possible for rsync to copy one directory to multiple remote destinations all in one go, or even in parallel. (not necessary, but would be useful.) Normally, something like the following would work just fine: $ rsync -Pav /junk user@host1:/backup $ rsync -Pav /junk user@host2:/backup $ rsync -Pav /junk user@host3:/backup And if that's the only option, I'll use that. However, /junk is located on a slow drive with quite a few files, and rebuilding the filelist of some ~12,000 files each time is agonizingly slow (~5 minutes) compared to the actual transfer/updating. Is it possible to do something like this, to accomplish the same thing: $ rsync -Pav /junk user@host1:/backup user@host2:/backup user@host3:/backup Thanks for looking!

  Read the article

• Building a new cluster for mathematical calculations (Win/Lin)

  - by Muhammad Farhan

  I would like to build a new cluster to perform heavy mathematical calculations in Matlab and Abaqus. One of my friend told me that distributed computing is way faster than parallel computing, which is very true after reading a bit on the internet. However, I have never clustered before. Current workstation I own: Dell Precision T5400 2 x Intel Xeon 2.5 GHz 16 GB RAM (2GB x 8) 1 x Western Digital 1TB HDD 7200 rpm 1 x nVidia Quadro FX4600 768MB GPU 1 x 870W PSU OS: Windows 7 Ultimate 64-bit 2nd WS: I can buy another WS similar configuration to the one I own I am not bothered about OS, I am willing to cluster with either Windows or Linux. However, my software are compatible with windows 64-bit only. Please help me setup a cluster. Thank you.

  Read the article

• How to execute a command on multiple hosts using IPv6 only?

  - by math

  First of all there is pdsh which is essentially a parallel distributed shell which may execute commands on a list of given hosts. However, I find myself in an IPv6 only problem setting. It seems that pdsh is not able to use IPv6, as I am getting error messages: pdsh -w ^hostnames my_command pdsh@myhost: gethostbyname("foobar") failed I also tried to use IPv6 addresses only, which also didn't work. So how do you run a single shell script for administrative purpose (no SGE stuff, or similar) on a bunch of hosts that is IPv6 reachable only?

  Read the article

< Previous Page | 26 27 28 29 30 31 32 33 34 35 36 37  | Next Page >