Search Results

Search found 3521 results on 141 pages for 'parallel computing'.

Page 42/141 | < Previous Page | 38 39 40 41 42 43 44 45 46 47 48 49  | Next Page >

  • Improve performance writing 10 million records to text file using windows service

    - by user1039583
    I'm fetching more than 10 millions of records from database and writing to a text file. It takes hours of time to complete this operation. Is there any option to use TPL features here? It would be great if someone could get me started implementing this with the TPL. using (FileStream fStream = new FileStream("d:\\file.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite)) { BufferedStream bStream = new BufferedStream(fStream); TextWriter writer = new StreamWriter(bStream); for (int i = 0; i < 100000000; i++) { writer.WriteLine(i); } bStream.Flush(); writer.Flush(); // empty buffer; fStream.Flush(); }

    Read the article

  • How can i connect two or more machines via tcp cable to form a network grid?

    - by Gath
    How can i connect two or more machines to form a network grid and how can i distribute work load to the two machines? What operating systems do i need to run on the machines, and what application should i use to manage the load balancing? NB: I read somewhere that google uses cheap machines to perform this fete, how do they connect two network cards( 'Teaming' ) and distribute load across the machines? Good practical examples would serve me good, with actual code samples. Pointers to some good site i might read this stuff will be highly appreciated.

    Read the article

  • Is a Service Bus a good option to communicate with multiple external servers?

    - by PFreitas
    We are developing an application that communicates, via web services or TCP/IP sockets, with multiple servers (up to 50 different external companies). Basically, the exchanged messages are the same (XML), but depending on the inputs of our application we should call 1 or more external servers. The benefits we would expect of introducing a Service Bus in the architecture would be: 1- Remove the need to manage all point-to-point configurations (all the 50 endpoints); 2- Simplify the communication layer of our application by having only one server to talk to; Is a Service Bus a good architectural option for this scenario? What is the best (simplest) Service Bus for this kind of communication? I read a few MSDN articles on Azure Service Bus Relay, but it didn’t seem to fit our needs. Am I wrong? Thanks for your help.

    Read the article

  • How can I merge two Linq IEnumerable<T> queries without running them?

    - by makerofthings7
    How do I merge a List<T> of TPL-based tasks for later execution? public async IEnumerable<Task<string>> CreateTasks(){ /* stuff*/ } My assumption is .Concat() but that doesn't seem to work: void MainTestApp() // Full sample available upon request. { List<string> nothingList = new List<string>(); nothingList.Add("whatever"); cts = new CancellationTokenSource(); delayedExecution = from str in nothingList select AccessTheWebAsync("", cts.Token); delayedExecution2 = from str in nothingList select AccessTheWebAsync("1", cts.Token); delayedExecution = delayedExecution.Concat(delayedExecution2); } /// SNIP async Task AccessTheWebAsync(string nothing, CancellationToken ct) { // return a Task } I want to make sure that this won't spawn any task or evaluate anything. In fact, I suppose I'm asking "what logically executes an IQueryable to something that returns data"? Background Since I'm doing recursion and I don't want to execute this until the correct time, what is the correct way to merge the results if called multiple times? If it matters I'm thinking of running this command to launch all the tasks var AllRunningDataTasks = results.ToList(); followed by this code: while (AllRunningDataTasks.Count > 0) { // Identify the first task that completes. Task<TableResult> firstFinishedTask = await Task.WhenAny(AllRunningDataTasks); // ***Remove the selected task from the list so that you don't // process it more than once. AllRunningDataTasks.Remove(firstFinishedTask); // TODO: Await the completed task. var taskOfTableResult = await firstFinishedTask; // Todo: (doen't work) TrustState thisState = (TrustState)firstFinishedTask.AsyncState; // TODO: Update the concurrent dictionary with data // thisState.QueryStartPoint + thisState.ThingToSearchFor Interlocked.Decrement(ref thisState.RunningDirectQueries); Interlocked.Increment(ref thisState.CompletedDirectQueries); if (thisState.RunningDirectQueries == 0) { thisState.TimeCompleted = DateTime.UtcNow; } }

    Read the article

  • MPI Odd/Even Compare-Split Deadlock

    - by erebel55
    I'm trying to write an MPI version of a program that runs an odd/even compare-split operation on n randomly generated elements. Process 0 should generated the elements and send nlocal of them to the other processes, (keeping the first nlocal for itself). From here, process 0 should print out it's results after running the CompareSplit algorithm. Then, receive the results from the other processes run of the algorithm. Finally, print out the results that it has just received. I have a large chunk of this already done, but I'm getting a deadlock that I can't seem to fix. I would greatly appreciate any hints that people could give me. Here is my code http://pastie.org/3742474 Right now I'm pretty sure that the deadlock is coming from the Send/Recv at lines 134 and 151. I've tried changing the Send to use "tag" instead of myrank for the tag parameter..but when I did that I just keep getting a "MPI_ERR_TAG: invalid tag" for some reason. Obviously I would also run the algorithm within the processors 0 but I took that part out for now, until I figure out what is going wrong. Any help is appreciated.

    Read the article

  • Executing functions parallelly in PHP

    - by binaryLV
    Hi! Can PHP call a function and don't wait for it to return? So something like this: function callback($pause, $arg) { sleep($pause); echo $arg, "\n"; } header('Content-Type: text/plain'); fast_call_user_func_array('callback', array(3, 'three')); fast_call_user_func_array('callback', array(2, 'two')); fast_call_user_func_array('callback', array(1, 'one')); would output one (after 1 second) two (after 2 seconds) three (after 3 seconds) rather than three (after 3 seconds) two (after 3 + 2 = 5 seconds) one (after 3 + 2 + 1 = 6 seconds) Main script is intended to be run as a permanent process (TCP server). callback() function would receive data from client, execute external PHP script and then do something based on other arguments that are passed to callback(). The problem is that main script must not wait for external PHP script to finish. Result of external script is important, so exec('php -f file.php &') is not an option.

    Read the article

  • Advantages of SQLServer vs. MySQL for C#/.NET4 Cloud Applications

    - by Ed Eichman
    I am considering building several C#/.NET4 applications all using a central, cloud based database. In addition, several LAMP (MySQL) web shops will be accessing the cloud DB. MySQL is the database that I'm most familiar with, and my default selection for the cloud DB would be MySQL on Amazon or Joyent. However, I was wondering what development "extras" are available for SQLServer in VisualStudio 2010 that are not available for MySQL. Are there any "killer features" that should make me consider SQLServer instead of MySQL?

    Read the article

  • Can a Perl subroutine return data but keep processing?

    - by Perl QuestionAsker
    Is there any way to have a subroutine send data back while still processing? For instance (this example used simply to illustrate) - a subroutine reads a file. While it is reading through the file, if some condition is met, then "return" that line and keep processing. I know there are those that will answer - why would you want to do that? and why don't you just ...?, but I really would like to know if this is possible. Thank you so much in advance.

    Read the article

  • Cluster of computers for rent?

    - by R S
    I am doing a project in the university which requires running of multiple instances (1000s) of a program I've written (in C++), which runs for quite a while (say 2 hours). The program is very self contained - it does not require input files, and the only dependency I think is boost. I'm currently using the university-owned cluster of computer. However, it's quite old and the jobs dispatching and monitors services are pretty bad. So I was wondering whether I can run my jobs elsewhere, for some money. For example, I looked a bit into Google App Engine, but as it seems every job must end after 30 seconds it is not suitable for me. Maybe Amazon EC2? Do you know of such options?

    Read the article

  • What alternatives are there to Google App Engine?

    - by Chris Marasti-Georg
    What alternatives are there to GAE, given that I already have a good bit of code working that I would like to keep. In other words, I'm digging python. However, my use case is more of a low number of requests, higher CPU usage type use case, and I'm worried that I may not be able to stay with App Engine forever. I have heard a lot of people talking about Amazon Web Services and other sorts of cloud providers, but I am having a hard time seeing where most of these other offerings provide the range of services (data querying, user authentication, automatic scaling) that App Engine provides. What are my options here?

    Read the article

  • How do I control output files name and content of an Hadoop streaming job?

    - by Eran Kampf
    Is there a way to control the output filenames of an Hadoop Streaming job? Specifically I would like my job's output files content and name to be organized by the ket the reducer outputs - each file would only contain values for one key and its name would be the key. Update: Just found the answer - Using a Java class that derives from MultipleOutputFormat as the jobs output format allows control of the output file names. http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputFormat.htmlhttp://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputFormat.html I havent seen any samples for this out there... Can anyone point out to an Hadoop Streaming sample that makes use of a custom output format Java class?

    Read the article

  • What kind of work benifits from OpenCL

    - by Daniel
    Hey All First of all: I am well aware that OpenCL does not magically make everything faster I am well aware that OpenCL has limitations So now to my question, i am used to do different scientific calculations using programming. Some of the things i work with is pretty intense in regards to the complexity and number of calculations. SO i was wondering, maybe i could speed things up bu using OpenCL. So, what i would love to hear from you all is answers to some of the following [bonus for links]: *What kind of calculations/algorithms/general problems is suitable for OpenCL *What is the general guidelines for determining if some particular code would benefit by migration to OpenCL? Regards

    Read the article

  • Simple C++ container class that is thread-safe for writing

    - by conradlee
    I am writing a multi-threaded program using OpenMP in C++. At one point my program forks into many threads, each of which need to add "jobs" to some container that keeps track of all added jobs. Each job can just be a pointer to some object. Basically, I just need the add pointers to some container from several threads at the same time. Is there a simple solution that performs well? After some googling, I found that STL containers are not thread-safe. Some stackoverflow threads address this question, but none that forms a consensus on a simple solution.

    Read the article

  • Understanding omission failure in distributed systems

    - by karthik A
    The following text says this which I'm not able to quite agree : client C sends a request R to server S. The time taken by a communication link to transport R over the link is D. P is the maximum time needed by S to recieve , process and reply to R. If omission failure is assumed ; then if no reply to R is received within 2(D+P) , then C will never recieve a reply to R . Why is the time here 2(D+P). As I understand shouldn't it be 2D+P ?

    Read the article

  • Who owes who money optimisation problem

    - by Francis
    Say you have n people, each who owe each other money. In general it should be possible to reduce the amount of transactions that need to take place. i.e. if X owes Y £4 and Y owes X £8, then Y only needs to pay X £4 (1 transaction instead of 2). This becomes harder when X owes Y, but Y owes Z who owes X as well. I can see that you can easily calculate one particular cycle. It helps for me when I think of it as a fully connected graph, with the nodes being the amount each person owes. Problem seems to be NP-complete, but what kind of optimisation algorithm could I make, nevertheless, to reduce the total amount of transactions? Doesn't have to be that efficient, as N is quite small for me.

    Read the article

  • Will "Programming in the Cloud" Ever Take off?

    - by Pierreten
    Just got a chance to try out a cloud programming environment that let's you develop .net apps within the browser (http://coderun.com/ide/) I found it pretty interesting, since I was able to develop a mockup ASP.net site on an IPad. With javascript engines in browsers becoming faster and faster, cheap server infrastructure to compile on; will the cloud become the IDE platform of the future?

    Read the article

  • Can I use MPI_Probe to probe messsages sent by any collective operation?

    - by takwing
    In my code I have a server process repeatedly probing for incoming messages, which come in two types. One type of the two will be sent once by each process to give hint to the server process about its termination. I was wondering if it is valid to use MPI_Broadcast to broadcast these termination messages and use MPI_Probe to probe their arrivals. I tried using this combination but it failed. This failure might have been caused by some other things. So I would like anyone who knows about this to confirm. Cheers.

    Read the article

  • How to provide an API client with 1,000,000 database results?

    - by Chris Dutrow
    What is a good way to provide an API client with 1,000,000 database results? We are cureently using PostgreSQL. A few suggested methods: Paging using Cursors Paging using random numbers ( Add "GREATER THAN ORDER BY " to each query ) Save information to a file and let the client download it Iterate through results, then POST the data to the client server Return only keys to the client, then let the client request the objects from Cloud files like Amazon S3 (still may require paging just to get the file names ). What haven't I thought of that is stupidly simple and way better than any of these options?

    Read the article

  • Does anyone know of any good tutorials for using the APIs from Amazon WeB Services, namely CloudWatc

    - by undefined
    Hi, I have been wrestling with Amazon's CloudWatch API with limited success. Does anyone know of any good resources (other than amazon's api docs) for using the APIs. I have tried to run them using the PHP library for CloudWatch but get nothing but error codes. I am configuring the GetMetricStatisticsSample.php file as follows: $request = array(); $endTime = date("Y-m-d G:i:s"); $yesterday = mktime (date("H"), date("i"), date("s"), date("m"), date("d")-1, date("Y")); $startTime = date("Y-m-d 00:00:00", $yesterday); $request["Statistics.member.1"] = "Average"; $request["EndTime"] = $endTime; $request["StartTime"] = $startTime; $request["MeasureName"] = "CPUUtilization"; $request["Unit"] = "Percent"; invokeGetMetricStatistics($service, $request); But this returns "Caught Exception: Internal Error Response Status Code: 400 Error Code: Error Type: Request ID: XML:" I have also tried from command line as follows - set JAVA_HOME=C:\Program Files\Java\jre1.6.0_05 set AWS_CLOUDWATCH_HOME=C:\AmazonWebServices\API_tools\CloudWatch-1.0.0.24 set PATH=%AWS_CLOUDWATCH_HOME%\bin mon-list-metrics but get C:|Program' is not recognized as an internal or external command... any suggestions? cheers

    Read the article

  • [C++] Needed: A simple C++ container (stack, linked list) that is thread-safe for writing

    - by conradlee
    I am writing a multi-threaded program using OpenMP in C++. At one point my program forks into many threads, each of which need to add "jobs" to some container that keeps track of all added jobs. Each job can just be a pointer to some object. Basically, I just need the add pointers to some container from several threads at the same time. Is there a simple solution that performs well? After some googling, I found that STL containers are not thread-safe. Some stackoverflow threads address this question, but none form a consensus on a simple solution.

    Read the article

  • Scientific Plotting in Python

    - by user100046
    I have a large data set of tuples containing (time of event, latitude, longitude) that I need to visualize. I was hoping to generate a 'movie'-like xy-plot, but was wondering if anyone has a better idea or if there is an easy way to do this in Python? Thanks in advance for the help, --Leo

    Read the article

  • Take advantage of multiple cores executing SQL statements

    - by willvv
    I have a small application that reads XML files and inserts the information on a SQL DB. There are ~ 300 000 files to import, each one with ~ 1000 records. I started the application on 20% of the files and it has been running for 18 hours now, I hope I can improve this time for the rest of the files. I'm not using a multi-thread approach, but since the computer I'm running the process on has 4 cores I was thinking on doing it to get some improvement on the performance (although I guess the main problem is the I/O and not only the processing). I was thinking on using the BeginExecutingNonQuery() method on the SqlCommand object I create for each insertion, but I don't know if I should limit the max amount of simultaneous threads (nor I know how to do it). What's your advice to get the best CPU utilization? Thanks

    Read the article

< Previous Page | 38 39 40 41 42 43 44 45 46 47 48 49  | Next Page >