Search Results

Search found 14169 results on 567 pages for 'parallel programming'.

Page 3/567 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

• Embarrassingly parallel workflow creates too many output files

  - by Hooked

  On a Linux cluster I run many (N > 10^6) independent computations. Each computation takes only a few minutes and the output is a handful of lines. When N was small I was able to store each result in a separate file to be parsed later. With large N however, I find that I am wasting storage space (for the file creation) and simple commands like ls require extra care due to internal limits of bash: -bash: /bin/ls: Argument list too long. Each computation is required to run through a qsub scheduling algorithm so I am unable to create a master program which simply aggregates the output data to a single file. The simple solution of appending to a single fails when two programs finish at the same time and interleave their output. I have no admin access to the cluster, so installing a system-wide database is not an option. How can I collate the output data from embarrassingly parallel computation before it gets unmanageable?

  Read the article

• how a pure functional programming language manage without assignment statements?

  - by Gnijuohz

  When reading the famous SICP,I found the authors seem rather reluctant to introduce the assignment statement to Scheme in Chapter 3.I read the text and kind of understand why they feel so. As Scheme is the first functional programming language I ever know something about,I am kind of surprised that there are some functional programming languages(not Scheme of course) can do without assignments. Let use the example the book offers,the bank account example.If there is no assignment statement,how can this be done?How to change the balance variable?I ask so because I know there are some so-called pure functional languages out there and according to the Turing complete theory,this must can be done too. I learned C,Java,Python and use assignments a lot in every program I wrote.So it's really an eye-opening experience.I really hope someone can briefly explain how assignments are avoided in those functional programming languages and what profound impact(if any) it has on these languages. The example mentioned above is here: (define (make-withdraw balance) (lambda (amount) (if (>= balance amount) (begin (set! balance (- balance amount)) balance) "Insufficient funds"))) This changed the balance by set!.To me it looks a lot like a class method to change the class member balance. As I said,I am not familiar with functional programming languages,so if I said something wrong about them,feel free to point out.

  Read the article

• Parallel processing via multithreading in Java

  - by Robz

  There are certain algorithms whose running time can decrease significantly when one divides up a task and gets each part done in parallel. One of these algorithms is merge sort, where a list is divided into infinitesimally smaller parts and then recombined in a sorted order. I decided to do an experiment to test whether or not I could I increase the speed of this sort by using multiple threads. I am running the following functions in Java on a Quad-Core Dell with Windows Vista. One function (the control case) is simply recursive: // x is an array of N elements in random order public int[] mergeSort(int[] x) { if (x.length == 1) return x; // Dividing the array in half int[] a = new int[x.length/2]; int[] b = new int[x.length/2+((x.length%2 == 1)?1:0)]; for(int i = 0; i < x.length/2; i++) a[i] = x[i]; for(int i = 0; i < x.length/2+((x.length%2 == 1)?1:0); i++) b[i] = x[i+x.length/2]; // Sending them off to continue being divided mergeSort(a); mergeSort(b); // Recombining the two arrays int ia = 0, ib = 0, i = 0; while(ia != a.length || ib != b.length) { if (ia == a.length) { x[i] = b[ib]; ib++; } else if (ib == b.length) { x[i] = a[ia]; ia++; } else if (a[ia] < b[ib]) { x[i] = a[ia]; ia++; } else { x[i] = b[ib]; ib++; } i++; } return x; } The other is in the 'run' function of a class that extends thread, and recursively creates two new threads each time it is called: public class Merger extends Thread { int[] x; boolean finished; public Merger(int[] x) { this.x = x; } public void run() { if (x.length == 1) { finished = true; return; } // Divide the array in half int[] a = new int[x.length/2]; int[] b = new int[x.length/2+((x.length%2 == 1)?1:0)]; for(int i = 0; i < x.length/2; i++) a[i] = x[i]; for(int i = 0; i < x.length/2+((x.length%2 == 1)?1:0); i++) b[i] = x[i+x.length/2]; // Begin two threads to continue to divide the array Merger ma = new Merger(a); ma.run(); Merger mb = new Merger(b); mb.run(); // Wait for the two other threads to finish while(!ma.finished || !mb.finished) ; // Recombine the two arrays int ia = 0, ib = 0, i = 0; while(ia != a.length || ib != b.length) { if (ia == a.length) { x[i] = b[ib]; ib++; } else if (ib == b.length) { x[i] = a[ia]; ia++; } else if (a[ia] < b[ib]) { x[i] = a[ia]; ia++; } else { x[i] = b[ib]; ib++; } i++; } finished = true; } } It turns out that function that does not use multithreading actually runs faster. Why? Does the operating system and the java virtual machine not "communicate" effectively enough to place the different threads on different cores? Or am I missing something obvious?

  Read the article

• Should functional programming be taught before imperative programming?

  - by Zifre

  It seems to me that functional programming is a great thing. It eliminates state and makes it much easier to automatically make code run in parallel. Many programmers who were first taught imperative programming styles find it very difficult to learn functional programming, because it is so different. I began to wonder if programmers who were taught functional programming first would find it hard to begin imperative programming. It seems like it would not be as hard as the other way around, so I thought it would be a good thing if more programmers were taught functional programming first. So, my question is, should functional programming be taught in school before imperative, and if so, why is it not more common to start with it?

  Read the article

• Parallel programming patterns for C#?

  - by VoidDweller

  With Intel's launch of a Hexa-Core processor for the desktop, it looks like we can no longer wait for Microsoft to make many-core programming "easy". I just order a copy of Joe Duffy's book Concurrent Programming on Windows. This looks like a great place to start, though, I am hoping some of you who have been targeting multi/many core systems would point me to some good resources that have or would have helped on your projects?

  Read the article

• Is LLVM suitable for parallel languages?

  - by DSblizzard

  What properties of LLVM makes it good choice for implementation of (parallel, concurrent, distributed)-oriented language, what makes it bad?

  Read the article

• How to join the World of Programming? [closed]

  - by litebread

  Name's Vlad and I am currently on my third year of Community College, studying Computer Science with emphasis on Programming in C++ and Networking. I have completed a few programming courses with general ease, but have not gained advanced understanding of programming through school. None of my friends are serious programmers working in the industry. Being an active lurker on many programming websites, and in general tech oriented sites I have noticed how little I know about the industry, the lingo and terminology. (I have no clue how Git hub works, but I generally understand what its for). So I am looking for help as to where I should look for information on the programming world and the industry in which I a very interested. By that I mean, what sites I should utilize to gain information on programming practices, introduction to advanced C++ and resources that simply introduce a 20some programming noob. I like programming, but I haven't dug my hands deep into it yet, I want to start to do so before I transfer to a University. All in all, where do I find information on becoming an actual programmer (Information that lays out a path). Thank you for reading. Have a great day!

  Read the article

• Async.Parallel or Array.Parallel.Map ?

  - by gurteen2

  Hello- I'm trying to implement a pattern I read from Don Syme's blog (http://blogs.msdn.com/dsyme/archive/2010/01/09/async-and-parallel-design-patterns-in-f-parallelizing-cpu-and-i-o-computations.aspx) which suggests that there are opportunities for massive performance improvements from leveraging asynchronous I/O. I am currently trying to take a piece of code that "works" one way, using Array.Parallel.Map, and see if I can somehow achieve the same result using Async.Parallel, but I really don't understand Async.Parallel, and cannot get anything to work. I have a piece of code (simplified below to illustrate the point) that successfully retrieves an array of data for one cusip. (A price series, for example) let getStockData cusip = let D = DataProvider() let arr = D.GetPriceSeries(cusip) return arr let data = Array.Parallel.map (fun x -> getStockData x) stockCusips So this approach contructs an array of arrays, by making a connection over the internet to my data vendor for each stock (which could be as many as 3000) and returns me an array of arrays (1 per stock, with a price series for each one). I admittedly don't understand what goes on underneath Array.Parallel.map, but am wondering if this is a scenario where there are resources wasted under the hood, and it actually could be faster using asynchronous I/O? So to test this out, I have attempted to make this function using asyncs, and I think that the function below follows the pattern in Don Syme's article using the URLs, but it won't compile with "let!". let getStockDataAsync cusip = async { let D = DataProvider() let! arr = D.GetData(cusip) return arr } The error I get is: This expression was expected to have type Async<'a but here has type obj It compiles fine with "let" instead of "let!", but I had thought the whole point was that you need the exclamation point in order for the command to run without blocking a thread. So the first question really is, what's wrong with my syntax above, in getStockDataAsync, and then at a higher level, can anyone offer some additional insight about asychronous I/O and whether the scenario I have presented would benefit from it, making it potentially much, much faster than Array.Parallel.map? Thanks so much.

  Read the article

• What could be my path? Networking, programming, or something else?

  - by momong

  Well first and foremost, I would like to give my brief description: I was an aviation student but I didn't pursue that path because I lost my interest. Now I'm an I.T. student and currently stopped schooling because of confusion. I don't know which path I should choose: could it be programming or networking? Someone told me that on networking the money is easy, the job is easy. Others told me that programming is best suited for me because I'm very skilled and excellent at figures. I want to chose networking, but I can't find my passion for it, my mind tells me but my heart doesn't... and on programming, I don't know which language I should pick or if I like it or not. A good mentor, even if only online, would be a very big plus to me, but I don't think if there are many who could spent their time on teaching a nobody... but I'm very eager to learn. My real passion is gaming! I want to work in the gaming industry, I want to be a man behind those games! I've been a gamer freak since birth. But I don't know how to get in to that industry. I don't know what to do. I don't know which path would really suit me. Sorry if some of you find this a pointless question, but please bear with me, this could be the turn of my life.

  Read the article

• First languages with generic programming support

  - by oluies

  Which was the first language with generic programming support, and what was the first major staticly typed language (widely used) with generics support. Generics implement the concept of parameterized types to allow for multiple types. The term generic means "pertaining to or appropriate to large groups of classes." I have seen the following mentions of "first": First-order parametric polymorphism is now a standard element of statically typed programming languages. Starting with System F [20,42] and functional programming lan- guages, the constructs have found their way into mainstream languages such as Java and C#. In these languages, first-order parametric polymorphism is usually called generics. From "Generics of a Higher Kind", Adriaan Moors, Frank Piessens, and Martin Odersky Generic programming is a style of computer programming in which algorithms are written in terms of to-be-specified-later types that are then instantiated when needed for specific types provided as parameters. This approach, pioneered by Ada in 1983 From Wikipedia Generic Programming

  Read the article

• Programming Language, Turing Completeness and Turing Machine

  - by Amumu

  A programming language is said to be Turing Completeness if it can successfully simulate a universal TM. Let's take functional programming language for example. In functional programming, function has highest priority over anything. You can pass functions around like any primitives or objects. This is called first class function. In functional programming, your function does not produce side effect i.e. output strings onto screen, change the state of variables outside of its scope. Each function has a copy of its own objects if the objects are passed from the outside, and the copied objects are returned once the function finishes its job. Each function written purely in functional style is completely independent to anything outside of it. Thus, the complexity of the overall system is reduced. This is referred as referential transparency. In functional programming, each function can have its local variables kept its values even after the function exits. This is done by the garbage collector. The value can be reused the next time the function is called again. This is called memoization. A function usually should solve only one thing. It should model only one algorithm to answer a problem. Do you think that a function in a functional language with above properties simulate a Turing Machines? Functions (= algorithms = Turing Machines) are able to be passed around as input and returned as output. TM also accepts and simulate other TMs Memoization models the set of states of a Turing Machine. The memorized variables can be used to determine states of a TM (i.e. which lines to execute, what behavior should it take in a give state ...). Also, you can use memoization to simulate your internal tape storage. In language like C/C++, when a function exits, you lose all of its internal data (unless you store it elsewhere outside of its scope). The set of symbols are the set of all strings in a programming language, which is the higher level and human-readable version of machine code (opcode) Start state is the beginning of the function. However, with memoization, start state can be determined by memoization or if you want, switch/if-else statement in imperative programming language. But then, you can't Final accepting state when the function returns a value, or rejects if an exception happens. Thus, the function (= algorithm = TM) is decidable. Otherwise, it's undecidable. I'm not sure about this. What do you think? Is my thinking true on all of this? The reason I bring function in functional programming because I think it's closer to the idea of TM. What experience with other programming languages do you have which make you feel the idea of TM and the ideas of Computer Science in general? Can you specify how you think?

  Read the article

• First languages with generic programming support

  - by oluies

  Which was the first language with generic programming support, and what was the first major staticly typed language (widely used) with generics support. Generics implement the concept of parameterized types to allow for multiple types. The term generic means "pertaining to or appropriate to large groups of classes." I have seen the following mentions of "first": First-order parametric polymorphism is now a standard element of statically typed programming languages. Starting with System F [20,42] and functional programming lan- guages, the constructs have found their way into mainstream languages such as Java and C#. In these languages, first-order parametric polymorphism is usually called generics. From "Generics of a Higher Kind", Adriaan Moors, Frank Piessens, and Martin Odersky Generic programming is a style of computer programming in which algorithms are written in terms of to-be-specified-later types that are then instantiated when needed for specific types provided as parameters. This approach, pioneered by Ada in 1983 From Wikipedia Generic Programming

  Read the article

• Parallel Desktops: installing Parallel Tools on Ubuntu

  - by Patrick

  hi, I get the following error when I try to install Parallel Tools on my Ubuntu in Parallel Desktop. I follow the istructions, running sh install from terminal: I follow the UI istructions and then the installation stops with this error message: E: Couldn't find package dkms Fri May 7 14:34:20 PDT 2010 Start installation or upgrade of Guest Tools Installed Guest Tools were not found Perform installation into the /usr/lib/parallels-tools directory cat: /usr/lib/parallels-tools/kmods/../version: No such file or directory Start installation of prl_eth kernel module make: Entering directory `/usr/lib/parallels-tools/kmods' cd prl_eth/pvmnet && make make[1]: Entering directory `/usr/lib/parallels-tools/kmods/prl_eth/pvmnet' make -C /lib/modules/2.6.32-21-generic/build M=/usr/lib/parallels-tools/kmods/prl_eth/pvmnet make[2]: Entering directory `/usr/src/linux-headers-2.6.32-21-generic' LD /usr/lib/parallels-tools/kmods/prl_eth/pvmnet/built-in.o CC [M] /usr/lib/parallels-tools/kmods/prl_eth/pvmnet/pvmnet.o LD [M] /usr/lib/parallels-tools/kmods/prl_eth/pvmnet/prl_eth.o Building modules, stage 2. MODPOST 1 modules WARNING: modpost: missing MODULE_LICENSE() in /usr/lib/parallels-tools/kmods/prl_eth/pvmnet/prl_eth.o thanks

  Read the article

• Inauguration Of My Laptop

  - by Pawan_Mishra

  Today I received my new laptop which is an Intel Core i5-2450M @ 2.50GHz 4 GB RAM machine . The other laptop(office provided) which I have used for past two years for programming is an Intel Core2 Duo T6570 @ 2.10GHz machine. Reason why I am talking about the laptops that I own is because of my interest in writing multi-threaded/parallel code using the new TPL API provided in the .Net 4.0 framework. I have spent significant amount of time in past one year writing code using the Parallel API of .Net...(read more)

  Read the article

• Combining TBB and IPP with Intel Parallel Studio

  When performance really counts Parallel computing - Intel Parallel Studio - Hardware - Programming - Components

  Read the article

• Designing Parallel Algorithms: Part 1

  Parallel programming as an engineering discipline Engineering - Parallel computing - Programming - Algorithm - Languages

  Read the article

• Granularity and Parallel Performance

  Determining the right granularity for parallel tasks Granularity - Parallel computing - Programming - Vendors - Projects

  Read the article

• Composing programs from small simple pieces: OOP vs Functional Programming

  - by Jay Godse

  I started programming when imperative programming languages such as C were virtually the only game in town for paid gigs. I'm not a computer scientist by training so I was only exposed to Assembler and Pascal in school, and not Lisp or Prolog. Over the 1990s, Object-Oriented Programming (OOP) became more popular because one of the marketing memes for OOP was that complex programs could be composed of loosely coupled but well-defined, well-tested, cohesive, and reusable classes and objects. And in many cases that is quite true. Once I learned object-oriented programming my C programs became better because I structured them more like classes and objects. In the last few years (2008-2014) I have programmed in Ruby, an OOP language. However, Ruby has many functional programming (FP) features such as lambdas and procs, which enable a different style of programming using recursion, currying, lazy evaluation and the like. (Through ignorance I am at a loss to explain why these techniques are so great). Very recently, I have written code to use methods from the Ruby Enumerable library, such as map(), reduce(), and select(). Apparently this is a functional style of programming. I have found that using these methods significantly reduce code volume, and make my code easier to debug. Upon reading more about FP, one of the marketing claims made by advocates is that FP enables developers to compose programs out of small well-defined, well-tested, and reusable functions, which leads to less buggy code, and low code volume. QUESTIONS: Is the composition of complex program by using FP techniques contradictory to or complementary to composition of a complex program by using OOP techniques? In which situations is OOP more effective, and when is FP more effective? Is it possible to use both techniques in the same complex program? Do the techniques overlap or contradict each other?

  Read the article

• Recommendations for Open Source Parallel programming IDE

  - by Andrew Bolster

  What are the best IDE's / IDE plugins / Tools, etc for programming with CUDA / MPI etc? I've been working in these frameworks for a short while but feel like the IDE could be doing more heavy lifting in terms of scaling and job processing interactions. (I usually use Eclipse or Netbeans, and usually in C/C++ with occasional Java, and its a vague question but I can't think of any more specific way to put it)

  Read the article

• New book in the style of Advanced Programming Language Design by R. A. Finkel [closed]

  - by mfellner

  I am currently researching visual programming language design for a university paper and came across Advanced Programming Language Design by Raphael A. Finkel from 1996. Other, older discussions in the same vein on Stackoverflow have mentioned Language Implementation Patterns by Terence Parr and Programming Language Pragmatics* by Michael L. Scott. I was wondering if there is even more (and especially up-to-date) literature on the general topic of programming language design. *) http://www.cs.rochester.edu/~scott/pragmatics/

  Read the article

• Game programming course materials: What should it include?

  - by Esa

  I am tasked to create the course materials for a game programming class, and I’d like your opinion on what aspects and areas of game programming, such as game state management, game object storing or simple AI, should I include in it? The course is intented to be the first step into game programming for students with novice skills in programming. There will be mathematics as well, but I found that there are multiple questions, with good answers, on that subject already.

  Read the article

• How do i write tasks? (parallel code)

  - by acidzombie24

  I am impressed with intel thread building blocks. I like how i should write task and not thread code and i like how it works under the hood with my limited understanding (task are in a pool, there wont be 100 threads on 4cores, a task is not guaranteed to run because it isnt on its own thread and may be far into the pool. But it may be run with another related task so you cant do bad things like typical thread unsafe code). I wanted to know more about writing task. I like the 'Task-based Multithreading - How to Program for 100 cores' video here http://www.gdcvault.com/sponsor.php?sponsor_id=1 (currently second last link. WARNING it isnt 'great'). My fav part was 'solving the maze is better done in parallel' which is around the 48min mark (you can click the link on the left side. That part is really all you need to watch if any). However i like to see more code examples and some API of how to write task. Does anyone have a good resource? I have no idea how a class or pieces of code may look after pushing it onto a pool or how weird code may look when you need to make a copy of everything and how much of everything is pushed onto a pool.

  Read the article

• How do i write task? (parallel code)

  - by acidzombie24

  I am impressed with intel thread building blocks. I like how i should write task and not thread code and i like how it works under the hood with my limited understanding (task are in a pool, there wont be 100 threads on 4cores, a task is not guaranteed to run because it isnt on its own thread and may be far into the pool. But it may be run with another related task so you cant do bad things like typical thread unsafe code). I wanted to know more about writing task. I like the 'Task-based Multithreading - How to Program for 100 cores' video here http://www.gdcvault.com/sponsor.php?sponsor_id=1 (currently second last link. WARNING it isnt 'great'). My fav part was 'solving the maze is better done in parallel' which is around the 48min mark (you can click the link on the left side). However i like to see more code examples and some API of how to write task. Does anyone have a good resource? I have no idea how a class or pieces of code may look after pushing it onto a pool or how weird code may look when you need to make a copy of everything and how much of everything is pushed onto a pool.

  Read the article

• Code Monster Helps Introduce Kids (and Curious Adults) to the Basics of Programming

  - by Jason Fitzpatrick

  If you’re looking for a fun way to introduce a kid to programming (or sate your own curiosity), Crunchzilla’s Code Monster is a real-time introduction to basic programming concepts. How does Code Monster work? Users are guided through the programming experience (using JavaScript) by a talkative blue monster that asks questions about the code and suggests courses of action. Play long enough and you travel from simple variables to more complex ideas like conditionals, expressions, and more. It’s not a comprehensive programming curriculum (nor does it claim to be) but it’s a great way to introduce people of all ages to programming. Hit up the link below to take it for a spin. Code Monster [via O'Reilly Radar] 8 Deadly Commands You Should Never Run on Linux 14 Special Google Searches That Show Instant Answers How To Create a Customized Windows 7 Installation Disc With Integrated Updates

  Read the article

• From Imperative to Functional Programming

  - by user66569

  As an Electronic Engineer, my programming experience started with Assembly and continue with PL/M, C, C++, Delphi, Java, C# among others (imperative programming is in my blood). I'm interested in add to my previous knowledge, skills about functional programming, but all I've seen until now seems very obfuscated and esoteric. Can you please answer me these questions? 1) What is the mainstream functional programming language today (I don't want to get lost myself studying a plethora of FP languages, just because language X has the feature Y)? 2) What was the first FP language (the Fortran of functional programming if you want)? 3) Finally, when talking about pure vs. non pure FP what are the mainstream languages of each category? Thank you in advance

  Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >