Look up values in a BDB for several files in parallel

Posted by biznez on Stack Overflow See other posts from Stack Overflow or by biznez
Published on 2009-09-15T17:03:20Z Indexed on 2010/04/07 21:53 UTC
Read the original article Hit count: 177

Filed under:

berkeley-db

|

multithreading

|

hadoop

What is the most efficient way to look up values in a BDB for several files in parallel? If I had a Perl script which did this for one file at a time, would forking/running the process in background with the ampersand in Linux work?

How might Hadoop be used to solve this problem?

Would threading be another solution?

© Stack Overflow or respective owner

Related posts about berkeley-db

Megjelent a Berkeley DB 11gR2 verziója

as seen on Oracle Blogs - Search for 'Oracle Blogs'
Kedden jelent meg az Oracle Berkeley DB legújabb, 11gR2 verziója. A Berkeley DB a piacvezeto nyílt forráskódú beágyazható adatbázis-kezelo. Mivel a Berkeley DB egy library formájában érheto el, így közvetlenül az alkalmazásba linkelheto, ennek köszönheto a rendkívül nagy teljesítmény és a zéró adminisztráció… >>> More
Is Berkeley DB a NoSQL solution?

as seen on Oracle Blogs - Search for 'Oracle Blogs'
Berkeley DB is a library. To use it to store data you must link the library into your application. You can use most programming languages to access the API, the calls across these APIs generally mimic the Berkeley DB C-API which makes perfect sense because Berkeley DB is written in C. The inspiration… >>> More
Choosing between Berkeley DB Core and Berkeley DB JE

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm designing a Java based web-app and I need a key-value store. Berkeley DB seems fitting enough for me, but there appears to be TWO Berkeley DBs to choose from: Berkeley DB Core which is implemented in C, and Berkeley DB Java Edition which is implemented in pure Java. The question is, how to choose… >>> More
The C++ Standard Template Library as a BDB Database (part 1)

as seen on Oracle Blogs - Search for 'Oracle Blogs'
If you've used C++ you undoubtedly have used the Standard Template Libraries. Designed for in-memory management of data and collections of data this is a core aspect of all C++ programs. Berkeley DB is a database library with a variety of APIs designed to ease development, one of those APIs extends… >>> More
How to retrieve all the records from a Berkeley DB in Ruby

as seen on Stack Overflow - Search for 'Stack Overflow'
I'd like to be able to get all the key-values stored in a Berkeley DB using the Ruby bindings from http://github.com/mattbauer/bdb/tree/master but I'm not sure how to proceed. Any pointers will be appreciated. UPDATE Here's a small script that loops over the keys and prints them. Based on Pax' answer: require… >>> More

Related posts about multithreading

C++ Multithreading on Unix

as seen on Programmers - Search for 'Programmers'
I have two related questions: 1) Are there any good books for multithreading in C++, especially now that C++11 contains multithreading in the standard library? 2) I have the Wrox Programming on Unix book (1000 pages fat red one) and within it, it uses the Unix Thread class. How does this code relate… >>> More
what are the difficulties of operating system multithreading?

as seen on Stack Overflow - Search for 'Stack Overflow'
I am reading a book that compares two ways of implementing threads, Middleware Threads and OS Threads. I have a question about these sentences: "A difficulty of operating system multithreading, however, is performance overhead. Since it is the operating system that is involved in switching threads… >>> More
TCP multicast and multithreading

as seen on Stack Overflow - Search for 'Stack Overflow'
I need to come up with clients that can multicast to other clients reliably. That implies I'll be using TCP to connect reliably between clients within a multicast group. Doesn't that come up to n^2 number of connections? That seems a little silly to me. Wouldn't/shouldn't there be a way to more easily… >>> More
Multithreading recommendation based on program description

as seen on Stack Overflow - Search for 'Stack Overflow'
I would like to describe some specifics of my program and get feedback on what the best multithreading model to use would be most applicable. I've spent a lot of time now reading on ThreadPool, Threads, Producer/Consumer, etc. and have yet to come to solid conclusions. I have a list of files (all… >>> More
How to achieve multithreading using JavaScript with IE6?

as seen on Stack Overflow - Search for 'Stack Overflow'
How to achieve multithreading using JavaScript with IE6? Is there a third party library to do this? >>> More