executorservice to read data from database in chuncks and run process on them
Posted
by
TazMan
on Stack Overflow
See other posts from Stack Overflow
or by TazMan
Published on 2014-06-03T03:21:34Z
Indexed on
2014/06/03
3:25 UTC
Read the original article
Hit count: 157
I'm trying to write a process that would read data from a database and upload it onto a cloud datastore.
How can I decide the partition strategy of the data? I want to query the table in chunks and process each chunk in 10 threads. Each thread basically will send the data to an individual node on a 10 node cluster on the cloud..
Where in the below multi threading code will the dataquery to extract and send 10 concurrent requests for uploading data to cloud would be?
public class Caller {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
Runnable worker = new DomainCDCProcessor(i);
executor.execute(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {
}
System.out.println("Finished all threads");
}
}
© Stack Overflow or respective owner