executorservice to read data from database in chuncks and run process on them

Posted by TazMan on Stack Overflow See other posts from Stack Overflow or by TazMan
Published on 2014-06-03T03:21:34Z Indexed on 2014/06/03 3:25 UTC
Read the original article Hit count: 150

I'm trying to write a process that would read data from a database and upload it onto a cloud datastore.

How can I decide the partition strategy of the data? I want to query the table in chunks and process each chunk in 10 threads. Each thread basically will send the data to an individual node on a 10 node cluster on the cloud..

Where in the below multi threading code will the dataquery to extract and send 10 concurrent requests for uploading data to cloud would be?

public class Caller {
    public static void main(String[] args) {

        ExecutorService executor = Executors.newFixedThreadPool(10);

          for (int i = 0; i < 10; i++) {
                Runnable worker = new DomainCDCProcessor(i);
                executor.execute(worker);
              }
            executor.shutdown();
            while (!executor.isTerminated()) {
            }
            System.out.println("Finished all threads");
        }


    }

© Stack Overflow or respective owner

Related posts about java

Related posts about multithreading