Downloading Large JSON File to local file using Java

Posted by user1279675 on Stack Overflow See other posts from Stack Overflow or by user1279675
Published on 2012-03-19T23:18:56Z Indexed on 2012/03/19 23:29 UTC
Read the original article Hit count: 167

Filed under:

I'm attempting to download a JSON from the following URL - http://api.crunchbase.com/v/1/companies.js - to a local file. I'm using Java 1.7 and the following JSON Libraries - http://www.json.org/java/ - to attempt to make it work.

Here's my code:

public static void download(String address, String localFileName) {
            OutputStream out = null;
            URLConnection conn = null;
            InputStream  in = null;
            try {
                    URL url = new URL(address);
                    out = new BufferedOutputStream(
                            new FileOutputStream(localFileName));
                    conn = url.openConnection();
                    in = conn.getInputStream();
                    byte[] buffer = new byte[1024];
                    int numRead;
                    long numWritten = 0;
                    while ((numRead = in.read(buffer)) != -1)
                    {
                            out.write(buffer, 0, numRead);
                            numWritten += numRead;
                            System.out.println(buffer.length);
                            System.out.println(" " + buffer.hashCode());
                    }
                   System.out.println(localFileName + "\t" + numWritten);
            } catch (Exception exception) {
                    exception.printStackTrace();
            } finally {
                    try {
                            if (in != null) {
                                    in.close();
                            }
                            if (out != null) {
                                    out.close();
                            }
                    } catch (IOException ioe) {
                    }
            }
    }

When I run the code everything seems to work until midway through the loop the program seems to stop and not continue reading the JSON Object.

Does anyone know why this would stop reading? How could I fix the issue?

© Stack Overflow or respective owner