Traditional IO vs memory-mapped

Posted by Senne on Stack Overflow See other posts from Stack Overflow or by Senne
Published on 2010-04-07T15:04:09Z Indexed on 2010/04/07 16:03 UTC
Read the original article Hit count: 367

Filed under:
|
|

I'm trying to illustrate the difference in performance between traditional IO and memory mapped files in java to students. I found an example somewhere on internet but not everything is clear to me, I don't even think all steps are nececery. I read a lot about it here and there but I'm not convinced about a correct implementation of neither of them.

The code I try to understand is:

public class FileCopy{
    public static void main(String args[]){
        if (args.length < 1){
            System.out.println(" Wrong usage!");
            System.out.println(" Correct usage is : java FileCopy <large file with full path>");
            System.exit(0);
        }


        String inFileName = args[0];
        File inFile = new File(inFileName);

        if (inFile.exists() != true){
            System.out.println(inFileName + " does not exist!");
            System.exit(0);
        }

        try{
            new FileCopy().memoryMappedCopy(inFileName, inFileName+".new" );
            new FileCopy().customBufferedCopy(inFileName, inFileName+".new1");
        }catch(FileNotFoundException fne){
            fne.printStackTrace();
        }catch(IOException ioe){
            ioe.printStackTrace();
        }catch (Exception e){
            e.printStackTrace();
        }


    }

    public void memoryMappedCopy(String fromFile, String toFile ) throws Exception{
        long timeIn = new Date().getTime();
        // read input file
        RandomAccessFile rafIn = new RandomAccessFile(fromFile, "rw");
        FileChannel fcIn = rafIn.getChannel();
        ByteBuffer byteBuffIn = fcIn.map(FileChannel.MapMode.READ_WRITE, 0,(int) fcIn.size());
        fcIn.read(byteBuffIn);
        byteBuffIn.flip();

        RandomAccessFile rafOut = new RandomAccessFile(toFile, "rw");
        FileChannel fcOut = rafOut.getChannel();

        ByteBuffer writeMap = fcOut.map(FileChannel.MapMode.READ_WRITE,0,(int) fcIn.size());

        writeMap.put(byteBuffIn);   

        long timeOut = new Date().getTime();
        System.out.println("Memory mapped copy Time for a file of size :" + (int) fcIn.size() +" is "+(timeOut-timeIn));
        fcOut.close();
        fcIn.close();
    }


    static final int CHUNK_SIZE = 100000;
    static final char[] inChars = new char[CHUNK_SIZE];

    public static void customBufferedCopy(String fromFile, String toFile) throws IOException{
        long timeIn = new Date().getTime();

        Reader in = new FileReader(fromFile);
        Writer out = new FileWriter(toFile);
        while (true) {
            synchronized (inChars) {
                int amountRead = in.read(inChars);
                if (amountRead == -1) {
                    break;
                }
                out.write(inChars, 0, amountRead);
            }
        }
        long timeOut = new Date().getTime();
        System.out.println("Custom buffered copy Time for a file of size :" + (int) new File(fromFile).length() +" is "+(timeOut-timeIn));
        in.close();
        out.close();
    }
}

When exactly is it nececary to use RandomAccessFile? Here it is used to read and write in the memoryMappedCopy, is it actually nececary just to copy a file at all? Or is it a part of memorry mapping?

In customBufferedCopy, why is synchronized used here?

I also found a different example that -should- test the performance between the 2:

public class MappedIO {
    private static int numOfInts = 4000000;
    private static int numOfUbuffInts = 200000;
    private abstract static class Tester {
        private String name;
        public Tester(String name) { this.name = name; }
        public long runTest() {
            System.out.print(name + ": ");
            try {
                long startTime = System.currentTimeMillis();
                test();
                long endTime = System.currentTimeMillis();
                return (endTime - startTime);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
        public abstract void test() throws IOException;
    }
    private static Tester[] tests = { 
        new Tester("Stream Write") {
            public void test() throws IOException {
                DataOutputStream dos = new DataOutputStream(
                        new BufferedOutputStream(
                                new FileOutputStream(new File("temp.tmp"))));
                for(int i = 0; i < numOfInts; i++)
                    dos.writeInt(i);
                dos.close();
            }
        }, 
        new Tester("Mapped Write") {
            public void test() throws IOException {
                FileChannel fc = 
                    new RandomAccessFile("temp.tmp", "rw")
                .getChannel();
                IntBuffer ib = fc.map(
                        FileChannel.MapMode.READ_WRITE, 0, fc.size())
                        .asIntBuffer();
                for(int i = 0; i < numOfInts; i++)
                    ib.put(i);
                fc.close();
            }
        }, 
        new Tester("Stream Read") {
            public void test() throws IOException {
                DataInputStream dis = new DataInputStream(
                        new BufferedInputStream(
                                new FileInputStream("temp.tmp")));
                for(int i = 0; i < numOfInts; i++)
                    dis.readInt();
                dis.close();
            }
        }, 
        new Tester("Mapped Read") {
            public void test() throws IOException {
                FileChannel fc = new FileInputStream(
                        new File("temp.tmp")).getChannel();
                IntBuffer ib = fc.map(
                        FileChannel.MapMode.READ_ONLY, 0, fc.size())
                        .asIntBuffer();
                while(ib.hasRemaining())
                    ib.get();
                fc.close();
            }
        }, 
        new Tester("Stream Read/Write") {
            public void test() throws IOException {
                RandomAccessFile raf = new RandomAccessFile(
                        new File("temp.tmp"), "rw");
                raf.writeInt(1);
                for(int i = 0; i < numOfUbuffInts; i++) {
                    raf.seek(raf.length() - 4);
                    raf.writeInt(raf.readInt());
                }
                raf.close();
            }
        }, 
        new Tester("Mapped Read/Write") {
            public void test() throws IOException {
                FileChannel fc = new RandomAccessFile(
                        new File("temp.tmp"), "rw").getChannel();
                IntBuffer ib = fc.map(
                        FileChannel.MapMode.READ_WRITE, 0, fc.size())
                        .asIntBuffer();
                ib.put(0);
                for(int i = 1; i < numOfUbuffInts; i++)
                    ib.put(ib.get(i - 1));
                fc.close();
            }
        }
    };
    public static void main(String[] args) {
        for(int i = 0; i < tests.length; i++)
            System.out.println(tests[i].runTest());
    }
}

I more or less see whats going on, my output looks like this:

Stream Write: 653
Mapped Write: 51
Stream Read: 651
Mapped Read: 40
Stream Read/Write: 14481
Mapped Read/Write: 6

What is makeing the Stream Read/Write so unbelievably long? And as a read/write test, to me it looks a bit pointless to read the same integer over and over (if I understand well what's going on in the Stream Read/Write) Wouldn't it be better to read int's from the previously written file and just read and write ints on the same place? Is there a better way to illustrate it?

I've been breaking my head about a lot of these things for a while and I just can't get the whole picture..

© Stack Overflow or respective owner

Related posts about java

Related posts about file-io