Storage for large gridded datasets

Posted by nullglob on Stack Overflow See other posts from Stack Overflow or by nullglob
Published on 2011-06-27T12:19:04Z Indexed on 2011/06/27 16:22 UTC
Read the original article Hit count: 246

I am looking for a good storage format for large, gridded datasets. The application is meteorology, and we would prefer a format that is common within this field (to help exchange data with others). I don't need to deal with special data structures, and there should be a Fortran API. I am currently considering HDF5, GRIB2 and NetCDF4.

How do these formats compare in terms of data compression? What are their main limitations? How steep is the learning curve? Are there any other storage formats worth investigating?

I have not found a great deal of material outlining the differences and pros/cons of these formats (there is one relevant SO thread, and a presentation comparing GRIB and NetCDF).

© Stack Overflow or respective owner

Related posts about compression

Related posts about file-format