I'm currently using JSON (compressed via gzip) in my Java project, in which I need to store a large number of objects (hundreds of millions) on disk. I have one JSON object per line, and disallow linebreaks within the JSON object. This way I can stream the data off disk line-by-line without having to read the entire file at once.

It turns out that parsing the JSON code (using http://www.json.org/java/) is a bigger overhead than either pulling the raw data off disk, or decompressing it (which I do on the fly).

Ideally what I'd like is a strongly-typed serialization format, where I can specify "this object field is a list of strings" (for example), and because the system knows what to expect, it can deserialize it quickly. I can also specify the format just by giving someone else its "type".

It would also need to be cross-platform. I use Java, but work with people using PHP, Python, and other languages.

So, to recap, it should be:

• Strongly typed
• Streamable (ie. read a file bit by bit without having to load it all into RAM at once)
• Cross platform (including Java and PHP)
• Fast
• Free (as in speech)

Any pointers?