We want to be able to create log files from our Java application which is suited for later processing by tools to help investigate bugs and gather performance statistics.

Currently we use the traditional "log stuff which may or may not be flattened into text form and appended to a log file", but this works the best for small amounts of information read by a human.

After careful consideration the best bet has been to store the log events as XML snippets in text files (which is then treated like any other log file), and then download them to the machine with the appropriate tool for post processing.

I'd like to use as widely supported an XML format as possible, and right now I am in the "research-then-make-decision" phase. I'd appreciate any help both in terms of XML format and tools and I'd be happy to write glue code to get what I need.

What I've found so far:

log4j XML format: Supported by chainsaw and Vigilog. Lilith XML format: Supported by Lilith

Uninvestigated tools:

Microsoft Log Parser: Apparently supports XML. OS X log viewer:

plus there is a lot of tools on http://www.loganalysis.org/sections/parsing/generic-log-parsers/

Any suggestions?