Obviously, the easiest way to solve a bug is to be able to reproduce it in-house. However, sometimes that is not practical. For starters, users are often not very good at providing you with useful information.

Customer Service: "what seems to be the issue?"
User: "It crashed!"

To further compound that, sometimes the bug only occurs under certain environmentally conditions that can not be adequately replicated in-house. With that in mind, it is important to build some sort of diagnostic framework into your product.

What types of built-in diagnostic tools have you used or seen used?

Logging seems to be the predominate method, which makes sense. We have a fairly sophisticated logging frame work in place with different levels of verbosity and the ability to filter on specific modules (actually we can filter down to the granularity of a single file). Error logs are placed strategically to manufacture a pretty good representation of a stack trace when an error occurs. We don't have the luxury of 10 million terabytes of disk space since I work on embedded platforms, so we have two ways of getting them off the system: a serial port and a syslog server.

However, an issue we run into sometimes is actually getting the user to turn the logs on. Our current framework often requires some user interaction.