The massive string manipulation/gathering in ExceptionReporter to create a textual report is a throw-back to it's ancient origins...

It seems to me that it should use a template-based approach - as is often used to generate HTML in javascript/frameworks and libraries like Mustache

I think using templates would produce clearer/simpler code for generating the report and remove all the StringBuilder operations.
It would also add the flexibility of creating new templates for different report formats like allowing for Text or HTML
A user could even create their own template and customize the report...

We could just roll our own simple text substitution for much of it, but some of the more complex rendering of lists might benefit from a templating library eg mustache-sharp (except this one doesn't support .NET 4)