MSTest is sapping my will to live

The team I'm currently working with has spent a lot of money buying MS's team suite and so naturally we wanted to use MSTest (AKA Team System Unit Testing). In theory using MSTest allows you to simply integrate tests into your build process and generate coverage reports etc. Unfortunately the grim reality is somewhat different.

Unit test tools have been around for several years. I've been using NUnit on a daily basis for around four years. I've never really paid much attention to it because it's one of those tools that just works; you write your tests, attribute them as [Test] and they either pass or fail. I keep hearing good things about MbUnit, but I haven't tried it yet because I feel no pain with NUnit and its constant companion, the excellent TestDriven.net. In simple terms, this is how NUnit/TestDriven.net works:

1. Attribute any class with TestFixture and any void-void method with Test. Use Asserts to check my expectations.
2. I have F8 mapped to run tests, so I hit F8 and TestDriven points NUnit at the test assembly in bin/debug folder (or wherever the project's output path points to).
3. NUnit loads the assembly and executes any method attributed with Test and outputs the results to the output window.

And that's it. If my tests need any attendant configuration or need to load any assemblies dynamically I just make sure they are copied to the output path.

As I said before NUnit has been around for a while and MbUnit does the same stuff but with some extra bells and whistles. Any unit testing framework should work in a similar way and should be as simple to use as possible.

Enter MSTest. Now you would have thought that with the great success of NUnit to copy, even Microsoft couldn't make too great a hash of it. But they have. Two major faults make it an unsuitable tool for running unit tests:

1. The test runner does not simply reflect over an assembly and run any method attributed with TestMethod, instead you have to create a special test project and your tests have to be present in an XML file in order for the test runner to find them. Why? Why should we need a special test project? There's no conceivable reason I can see. It means you have to have your tests in a separate assembly than the code under test. Now it's my practice to do this, but you shouldn't be forced to do it. Worse though is that it's just too easy for the XML file to get out of sync with the tests and for your tests to mysteriously not get executed. That's not to mention the source control nightmare it causes.
2. The test runner runs the test assemblies in some other place that's different each time. Now you can't rely on an assembly manifest to tell you everything that some code might need to run. I'm a big fan of IoC containers which means that much of my application's object graph gets loaded at runtime. I get frequent test failures because my tests can't find assemblies they need to load. Configuration is a similar issue, miscellaneous files don't get copied to the test run location. Now I know that you can do add attributes to every single test or add stuff into the test configuration file to get the test runner to copy certain files to its run directory, but why should I have to do this? Also It doesn't clean up after itself and I run tests every minute of every day, this means it builds up not an insignificant amount of crud on my disk.

I'm just going to mention in passing that the MSTest Asserts are pre-historic and look a bit like NUnit version-a-long-time-ago and that the default unit test project is full of unnecessary files and that the unit test template generates a page of crud that you have to delete before you can start work. That's no so important and I could live with it if the core design wasn't so FUBAR.

MSTest looks like a tool that was designed by someone who'd never really done much unit testing. They'd certainly not worked with NUnit to any degree. I really can't understand the thought process that resulted in such bad design decisions, but it looks like it's designed to be run once a week with an great deal of ceremony. In other words MS have totally misunderstood TDD.

So now I'm lobbying hard to get us to drop MSTest and use NUnit or MbUnit instead. In fact experience with the whole team system is that it's not worth spending the money, you get far better results and much less stress from using a combination of open and third party tools. But that's another post...