I recently had a long and considered comment from Woonboo for my post "MSTest is sapping my will to live". Woonboo is a happy user of MSTest. I started writing a reply in the comments, but it was getting so involved I thought it would be nice to promote it to a new post.
Here's Woonboo's comment:
I know I'm late to the game, but mileage is everything.
I used to use NUnit, but after using MSTest for the past 2.5 years, I wouldn't go back, even with the bugs in VS2008 (which are minor and only hit if you are doing something specific with AppDomain).
Your config file thing is simple - you don't have to use attribute...in the 'settings' (right-click properties) add files to deploy.
It creates a separate test project because too many Morts out there will just put it in the same project otherwise and all your tests will be deployed with the code. I've worked with a number of these folks.
Having the built in ability to test private and internal methods/properties/fields is the biggest reason I love it. No extra code required. No need to loosen the scope (make things public) on what you're testing.
When a test fails, looking at the test results that gives me a hyper-link to every part of the stack trace where the test failed (or threw an exception) as saved me hundreds if not thousands of hours by now not having to navigate to the file, hit CNTL+G and enter the line number; especially if I have to walk up the stack to see if there was a path taken that shouldn't of been causing the problem.
Having the ability to have tests run automatically when I do a build or check-in (easier than I was ever able to with NUnit - but that's a SCC).
Code coverage gets tied to the tests you write.
Pulling up archives into the GUI of who ran what tests when on what machines. Great from the 'team lead' perspective.
You said in another post that technologies change but methodologies don't - don't let NUnit suck you into the 'one tool' mentality (although I could say the same about myself with MSTest). Love your posts generally - but I think you need to give MSTest another chance and ask someone who's been successful with it for help. It was hard for me to switch from NUnit too - but it was like when I switched from VB to C# - I never looked back once I got past the frustration point.
Woonboo's points get to the nub of the problem for me: I don't think MSTest was designed for TDD but for some other high-ceremony style of integration testing. I would submit that he is probably not working in that style. Let me take his points one by one to explain what I mean.
"you don't have to use attribute...in the 'settings' (right-click properties) add files to deploy". Yes, but I shouldn't have to do even this. Why can't it just test what's in the target directory? Building a separate run directory for every test and forcing the user to deliberately choose files to deploy is just another symptom of the high-ceremony MSTest approach.
"It creates a separate test project because too many Morts out there will just put it in the same project otherwise and all your tests will be deployed with the code." I agree that it's best to have a separate test project. But any testing framework should be unintrusive. I shouldn't have to have a separate project type. The only reason it's needed by MSTest is because MSTest requires so much configuration to be useful. And you shouldn't rely on project types to force you to correctly organise your source, otherwise we'd have some crazy Microsoft scheme to have the "domain-project", the "data-access" project, the "service-project". Don't suggest that too near to someone from the TFS team!
Now I have to have a rant here about "The Morts won't understand it" argument. I hear this over and over and over again. It usually goes along the lines of "I understand it fine, but the people I work with, or the 'maintenance' people won't". It's the lamest and most often deployed excuse for bad decisions. If the Morts can't understand something, whoever they are, get rid of them. What's the point in employing people who don't know how to do their job? But usually it's not the real reason, the real reason is lack of leadership, and lack of trust. Most advances in development practices actually make things simpler but it requires that you, if you are the team lead, sit down and work with the people you are supposed to be leading.
"Having the built in ability to test private and internal methods/properties/fields is the biggest reason I love it." One of the core reasons for doing TDD is way it drives your design. If you have to access private members in your tests I would submit that your design is not correctly decoupled. I *want* my testing framework to force me to behave like any other client when I'm writing tests.
Now there might be a situation where you have to write tests for some monolithic legacy code base, but MSTest won't help your there, you need to go and talk to Roy Osherove :)
"When a test fails, looking at the test results that gives me a hyper-link to every part of the stack trace where the test failed." You get exactly the same thing with Testdriven.NET. I have "run-tests" mapped to F8 (I've never found a use for bookmarks). "Run-tests" is context specific so if the cursor is currently inside a test method, only that test gets run. So I just hit F8 to run the test(s), the results appear on the console and I can click on any part of the stack trace to go directly to that line of code. I actually dislike fancy test runners. I don't want to have to click here and there to get to stack traces, I'd much rather everything was just thrown onto the console.
Of course running tests on your CI server is essential. I've always found it simple to do that both with Cruise Control and TFS. I don't think MSTest really adds much here and it doesn't make much sense unless you're using TFS.
"Pulling up archives into the GUI of who ran what tests when on what machines. Great from the 'team lead' perspective." OK, I don't get this. Surely what you should be looking at is code coverage. I run tests every few mintues when I'm coding, you probably wouldn't want to look at or store all those test runs. I think this goes back to my first point. A tool that's designed for high ceremony infrequent testing like MSTest would see storing a test run archive as a useful thing, anyone who's done TDD would see it as a huge waste of resources.
I gave MSTest a chance. I've also recently tried to use MBUnit. neither worked as well as TestDriven.NET + NUnit do for me. I must say that MBUnit came very close, and I wouldn't have too much problem with using if I was in a team that has already settled with it. MSTest was just a nightmare from start to finish. As I said to our project manager, if it was an open source tool, it would never have got any traction and would now be sitting unloved in sourceforge. Maybe it suits some people with a very non-agile, non-TDD methodology, but for anyone doing TDD I would stay well clear.
Woonboo. Thanks very much for provoking me to write this. I love a good debate and would really like to hear your reply. Thanks again!