More on Tests and Documentation

I can't think of a better way to learn how a complex application scenario truly works than to look at the test for that scenario (and the tests for related scenarios). What would you rather have:

  1. A wordy document that is difficult to understand even if written by a solid writer?
  2. A test that creates the complex scenario and tests the expected behavior?

I'm currently writing some rather complex code that is at the heart of the app I'm working on, and although I'm trying my best to come up with test method names to explain the exact scenario each method tests, there are so many permutations to the scenarios that each scenario is difficult to communicate via a method name. However, even if I'm not perfectly clear with my method name, another developer can look at the test-code and probably figure out exactly what behavior is being tested--and if that fails then step through the code. Voila! Instant, clear, correct understanding worth its weight in gold.

Now imagine coming aboard this project and you're quickly assigned the task of fixing an issue related to a particular behavior of the system. If we had no tests you'd spend a considerable amount of time coming to an understanding of how things work. I mean this not only in a sense of understanding the code, but understanding the higher level aspects of whatever it is in the system that you're trying to understand. You'd probably come to me and ask me where the best documentation is for the behavior, and which documents give the technical details of the behavior (which classes implement which behavior). I'm sure you'd hit me up for some of my own thoughts, too (as you should).

Since we have tests, though, your entire process would be different and more effective. You'd explain to me the issue you've been assigned, and I could give you a quick overview of how I think the behavior should work, and then point you to the general area in the code that is responsible for the behavior. Then I can say, "Take a look at that code, and anything you see is under test, so then look at the tests." Imagine the application deals with gambling; your flow might be something like this:

"Hmmm...the bug is that the blackjack player should be hitting on 20 but isn't. Steve says there is an IBlackjackPlayer interface that is more than likely a good place to start looking. Okay, let me find all the usages of this interface. Wow! There are a lot. Hmmm....but I see that IBlackjackPlayer implements IPlayer, and there is a PlayerFactory class that returns an IPlayer instance. Let's see here...my first thoughts are that either one of the IBlackjackPlayer implementations has an issue with it, or that the PlayerFactory isn't returning the correct implementation of IBlackjackPlayer. So, let me go look at the tests to see the expected behavior of each IBlackjackPlayer implementation and to see how the PlayerFactory works. That ought to give me a real picture of what is going on with these 'players' (pun intended) in the system."

A short time later your thoughts might be like this:

"Interesting. The RecklessBlackjackPlayer implementation of IBlackjackPlayer has a test called 'ChoosesHitOnTwentyAndUnder.' Hmmm...but that seems correct. I can't find another implementation of IBlackjackPlayer that hits on more than nineteen, though, so I think I might be close. Let me try debugging through this test to see what's going on."

And a very short time later your thoughts are:

"Ahhh....this test is 'ChoosesHitOnTwentyAndUnder' but it only tests to nineteen, and the RecklessBlackjackPlayer stops hitting at nineteen. Let me change the card total to twenty....yep, the test is red now...so I'll change the RecklessBlackjackPlayer to hit on twenty and under....cool, the test is green now. Now I'll run all the tests again to make sure I didn't break anything else. Sweet! They all pass! Wow, it's pretty cool how these guys have the tests independent of the database! The tests fly! Anyway, last step is following the steps to reproduce the issue through the UI....beautiful, it's fixed. Since I'm new I'll circle back with Steve to see if what I've done makes sense."

I concur and inside I think, "Great. This guy knows his stuff." I also note in my mind how the developers I've worked with appear so much more on top of things ever since we switched our process to disciplined TDD. Hmmm...might there be a connection there?

Think of this flow in relation to the non-test flow. How long would it have taken you to capture all this information about the high-level and low-level aspects of the system? Which document contains the details of this behavior? The functional requirements? If so, which one? If someone tells me which document, where is it? Is it in source control or on a network share somewhere? Do I have access to the share if that's where it is? Did the writer of the document remember to include this, and if they did include it did they put the document back in the right place? Did this behavior arise later, making it likely it was never documented? Is there a document that details the classes responsible for this behavior?

After you finally think you have the knowledge you need, how safe would you feel in making the change?

You may argue that you would have taken the same steps as above in terms of starting off: you would have come to see me, and I would have said it's likely the problem is with an IBlackjackPlayer implementation. However, how would you have found the true problem? There wouldn't have been any tests, so how would you have known that the RecklessBlackjackPlayer is supposed to hit on twenty or under? Where would you have gathered that information? Since it's clearly right there in the test, the test is not only documenting the behavior but allowing you to easily debug through the exact behavior that arose in the issue. Testing the UI after you fix the behavior becomes exactly what it is: [quickly] testing the UI and not reproducing through the UI over and over to debug through why there is such an issue, fixing the issue, and repeating again and again. When you finally got the documentation you were looking for on a non-TDD project, how much did it really help you fix the problem?

Yet another reason for believing the tests and the code serve as the documentation.