Awesome Acceptance Testing
Dan North, Joe Walnes
There’s nothing wrong with having asserts in your tests that are unrelated to what you are actually trying to test but instead check the sanity of your test so that when they fail it’s easier to workout why.
As an example:
You have a test that logs into the system, goes to a search page, fills it in and checks the results. One day this test suddenly starts to fail, it turns out that the password for the user the test is login in as has expired, if after the login step you checked that the title of the page was “Welcome” proving that you had logged in successfully it would be very easy to spot that the test failed as a result of failing to log in rather than something being wrong with the search functionality.
How fast do your acceptance tests run? Joe’s rule of thumb is about 100 ATs/minute.
How do you go about setting up test data – do you do inserts directly into the database using SQL or do your tests click about in the GUI creating data for use later in the test? Why not use the code for the domain model you’ve written to set up the data – so you’ve written your own blogging system, rather than poking around in the database to create a blog entry with a comment for the test or getting the test to click around in the GUI to create the blog entry and comment why not just instantiate a new Blog domain object and Comment object and persist them to the database using your BlogDAO object. Of course to be able to do this your AT must be implemented in the same language as your domain language but I think thats quite a strong argument for doing so (or of course now if your system is implemented in Java you can write your tests in Ruby but still access you domain code via the magic of JRuby ).
How readable are your ATs? Could someone who doesn’t know, for example, Java read them? With the grouping of steps into carefully named methods you can end up with something very readable.
Implicit assumptions:
You complete a story, later you think of a new scenario, do you reopen the old story? If this situation comes up it suggests that your initial story had an implicit assumption so rename your original story to include that assumption explicit and create a new story for the other case. e.g. you write a story for withdrawing money from an ATM – WithdrawMoneyFromATM. You then think “What happens if the network connection from the ATM to the bank is flaky?”. The original story had the implicit assumption that the ATM was working correctly so rename it WithdrawMoneyFromWorkingATM making it explicit that it’s for a working ATM and add a second story WithdrawMoneyFromFlakyATM.
Given – When – Then – Behaviour Driven Development
Dan explained the ideas behind Behaviour-Driven Development and the idea of splitting a test down into a precondition – the given – an event – the when – and the expected result – the then. I quite like this idea, if done properly it makes the test very readable. Some code was presented from jBehave, a BDD tool for Java, illustrating the idea. The code (if I understood right) represented the latest thinking of those working on JBehave and was being carried out in private, what is present on the jBehave site is out of date.