Paul Moser » Behaviour-Driven Development

Don’t write tests to test your code

Paul — Tue, 22 Apr 2008 18:45:39 +0000

Wisdom tells us what we should do, experience tells us what we should never do again

The theory goes that you should write tests to check the functionality of your code, then when you refactor your code or add new features you’ve got a safety net to check that what you’ve done hasn’t broken existing functionality.

The theory is good, but in practice it doesn’t work.*

If you write tests to test your code you won’t write your tests, at least you won’t write all of them and you’ll cut corners. Writing tests to test a piece of code you already have is tedious. You’ll end up compromising your tests. You won’t test the simple stuff, because, well it’s simple, what could go wrong? It may be simple, but that’s just the sort of thing you’ll change without a moments thought but somewhere that’ll have a consequence you didn’t think of. You might skip testing the obvious stuff, but what’s obvious to you isn’t obvious to the next person to look at the code. You won’t test the difficult stuff, because that’s too time consuming and you’ve tested most of the other stuff so you figure you’re fairly safe. You won’t test the stuff that’s ‘impossible’ to test. That safety net that makes you feel secure, the holes are too big – just when you really need it you’ll fall straight through it.

So don’t write tests to test your code, write the tests before you write the code. Write tests to drive the design of your code (TDD/BDD), write tests so you know when you’re done. Only write code to fix a failing test. Yes you’ll still find your tests becoming complicated and stuff that’s difficult or ‘impossible’ to test. Listen to your tests – they are telling you something about your design. And the great thing is at the end of it, a side effect almost, you end with that safety net that really does allow you to refactor or add functionality to your code.

I think TDD/BDD and automated acceptance testing has the potential to significantly change not only how we initially produce a piece of software but also more importantly how it is enhanced and maintained throughout its ‘working life’ and it’s that which is radically going to change the value the software can deliver to the business and so the test become crucial to the success of your business. More on that later.

* when blogging you’re suppose to go for big bold controversial statements aren’t you?

Improving Java unit test readability and maintainability

Paul — Tue, 11 Dec 2007 23:17:06 +0000

The last year has seen me working with Ruby rather than Java and I have really come to like the the way tests (or rather specs) are structured in RSpec and the general way BDD approaches tests (specs).

RSpec splits the behaviour of a class up into what happens in different contexts. An example probably explains things better, taking a trivial case of a stack (I'll get to Java in a minute but even if you don't know Ruby I think this code should be understandable):

PLAIN TEXT

RUBY:

require File.dirname(__FILE__) + "/stack"
context "A populated Stack" do
setup do
@stack = Stack.new
["a","b","c"].each { |x| @stack.push x }
end
specify "should add to the top when sent #push" do
@stack.push "d"
@stack.peek.should == "d"
end
specify "should return the top item when sent #peek" do
@stack.peek.should == "c"
end
specify "should NOT remove the top item when sent #peek" do
@stack.peek.should == "c"
@stack.peek.should == "c"
end
specify "should return the top item when sent #pop" do
@stack.pop.should == "c"
end
specify "should remove the top item when sent #pop" do
@stack.pop.should == "c"
@stack.pop.should == "b"
end
end
context "An empty stack" do
setup do
@stack = Stack.new
end
specify "should be empty" do
@stack.should be_empty
end
specify "should no longer be empty after #push" do
@stack.push "anything"
@stack.should_not be_empty
end
specify "should complain when sent #peek" do
lambda { @stack.peek }.should raise_error(StackUnderflowError)
end
specify "should complain when sent #pop" do
lambda { @stack.pop }.should raise_error(StackUnderflowError)
end
end
context "An almost empty stack (with one item)" do
setup do
@stack = Stack.new
@stack.push 3
end
specify "should not be empty" do
@stack.should_not be_empty
end
specify "should remain not empty after #peek" do
@stack.peek
@stack.should_not be_empty
end
specify "should become empty after #pop" do
@stack.pop
@stack.should be_empty
end
end

So a stack should behave in a particular way:

A populated Stack
- should add to the top when sent #push
- should return the top item when sent #peek
- should NOT remove the top item when sent #peek
- should return the top item when sent #pop
- should remove the top item when sent #pop

An empty stack
- should be empty
- should no longer be empty after #push
- should complain when sent #peek
- should complain when sent #pop

An almost empty stack (with one item)
- should not be empty
- should remain not empty after #peek
- should become empty after #pop

(incidentally the above is output generated from the spec tool, a bit like generating JavaDocs from Java code)

I think splitting the behaviour up into these different contexts greatly improves the readability and maintainability of both the tests themselves and the code under test. But how to do it in Java?

It's possible to use RSpec to test Java code because you can run RSpec using JRuby, however I'm not sure if it is wise to drive the development of code in one language with tests in a different language, Ruby and Java approach numerous things very differently.

Now it would be possible to have numerous JUnit test classes for a given class, each one for a different context but this has it's drawbacks, when you make a change to your class you'd need to run numerous test cases, which you could get around by having a master testcase with a suite method that includes all the others, but then you have to maintain the list of testcases in the suite() method and it's all too easy for the list to get out of sync with the actual test cases. JUnit 4 provides a solution, you can use the Enclosed runner. The code below is an equivalent example to the RSpec one in Java.

PLAIN TEXT

JAVA:

import static org.junit.Assert.assertThat;
import static org.hamcrest.CoreMatchers.is;
import java.util.EmptyStackException;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Enclosed;
@RunWith(Enclosed.class)
public class TestStack {
public static class AnEmptyStack {
private Stack stack = null;
@Before
public void setupAnEmptyStack() {
stack = new Stack();
}
@Test
public void shouldBeEmpty() {
assertThat(stack.isEmpty(), is(false));
}
@Test(expected = EmptyStackException.class)
public void shouldComplainWhenPeeked() {
stack.peek();
}
@Test(expected = EmptyStackException.class)
public void shouldComplainWhenPoped() {
stack.pop();
}
@Test
public void shouldNotBeEmptyAfterItemIsPushedOntoIt() {
stack.push(new Object());
assertThat(stack.isEmpty(), is(false));
}
}
public static class AStackThatIsNeitherEmptyNorFull {
private Stack stack = null;
@Before
public void setupAStackWithThreeItems() {
stack = new Stack();
stack.push("one");
stack.push("two");
stack.push("three");
}
@Test
public void shouldAddToTheTopWhenSentAPush() {
stack.push("four");
assertThat(stack.peek(), is("four"));
}
@Test
public void shouldReturnTheTopItemWhenSentAPeek() {
assertThat(stack.peek(), is("three"));
}
@Test
public void shouldNotRemoveTheTopItemWhenSentAPeek() {
assertThat(stack.peek(), is("three"));
assertThat(stack.peek(), is("three"));
}
@Test
public void shouldReturnTheTopItemWhenSentAPop() {
assertThat(stack.peek(), is("three"));
}
@Test
public void shouldRemoveTheTopItemWhenSentAPop() {
assertThat(stack.pop(), is("three"));
assertThat(stack.peek(), is("two"));
}
}
public static class AStackThatContainsOnlyOneItem {
private Stack stack = null;
@Before
public void setupAStackWithOneItem() {
stack = new Stack();
stack.push("one");
}
@Test
public void shouldNotBeEmpty() {
assertThat(stack.isEmpty(), is(false));
}
@Test
public void shouldNotBeEmptyAfterAPeek() {
stack.peek();
assertThat(stack.isEmpty(), is(false));
}
@Test
public void shouldBeEmptyAfterAPop() {
stack.pop();
assertThat(stack.isEmpty(), is(true));
}
}
}

As an aside I've also used the new assertThat() method and Hamcrest matchers as you get significantly better error messages when your tests fail that way.

To get a plain test output you could probably write a XSLT stylesheet that parses the test report, the stylesheet Kerry produced for use with Junit 3.x tests maybe a good starting point, or write a custom formatter and specify that in your JUnit Ant task.

There are also the JBehave and JDave BDD frameworks for Java which would be worth looking at if you like the look of BDD.

XP Day 2006 – Are your tests really driving your development?

Paul — Fri, 29 Dec 2006 20:49:02 +0000

Are your tests really driving your development?

Nat Pryce and Steve Freeman

I won't write much about this because should this be presented again and you attend knowing too much about it, it will, I think, spoil it a bit - but if you do get a chance to see this do, Nat and Steve very effectively make their point.
So, hopefully, without giving things away...

Do your tests really communicate their intent?

don't use magic numbers in your tests just as you wouldn't in your production code
consider creating value types where you might typically use primitive types - a parameter to a method of type PhoneNumber rather than just a simple String conveys far more information, even if the PhoneNumber class is just a very simple wrapper around a String
a comment here and there can make a big difference
where a method under test is returning a numerical result based on values passed to the method, express the expected result in terms of the values passed to the method, e.g. assertEquals(2+2, billingService.caclculateCharge(2,2)) rather than assertEquals(4, billingService.caclculateCharge(2,2))

XP Day 2006 – Awesome Acceptance Testing

Paul — Fri, 29 Dec 2006 20:41:52 +0000

Awesome Acceptance Testing

Dan North, Joe Walnes

There's nothing wrong with having asserts in your tests that are unrelated to what you are actually trying to test but instead check the sanity of your test so that when they fail it's easier to workout why.

As an example:

You have a test that logs into the system, goes to a search page, fills it in and checks the results. One day this test suddenly starts to fail, it turns out that the password for the user the test is login in as has expired, if after the login step you checked that the title of the page was "Welcome" proving that you had logged in successfully it would be very easy to spot that the test failed as a result of failing to log in rather than something being wrong with the search functionality.

How fast do your acceptance tests run? Joe's rule of thumb is about 100 ATs/minute.

How do you go about setting up test data - do you do inserts directly into the database using SQL or do your tests click about in the GUI creating data for use later in the test? Why not use the code for the domain model you've written to set up the data - so you've written your own blogging system, rather than poking around in the database to create a blog entry with a comment for the test or getting the test to click around in the GUI to create the blog entry and comment why not just instantiate a new Blog domain object and Comment object and persist them to the database using your BlogDAO object. Of course to be able to do this your AT must be implemented in the same language as your domain language but I think thats quite a strong argument for doing so (or of course now if your system is implemented in Java you can write your tests in Ruby but still access you domain code via the magic of JRuby ).

How readable are your ATs? Could someone who doesn't know, for example, Java read them? With the grouping of steps into carefully named methods you can end up with something very readable.

Implicit assumptions:

You complete a story, later you think of a new scenario, do you reopen the old story? If this situation comes up it suggests that your initial story had an implicit assumption so rename your original story to include that assumption explicit and create a new story for the other case. e.g. you write a story for withdrawing money from an ATM - WithdrawMoneyFromATM. You then think "What happens if the network connection from the ATM to the bank is flaky?". The original story had the implicit assumption that the ATM was working correctly so rename it WithdrawMoneyFromWorkingATM making it explicit that it's for a working ATM and add a second story WithdrawMoneyFromFlakyATM.
Given - When - Then - Behaviour Driven Development

Dan explained the ideas behind Behaviour-Driven Development and the idea of splitting a test down into a precondition - the given - an event - the when - and the expected result - the then. I quite like this idea, if done properly it makes the test very readable. Some code was presented from jBehave, a BDD tool for Java, illustrating the idea. The code (if I understood right) represented the latest thinking of those working on JBehave and was being carried out in private, what is present on the jBehave site is out of date.