This post was previously on the Pathfinder Software site. Pathfinder Software changed its name to Orthogonal in 2016. Read more.
It’s been about ten weeks since I wrote about Cucumber the first time. Since then, I’ve continued to use Cucumber and now seemed like a good time to update some thoughts on how and why it seems to be working for us.
The big headline, of course, is that I’m still using it after ten weeks. I’m pretty quick to abandon tools that aren’t pulling their weight, so just the fact that Cucumber is still in the toolbox means that despite the time that it takes to write Cucumber tests and step definitions, I’m finding the process of writing the tests and the tests themselves to be valuable.
Right now, I’ve been using Cucumber primarily on two different projects.
Cucumber plays out a little differently in each case.
I struggled a bit with how to integrate Cucumber into the testing process, and eventually came to the following rule of thumb:
Just as all lines of code should be in response to a failing unit or functional test, all new unit or functional tests should be in response to a failing line of Cucumber.
The process seems to go something like this:
The big question is to what extent the Cucumber tests duplicate or replace functional tests. At the moment, my answer is somewhat, but not completely. I see the Cucumber tests as being the way to go for testing a work flow of more than one one action, but I still find it a little easier to set up data in the functional test framework. That said, I don’t think that adding the functional tests has cost me that much more time, and there are still enough gaps in Cucumber/Webrat (most notably Ajax) that I’m reluctant to give up controller tests entirely. I expect my thinking on this will change as we have projects that had Cucumber from the very beginning.
I’m having a stylistic issue with the Cucumber tests between putting the specifics in the Cucumber or in the step definition. This came up most sharply in the legacy project. We were working on a feature that made a distinction between new users and old users. My first pass at the Cucumber test started off like this:
Given a user named "Noel Rappin" who has been a member for 5 months
With a step definition that was something like this:
Given /^A user named "(.*)" who has been a member for (.*) months$/ do |name, mo| @user = Factory.create(:user, :name => name, :created_at => mo.months.ago) end
The idea was that “5 months” was just a stand in for “an old user”, but the client, quite understandably, wanted to know what the deal was with five months — the specificity of the rule seemed to imply that it was important functionally. Eventually, I changed the rule to be more like this:
Given /^A user who is not a new member$/ do @user = Factory.create(:user, :created_at => 5.months.ago) end
Once I saw this issue, I started seeing it everywhere. The second form has the advantage of being more general and more requirements like, and the disadvantage of hiding a lot of magic detail in the step definition. The original version has the advantage of putting specific data front and center and the disadvantage of not making it clear which details are important.
I’m not aware that there’s a consensus on the style issue. My tendency, right now, is to favor the general form on the theory that it’s a good thing to make the Cucumber test as readable as possible and let the step definitions sort it out. On the other hand, this seems like a good way to get really messed up by having the step definition not quite match what you expect, and have that be hidden.