Article
Essential SaMD Regulatory Documents: Curated List
This post was previously on the Pathfinder Software site. Pathfinder Software changed its name to Orthogonal in 2016. Read more.
A consistent nuisance problem when testing Rails applications is the “unit test gap”. This happens when the model test passes and the controller test passes, but the application as a whole fails because there’s a mismatch between the output produced by the model and the input given to the controller test. In theory, Rails integration tests can solve this problem, but they aren’t really designed for it, and nobody uses them much anyway.
An easy-to-use tool that solves that end-to-end testing would be great. Somebody should really write a blog post about that.
So, as I was saying. We continually run into an issue with our clients over defining requirements at the level that developers need to keep going, without getting bogged down in long stretches of design. There’s a big gap between “users should be able to upload photos to their pages” and all the different details of permissions, validation and the like than need to be answered at some point. The Agile process suggests that those requirements be created as closely as possible to the code, which leads to the question of how best to keep the customer in the loop while the developers need these decisions to be made.
An easy-to-use tool that lets users read or create requirements that developers can build from and run as acceptance tests would be great. Somebody should definitely write a blog post about that.
Which brings us — finally — to Cucumber, a tool for creating automated acceptance tests. It’s flexible enough to solve both problems. It can be used as a developer tool to drive regular TDD testing, and as a client tool for managing requirements.
It’s got loads of potential. And some pitfalls and tricks. The Cucumber web site has great documentation on installing and using. I want to focus on how Cucumber can work within a development process.
Here’s a Cucumber test:
Background: Given I am logged in Given I am in a project Scenario: Make High Priority Given I have a task with medium priority And I am on a page that displays tasks When I click on "priority_up" for that task Then the task should display high priority
The structure of a Cucumber test is pretty simple. You’ve got your “Given” lines, which specify prerequisites, then you’ve got your “When” lines which are user actions, finally the “Then” lines list results. The Background items are loaded before each scenario in the feature, and therefore consist primarily of “Given” lines.
This test is actually runnable as is, although Cucumber will just tell you that it doesn’t know what any of the lines actually mean yet. The Cucumber test runner output, by the way, is outstanding. Color coded results of each step, tied to the line where each step is defined, a nice summary, and snippets of code to insert for undefined steps. The output goes a long way toward making Cucumber easy to use.
In order to make this test actually work, you have to define it. Cucumber gives you a handy file to place the definitions in. Cucumber task definitions are Ruby code. Here is the definition for the first step in the scenario above:
Given /^I have a task with (.*) priority$/ do |priority| @task = Task.make(:priority => priority, :project => @project, :feature => @feature) end
Let’s break this down a little bit. The starting point of a Cucumber step definition is a definition line containing a regular expression and taking a block. Any group specified in the regular expression is passed as an argument to the block (and matching text is very helpfully presented in bold from the command line output).
In this particular case, the text “I have a task with high priority” matches this step definition, and passes “high” to the block, where it’s used to create the new task (using the Machinist fixture replacement plugin — Cucumber plays very nicely with all the fixture replacements).
Here are the definitions for the other three steps.
Given /^I am on a page that displays tasks$/ do visit("/tasks") end
This one is pretty straightforward and uses Webrat to simulate a browser hit to that specific URL.
When /^I click on "(.*)" for that task$/ do |button| #click_link(dom_id(@task, button)) if button == "priority_up" visit("/tasks/#{@task.id}/upgrade") elsif button == "priority_down" visit("/tasks/#{@task.id}/downgrade") end end
The commented line here is what I would do if it was a normal link — again using Webrat to simulate the link. However, Webrat doesn’t follow rails link_to_remote
links (at least not yet), which is what I actually have in this project. So I’m faking it with a truly ugly hack. I’m assuming there’s a better way to do this.
Finally, we get to the actual assertion.
Then /^the task should display (.*) priority$/ do |priority| response.body.should =~ /priority_task_#{@task.id}/ response.body.should =~ /#{priority}/ end
Since the response here is an Ajax response, all I can do is a match on the output. If it was ordinary HTML, I could use the full weight of assert_select
and the like.
With these definitions in place, the tests will run and fail. You can then spin off into your regular TDD process to make everything work. In practice, I actually do this one step at a time. Write the step definition, then make it work, then on to the next step. Over time, this takes the place of some controller and (especially) view testing. Also, you should find some ability to reuse step definitions.
I have several points to make about this.
One thing that kept me from trying out Cucumber in the past was a variant on the same thinking that keeps people from testing in general – the fear that you’re just going to be writing more code, and taking more time, to little or no benefit.
My experience so far, as I explore what Cucumber can do, has been largely positive. Where I was starting with Cucumber and only a vague idea of how the user interaction would play out, writing the scenarios at the Cucumber level felt very valuable and gave the development a clear path that I wouldn’t have otherwise had. That said, there is extra code being written, and it’s clearly possible to get really tangled in getting the step definitions right.
As developers, we’re kind of conditioned to believe that if we’re solving a complicated problem, we’re doing something valuable. At its worst, writing the step definitions felt like a complicated problem that wasn’t adding much. Luckily, a lot of that was due to my own unfamiliarity with Cucumber. As I got better at it, most of the step definition part came quickly. Ultimately, I think I wrote better code as a result of starting with the Cucumber definitions — it definitely kept me from writing unneeded code, since most everything flowed from a Cucumber step.
I definitely recommend giving Cucumber a try — the potential for really cleaning up what is often the messy process of determining what a feature should do is very high. I’m really looking forward to reading (and writing) more details of what successful strategies with Cucumber look like.
Related Services: Custom Software Development
Related Posts
Article
Essential SaMD Regulatory Documents: Curated List
Article
Case Study: Capturing Quality Images for ML Algorithm
Article
5 Keys to Integrating UX Design With Agile for SaMD
Article
You Had Me at Validation