Your browser is no longer supported. Please upgrade your browser to improve your experience.

Post-Its on a whiteboard.

Written by Jane Orme, Software Developer

Having spent a large amount of time focusing on writing Behaviour-Driven Development (BDD) tests in Gherkin, I have learned some tricks for writing effective and meaningful tests, and I will share my five best tips. Some of them sound obvious and apply to testing in general, but without putting effort into them it makes a test harder to maintain in future sprints when requirements might change. Here is a little teaser:

  • Tags are powerful
  • Meaningful titles
  • One test per scenario
  • User-centred wording
  • Automation-friendly tests

For a reminder about what BDD is and how you can use Gherkin to do it, have a look at my recent blog post on BDD.

1. Tags are powerful

As someone who likes statistics, tags are one of my favourite tools when it comes to Gherkin. Tags are related words for each scenario. Each tag is prefixed with an “@” symbol and is listed before the scenario title that they apply to. We use them for reporting statistics (% of tests with a specific tag which pass/fail), and for filtering (for refining test runs). These help to reflect the current quality of the system to a stakeholder. An example is as follows:

Feature: User access
@ID-00001 @REQ-004 @login
Scenario: The user logs in and sees the Home page
Given …

Most of our tags are to do with requirements for a medical product that we are tracking test coverage of. If requirement 004 is to do with user login, then any tests tagged @REQ-004 are known to apply to login functionality. The clients decide the priority of the requirements, and this can be used to plan developer/tester effort if tests fail. It also makes it easier to filter out scenarios to do with login during test runs.

Tip: If you have a large number of requirements it can be handy to put a summary of the relevant ones for the feature file at the top of the feature file for quick-reference.

Each scenario my team writes has a unique ID tag, such as @ID-00001, for traceability. A simple tool developed in-house manages these IDs; adding IDs to all scenarios and preventing duplicates.

This enables us to track revisions of each scenario, the approval rating of the scenario, and to quickly find any test results for it—useful for software testing products needing FDA approval or similar compliance standard.

We also use tags to reflect the version that the scenario applies to, which is helpful if you want to test against a specific version of the software.

There are tools, such as Pickles, which can auto-generate reports from test results using these Gherkin tags, although we also use our own version to collate test information per requirement.

2. Meaningful titles

The title of a scenario should describe the purpose of the requirement and the outcome of the test in one concise line, making it easy to find a scenario in the test suite when needed just by scanning the titles. A good title means that someone who is short on time (or not interested in the details) does not need to read the whole test to understand the requirement. Here is an example of a good title:

Feature: User access
Scenario: When the user logs out they are shown a login prompt

You do not want it so long that it scrolls off the screen (harder to review and risks containing multiple smaller tests), but not too vague that you have to guess what the scenario will cover. Here are some bad examples:

Scenario: Log out (too vague)
Scenario: When a user is logged in they can log out by locating and pressing the black Logout button on the current screen, and the ‘Log In’ screen is then displayed with no user details entered in the username and password field (too lengthy and contains multiple tests)

Spending some time up-front thinking about a title before you write the test ensures that you are clear about the content of the test.

We often write a suite of new tests by initially writing only the scenario title, and then filling in the test details and steps once we know that we have good test coverage of the requirements.

3. One test per scenario

It’s very easy to write long scenarios that test multiple areas; for example, following a navigation path through the system with several user interactions. A warning sign is that these scenarios might contain several When/Then steps. Multi-test scenarios have several disadvantages:

  • More brittlechanges in one area more easily break the entire test.
  • More stakeholders need to be involved if several functional areas are included.

Using the log out above example, you could break the 3rd example into several tests:

Scenario: The Logout button is black
Scenario: When the Logout button is pressed the ‘Log in’ page is displayed
Scenario: Following a logout, the Username and Password fields are cleared on the ‘Log in’ page

Doing this, if the Logout button happens to be blue instead of black it would be easy to see that the Logout functionality still works and that the Log In fields are cleared because those other two scenarios would still pass. That makes fixing the bug a lot easier.

4. User-centred wording

One of the main benefits of BDD tests is that the tests are written in a ubiquitous language that is easily understood by anyone from managers to engineers. Keep things simple and write from the user perspective to describe the user journey.

For example, this is long-winded and too low level:

Feature: User access
Scenario: The user logs in and sees the Home page
Given a system
And the user is not logged in
And the “Log In” is displayed
And the logged in flag is “Loggedin=0”
When the username is entered into the “Username” field
And the correct password is entered into the “Password” field
And the “Login” button is pressed once
Then the “Loggedin=0” flag changes to “Loggedin=1”
And the “Loading” spinner is displayed
And the “Home” page is displayed within 30 seconds
And the “Home” page contains the following:
| ‘Logout’ button            |
| ‘Profile’ section          |
| ‘News’ section             |
| ‘Report’ button            |
| ‘Home’ title               |
And a log entry is created for the login
And the log entry contains the date and time of the log in
When the “Profile” section is pressed
Then the profile section is expanded
And the user icon is displayed
And the user name is displayed
And the “Last Login” date contains the date and time of the log in

And is much better when written like this:

Scenario: The user logs in and sees the Home page
Given the “Log In” page is displayed
When the correct user credentials are entered
And the “Login” button is pressed
Then the “Home” page is displayed
And the user details are displayed in the “Profile” section

Keeping steps focused on the interactions of the user, and the visible outputs, ensures that the reader will be able to understand the purpose without getting bored (hopefully). Technical details will also be less meaningful to a higher-level reader such as a product owner, so avoid this where possible. Another test could then check the layout of the “Home” page, logging, and so on, if those details are important.

5. Keep tests automation-friendly

Last, but not least, is to write scenarios that can be automated. Automated tests are valuablethey can be linked to the build server so you could (if you wanted) run all your tests directly on the hardware every time someone pushed a branch.

In Gherkin, variables are passed to the C# automation framework within quotes in the steps. This enables a single function to be written in the step handler that is re-used whenever a scenario in the feature has the same step. For example, the following step and corresponding step handler function enable variable usernames to be entered into the username field:

When “SomeUsername” is entered into the “Username” field

We highlight any scenarios which require manual steps with a @manual tag, so these can be excluded from automated test runs.

The system we have working is very coolwith a camera positioned above the hardware running OCR on-the-fly to check the contents of the screen, and an embedded gadget to simulate screen touches. It’s quite something to see it navigating through the system by itself.

Looking for even more tips?

We realise that five tips are only just the beginning of the conversation. If you are interested in learning more about using Gerkin and BDD within your software projects, please get in touch. In addition to providing software development teams, we also provide training and consulting to in-house development teams.

Why is psychological safety critical to managing software risk?

Are you wondering what it really takes to prevent hazards and harms from embedded devices and systems? The tool many organisations are missing from their software development and product teams’ toolboxes:

Psychological safety.

You can download your copy of Why is psychological safety critical to managing software risk today.  It doesn’t matter if you’re new to the concept or need a refresher; in our new ebook, you’ll learn about:

  • What is psychological safety?
  • The four stages vital to building psychological safety within teams.
  • How psychological safety diminishes project risk.
  • Project management practices that work well in organisations with psychological safety.
  • How two businesses squandered psychological safety and paid the price for it.

 

Download your ebook

Further reading

Did you know that we have a monthly newsletter?

If you’d like insights into software development, Lean-Agile practices, advances in technology and more to your inbox once a month—sign up today!

Find out more