Top 5 BDD tips in Gherkin
Written by Jane Orme, Software Developer
Having spent a large amount of time focusing on writing BDD tests in Gherkin, I have learned some tricks for writing effective and meaningful tests; and I will share my 5 best tips. Some of them sound obvious and apply to testing in general, but without putting effort into them it makes a test harder to maintain in future sprints when requirements might change. Here is a little teaser:
- Tags are powerful
- Meaningful titles
- One test per scenario
- User-centred wording
- Automation-friendly tests
For a reminder about what BDD is and how you can use Gherkin to do it, have a look at my recent blog articles!
1. Tags are powerful
As someone who likes statistics, tags are one of my favourite tools when it comes to Gherkin. Tags are related words for each scenario. Each tag is prefixed with an “@” symbol and is listed before the scenario title that they apply to. We use them for reporting statistics (% of tests with a specific tag which pass/fail), and for filtering (for refining test runs). These help to reflect the current quality of the system to a stakeholder. An example is as follows:
Feature: User access @ID-00001 @REQ-004 @login Scenario: The user logs in and sees the Home page Given ...
Most of our tags are to do with requirements for a medical product that we are tracking test coverage of. If requirement 004 is to do with user login, then any tests tagged @REQ-004 are known to apply to login functionality. The clients decide the priority of the requirements, and this can be used to plan developer/tester effort if tests fail. It also makes it easier to filter out scenarios to do with login during test runs.
Tip: If you have a large number of requirements it can be handy to put a summary of the relevant ones for the feature file at the top of the feature file for quick-reference.
Each scenario my team writes has a unique ID tag, such as @ID-00001, for traceability. A simple tool developed in-house manages these IDs; adding IDs to all scenarios and preventing duplicates. This enables us to track revisions of each scenario, the approval rating of the scenario, and to quickly find any test results for it – useful for software testing products needing FDA approval or similar compliance standard.
We also use tags to reflect the version that the scenario applies to, which is helpful if you want to test against a specific version of the software.
There are tools, such as Pickles, which can auto-generate reports from test results using these Gherkin tags, although we also use our own version to collate test information per requirement.
2. Meaningful titles
The title of a scenario should describe the purpose of the requirement and the outcome of the test in one concise line, making it easy to find a scenario in the test suite when needed just by scanning the titles. A good title means that someone who is short on time (or not interested in the details) does not need to read the whole test to understand the requirement. Here is an example of a good title:
Feature: User access Scenario: When the user logs out they are shown a login prompt
You do not want it so long that it scrolls off the screen (harder to review and risks containing multiple smaller tests), but not too vague that you have to guess what the scenario will cover. Here are some bad examples:
Scenario: Log out (too vague) Scenario: When a user is logged in they can log out by locating and pressing the black Logout button on the current screen, and the ‘Log In’ screen is then displayed with no user details entered in the username and password field (too lengthy and contains multiple tests)
Spending some time up-front thinking about a title before you write the test ensures that you are clear about the content of the test.
We often write a suite of new tests by initially writing only the scenario title, and then filling in the test details and steps once we know that we have good test coverage of the requirements.
3. One test per scenario
It’s very easy to write long scenarios that test multiple areas; for example, following a navigation path through the system with several user interactions. A warning sign is that these scenarios might contain several When/Then steps. Multi-test scenarios have several disadvantages:
- More brittle – changes in one area more easily break the entire test.
- More stakeholders need to be involved if several functional areas are included.
Using the log out above example, you could break the 3rd example into several tests:
Scenario: The Logout button is black Scenario: When the Logout button is pressed the ‘Log in’ page is displayed Scenario: Following a logout, the Username and Password fields are cleared on the ‘Log in’ page
Doing this, if the Logout button happens to be blue instead of black it would be easy to see that the Logout functionality still works and that the Log In fields are cleared because those other two scenarios would still pass. That makes fixing the bug a lot easier.
4. User-centred wording
One of the main benefits of BDD tests is that the tests are written in a ubiquitous language that is easily understood by anyone from managers to engineers. Keep things simple and write from the user perspective to describe the user journey.
For example, this is long-winded and too low level:
Feature: User access Scenario: The user logs in and sees the Home page Given a system And the user is not logged in And the “Log In” is displayed And the logged in flag is “Loggedin=0” When the username is entered into the “Username” field And the correct password is entered into the “Password” field And the “Login” button is pressed once Then the “Loggedin=0” flag changes to “Loggedin=1” And the “Loading” spinner is displayed And the “Home” page is displayed within 30 seconds And the “Home” page contains the following: | ‘Logout’ button | | ‘Profile’ section | | ‘News’ section | | ‘Report’ button | | ‘Home’ title | And a log entry is created for the login And the log entry contains the date and time of the log in When the “Profile” section is pressed Then the profile section is expanded And the user icon is displayed And the user name is displayed And the “Last Login” date contains the date and time of the log in
And is much better when written like this:
Scenario: The user logs in and sees the Home page Given the “Log In” page is displayed When the correct user credentials are entered And the “Login” button is pressed Then the “Home” page is displayed And the user details are displayed in the “Profile” section
Keeping steps focused on the interactions of the user, and the visible outputs, ensures that the reader will be able to understand the purpose without getting bored (hopefully!). Technical details will also be less meaningful to a higher-level reader such as a product owner, so avoid this where possible. Another test could then check the layout of the “Home” page, logging, and so on, if those details are important.
5. Keep tests automation-friendly
Last, but not least, is to write scenarios that can be automated. Automated tests are valuable – they can be linked to the build server so you could (if you wanted) run all your tests directly on the hardware every time someone pushed a branch.
In Gherkin, variables are passed to the C# automation framework within quotes in the steps. This enables a single function to be written in the step handler that is re-used whenever a scenario in the feature has the same step. For example, the following step and corresponding step handler function enable variable usernames to be entered into the username field:
When “SomeUsername” is entered into the “Username” field
We highlight any scenarios which require manual steps with a @manual tag, so these can be excluded from automated test runs.
The system we have working is very cool – with a camera positioned above the hardware running OCR on-the-fly to check the contents of the screen, and an embedded gadget to simulate screen touches. It’s quite something to see it navigating through the system by itself!