• Requirements Engineering — BDD Example Mapping

In one of our retrospective meeting two months back, we were looking at RQ statistics of our freshly delivered stories. There was a chart showing in which phase we discovered the bugs.

Bug report by phase

The chart was impressive as we caught big chunk of bugs before delivery(after feature testing) and there were very few bugs that we received from system test team. However, there was one entry that caught our attention — Requirement bugs. There were quite a few.

What does that mean? We started the discussion. The story was not correctly described? The acceptance criteria was incomplete? There was a debate around it. One or two RQs found their place in ‘incomplete acceptance criteria’ but what about other bugs?

Requirement phase RQ categorization

In the discussion, we figured out that the acceptance criteria was ‘complete’ however the criteria was not ‘self-explanatory’. What does that mean? Let’s see an example

Complete criteria

Self-explanatory criteria

The conclusion we drew out of the discussion was — we did not extract enough information out of acceptance criteria.

We thought we would research on some techniques to improve on this aspect. There are many techniques to extract information out of acceptance criteria. We selected ‘Example mapping’ and thought to give it a try. This article is feedback of the technique based on our usage.

We took this as an experiment before adapting our process permanently. We started collecting the thorough knowledge about the concept and how it is used. Here are some notes-

• BDD — Behavior-driven development is as a flavor of TDD (Test driven development) focuses on functionality and scenarios before development.

BDD as a flavor of TDD

• One of the widely accepted technique of BDD is ‘Example mapping’. Example mapping is very simple technique to derive scenarios(and in turn test cases) from acceptance criteria.
• In this technique, we read one acceptance criteria point and draw examples around it. There are certain examples that PO brings in with the story. other team members can add more or their examples are considered as feedback that acceptance criteria is properly understood.

Example mapping board

• For every rule, there are examples provided. So if rule is related to validation, then there are minimum 2 examples — when value is valid, when value is invalid.
• This activity is quiet fast and does not take more than 30 minutes. This is done in refinement meeting.
• These examples can then transformed into test scenarios.

The technique is easy to follow and so we planned to carry out this activity on our story. The refinement meeting agenda was bit modified to accommodate mapping activity as well.

Story — As a field manager I want a new editor support so that I can enter user details who can access the field data remotely. (Note — Vague details as we are not allowed to publish details)

Acceptance criteria-

1. Two columns are displayed on editor — Username and password. Editor header — Users. Username length — not more than 16. Password — * based(12 chars).
2. Deletion, modification of data supported.
3. Rows cannot be copied and pasted on the editor

… (many other rules)

When we started the example mapping, we asked our PO not to bring in example for the first story, We wanted to learn together how it is done. So we, as a team, started fetching examples out of rules. We distributed sticky notes to participants and asked then to write down examples for the rule. We created board like-

Board before discussion

After few minutes the board was filled with examples

Board after some time

Many examples were repeated and we bunched them together and decided to consider best example out of them.

So for the first rule line — the examples we considered were

1. When user enters username ‘This is more than 16’ the cell is highlighted with error ‘Not more than 16 characters allowed’. (Error message was not mentioned before and field highlight was also missed— we saved 2 requirement bugs).
2. When user enters username ‘# is spl char’, the cell is highlighted with error ‘No special characters allowed except _’. (This example triggered discussion and we saved many bugs here)
3. When user enters ‘ValidUsername’, the value is accepted with no error messages.
4. When user enters password ‘Password123’ and commits, the field shows only 6 * to avoid people knowing the length of password. (No mention in acceptance criteria but triggered discussion)
5. Similar examples for password.

For the first time, we actually crossed recommended 25 minutes mark. It took us 45 minutes to come out of the discussion with collected examples. However, we found ourselves pretty confident about story. Even the newcomers understood the core of the story. The best part is, the development team had test scenarios as input.

Now after two months of practicing the activity on many of our stories, we have adapted our Requirement Engineering process to include the Example mapping technique.

Some results of our activities were

1. Before this activity, our requirement bugs were 7–8. The number reduced to 0–1.
2. The defect slippage from developer check-in was reduced by 90% as the scenarios are present before-hand.
3. Helped in proper estimation of story as we get to know which scenarios may take time.
4. Since we automate most of the scenarios through nUnit, we have tests ready to automate.

So the experiment result was ‘successful’ for us. From personal experience, I would recommend Example mapping technique to everyone who are trying to ‘shift-left’.