Checklists, Refactored

Part 1 – Background

Guided exploratory testing is using checklists to support and guide the exploration. In addition, it also means using those checklists as the backbone for communicating to other stakeholders on the project/initiative.

I’ve started to advocate the term ‘testlists’ to replace ‘checklists’ when they are used this way, that is, when checklists are used to support testing and the communications on and about a test initiative/mission (credit goes to Michael Bolton for positing the difference between checking and testing).

Supporting the Acceptance Decision

The procurement process for a new solution includes an acceptance decision, well-described by Gerard Meszaros, Grigori Melnick and Jon Bach in Acceptance Test Engineering Guide – Volume 1 (http://testingguidance.codeplex.com/).

The acceptance decision is usually undertaken late on the project life cycle in a critical-path “acceptance test phase”. This is true even if agile testing, acceptance test driven development (ATDD), or behaviour-driven development (BDD) is the development style of the team delivering the solution. I can’t emphasize that point enough. Ron Jeffries put it succinctly in a tweet:

“don't care if they are ATDD'd. I'm a user of this stuff. Needs to be in the farking manual. if they had a farking manual” – Ron Jeffries

Ron was talking about adopting the software in question, the step that comes after the development and deployment. In short, what happens after ‘done’.

I’m advocating exploratory testing for finding the information needed to make the acceptance decision, as do many others. The tweak that I propose is the use of testlists to support collaboratively planning the exploration, for understanding where the explorers currently are, and for reporting to outsiders on the results, outsiders like internal/external auditors, steering committee members, board members, etc.

Checklists and Testlists

The difference between a checklist and a testlist is the following:

• A checklist lists things to check; a testlist lists things to explore in and around.
• To most people, a checklist is a means of tracking to completion. Check these items and you are done. A testlist is a list of places to start. Where you “end” depends on what you find in the micro-context of each item on the list.
• A testlist has attributes such as risk, business value ranking, persona attached to the items so that explorers can use that information to decide what to explore first, given a limited amount of time.
• A testlist might be hierarchical to support communicating to different audiences. A testlist might also evolve into a mind map depending on the usage and in that case be called a testmap.

We use testlists in the format of the one-page project manager (http://oppmi.com/) so that we can also provide testlist audiences with an indication of status and when/who will do the exploration/investigation. But really any format will do. The content of the testlist has been business processes/workflows/functions from the business user’s perspective, or if one exists, the starting point for the testlist content can be the story map that guided the solution development.

More on format, building testlists, and refactoring testlists in upcoming posts.

A.