Beyond the Agile Testing Quadrants

You can build the right product and you can build it right, and still not deliver value to the customer/user. For any number of reasons, they don’t adopt it easily, completely, or on time.

You can blame them. Luddites.

You can  blame others. Microsoft.

You can blame yourself. Incompetent. Need a hairshirt.

Or you can evolve your testing to include user adoption and benefits realization. Ask questions aimed at finding user adoption issues and explore risks to benefits realization.

I remember reading a Twitter post from Joshua Kerievsky about usage data and how evolving a product without it now seemed so inadequate once he/they started using it. In my mind, this extends the agile testing quadrants to finding adoption issues in addition to helping discover the right product.

A/B testing is another example. This is using real customer behaviour to guide the evolution of the product. Imagine what a tester could do – with their skills tuned towards asking the important questions at the right time – to this field. I think it’s filled with opportunity.

In the enterprise domain (where I spend all my time) the agile testing quadrants are not enough. You can follow them bravely and assuredly and still not deliver value because of:

• an inadequate end-user communications plan
• an inadequate end-user training program, or having a gap between training and when the end-users need to start using the solution
• using the same testers in every test cycle or continuously, making them more expert and less skilled in finding adoption issues
• not translating pre-release performance testing to post-release performance monitoring
• …

Consequently, because I love the agile testing quadrants for getting the right product built the right way, I thought to extend the idea to include supporting adoption and supporting benefits realization.

Q1: Discover barriers to solution adoption by the support/sustainment/operations group(s). For example, assess the communications and training for support/sustainment personnel.

Q2: Discover barriers to solution adoption by end-users/customers. For example, assess the communications and training for end-users/customers, use A/B testing to determine/optimize features/interactions the customer cares about.

Q3: Discover risks to benefits realization that originate with the end-users/customers using the solution. For example, monitor the benefits to see if there is progress towards their realization.

Q4: Discover risks to benefits realization that originate in support/sustainment/operations. For example, monitor the solution performance on an on-going basis so that performance bottlenecks don’t become barriers to effective usage that would in turn lead to the planned benefits being realized (already a common practice).

It’s a start. I almost put the agile testing quadrants themselves in Q2 and I might yet put them back there. For now, I’m hoping this conveys the way that testers need to start thinking:

• write a test strategy that lasts forever, not merely to the end of the project and/or release; deployment is not done 
• create checklists that the solution delivery team can evolve forever, not to just get the solution shipped/in production but one they can use to continually increase the likelihood of user adoption and benefits realization; deployment is not done 
• explore barriers to solution adoption; expand the list of test targets to include end-user communications and training; deployment is not done 
• explore risks and barriers to benefits being realized; deployment is not done 
• do this continually; deployment still isn’t done

Maybe you detected the theme that deployment is not done in the above points.

You can’t afford to think like this if there are product defects that decrease the likelihood of user adoption. You’ll be too busy writing bug reports. So yeah, this is complementary to agile testing (and the agile testing quadrants concept). These new quadrants are the next step, at least in the enterprise.

Vacation Smoke

As test managers, we ought to coach others on giving and receiving feedback with grace since creating feedback cycle is so much a part of testing. In that line of thinking, Scott Berkun’s latest book Mindfire contains an essay on feedback that should be required reading for test managers and testers alike.

I have a story about giving/receiving feedback that has stuck with me for many years.

---------------------------------------

A boss from a previous life time told me this story. It was from an ancient time period when you could still smoke on airplanes.

Brian and his wife were going on a vacation to a sunny destination from their home in Winnipeg, Canada – I think it was Cuba. Festive and feeling good, Brian lit up a small cigar just after they got seated and looked out the window, on top of the world.

His wife was beside him in the three-seat aisle – he being in the window seat - and asked him to put out the cigar in her usual way. He made a special point when he was repeating this story to me that her usual way was loving and warm. Not a demand in any sense of the word. They were both feeling great about the vacation, there was no tension there. A simple request.

“It’s bothering me in this space. I don’t care when we’re at home because I can open a window or leave the room but here I can’t leave my seat. Do you mind?”

In relaying his response, Brian choked up a tiny bit. He told me that his response was swift and direct. It was not mean, but it was without hesitation, “No way. I’m on vacation. This is something that I do on vacation.”
She sat back. Disappointed, but committed to enjoying every bit of the vacation including the flights there and back. She closed her eyes, cuddling his arm with both of her hands.

A few minutes later, a woman sat in the aisle seat next to Brian’s wife, settled in, read the in-flight magazine for a bit and then leaned forward, asking Brian if he would mind putting out his cigar. Another polite, non-demanding request (these are Canadians after all). Brian put it out immediately.

This was when Brian really choked up. “There was the woman beside me, the one I married, the beautiful woman that I have planned my life with asking for a simple favour and I refuse her. Yet a stranger asks the exact same favour and I don’t hesitate to comply.” He paused for effect. “Never again.”

To his credit, Brian told the story to all his staff. There are many lessons here; the one about serving those closest to you as you would treat a stranger, and a second about teaching others what you’ve learned. The third is more obscure but clear once I tell the rest of the story.

Brian looked at his wife. She opened one eye at him, a small, sly smile curling the edges of her mouth upward. He was looking at her with an embarrassed smile on his face.

“Sorry honey. That should have been for you.”

“I know.”

The third lesson is his willingness to stop and talk and to do a micro-retrospective on his own behaviour and adapt.

Then there is her response. The graceful acknowledgement and gentle teaching she chose to give him. She could have been mad, she could have embarrassed him further, and she could have spent the rest of the flight making fun of his mistakes, perhaps even with the woman sitting beside her. She chose none of those options; instead she respectfully and gracefully showed him the way.

As test managers, we ought to coach others on giving and receiving feedback with grace. A feedback cycle is an important part of testing. Turning the feedback cycle negative is generally harmful and rarely, if ever, needed. Sure graceful feedback takes energy, but it is worth it.