Essential Software Testing Tools Blog

A well respected friend of mine compares User Acceptance Testing (UAT) to putting up wallpaper while the plasterer is still plastering the walls.

As a specialist in implementing large CRM (customer relationship management) systems Richard has been through his fair share of user acceptance testing. He says the UAT testing phase usually gets dispensed with when the supplier finds themselves under time or budget pressure. As a result the customer effectively gets landed with the suppliers testing responsibilities, or at worse, ends up testing something that’s effectively still in development. Simply put the test team are wallpapering whilst the developers are still putting the plaster on the walls.

Adding to the pressure is the obvious fact that UAT is the final step before go live. So in many projects everything gets lined up for the live date with little scope to move dates. And guess who bears the brunt of this. The test team yet again who generally end up dispensing with life’s luxuries such as sleep.

All the books say that what is supposed to happen is that development is completed and a rigorously tested system is delivered to UAT. UAT is then just a question of putting a series of ticks in boxes in front of the customer.

What happens though is that a clam tick the box activity is turned into the most stressful phase of the whole project. Deadlines loom. Contract commitments get stretched. Testers and developers work all hours.

The onus is on the test team to act as the thermometer. Is the evidence from your early testing looking bad? The sooner you demonstrate this, the sooner people can consider the implications on the UAT phase. You’re not trying to depress people here. You just need to tell the rest of the project team how it is.

If you don’t think the software is ready for UAT you need evidence to show that it it won’t be. A track record of high priority bugs raised over the duration of the project is a good start. Explaining how embarrassing it would be for the customer to see a bug during UAT tends to hit home harder.

You could just say we’ve got 3 blocker bugs stopping us from going into UAT. Alternatively you could paint a picture. If we go into UAT, and we test feature X in front of the customer, then he’s going to see Y. And that’s going to make us look pretty stupid. Perhaps we should put back the UAT? We need time to resolve this. Then when we run through these UAT tests we’ll look like the professionals we really are. Okay, I’m stretching it a bit here but you get the idea.

Putting back UAT and delivery deadlines is never a comfortable discussion to have with the customer. In my experience you are better off having one difficult discussion before the UAT starts, rather than several very difficult discussions with the customer during UAT.

Maybe we’ll never reach the utopia of completing development and delivering a rigorously tested system to UAT. As testers, part of our remit is to let everyone know how likely we are to have a successful UAT phase. This process of telling it like it is, keeps the whole team focused on what really matters. And what really matters is the customer.

For the customer it’s about finding that balance between delivering the features, to an acceptable level of quality, within the time frame. A sort of ‘the plaster still needs a bit of drying time but we should be okay hanging the wallpaper anyway’ approach.

It would seem that the blog post about treating project managers like 3 year olds hit a few nerves. This response from Joe hit a nerve with me.

“Make it their problem, not yours. The trick here is to turn the tables so the child bears the consequence of their action. So when he writes on the wall, instead of shouting at him and cleaning it up yourself, take a deep breath then show him how to clean it himself.”

The reason it hit a nerve was because I drew on the walls of my parents house when I was 6. Normally I wouldn’t have been shouted at. They we’re in the process of trying to sell the house though. With people due to view in a few hours time, crayon drawings all over the walls wasn’t going to be a strong selling point.

In contrast I also remember finding some screw drivers and starting to take the plug sockets in the house apart. That tester inquisitiveness came out at an early age. In this instance the reaction from my mum was swift but firm. She wasn’t going to let me make that mistake. Fortunate really. Otherwise that tester inquisitive would have ended quite quickly.

It does highlight the fact that the best test managers are always the ones that allow you to draw on the walls. More than that when you know things aren’t looking so good (perhaps your drawings weren’t that good) the good test managers are so approachable that you always feel supported whilst making those mistakes. An essential part of learning.

The worst test managers are the ones that let you stick screw drivers in plug sockets.

A number of very good test managers have really impressed over the years. To the extent that I always thought, when I’m in that position I’ll make sure I do it that way too. Like a lot of things in life, when someone makes it look simple, it’s usually because they’ve spent ages learning and perfecting.

Whilst I’d like to think I’m good at letting people draw on the walls, I know I’ve still got some way to go to perfect this. Usually when the team is up against it with a looming deadline, mistakes always seem much larger than they really are. The temptation to take over and correct the mistake, in order to keep things on track, can be very strong. Resisting that desire to put it right is far from easy.

The consequences, and I know this from experience, of putting others mistakes right are usually far worse. Stepping in to take over something will, 99% of the time, result in de-motivated and de-moralised staff. When people have stepped in, and taken over something I’ve been doing, I know I’ve switch off immediately. I find myself thinking, if you want to do this better then I’ll leave you too it.

So whilst the temptation to step in and clear up is difficult to resist, the messages is simple.

Let them draw on the walls. Just don’t let them stick screw drivers in the plug sockets.

I’ve never had much time for people that play mind games. Yet I find myself playing mind games with my 3 year old son as he grows up.

I hate the thought of people playing mind games with me. So why I find it acceptable to play mind games with my son I’m not sure? Still, anything for a quiet life.

I don’t play mind games because I enjoy it. I do it because, well, I want him to do what I tell him to do. With two small children about the house, a quite life every now and again is something to be cherished. I mean it’s rare these days.

So when I think I can pull it off I’ll stop at nothing. Not even mind games.

I think its just that getting a 3 year old to do what you want them to do can leave you with an immense feeling of frustration. The same feeling of frustration you get when you try to get project managers to do what you want them to do. Like put realistic dates in the plan.

Now that’s not meant to be an insult to projects managers. I’m just trying to emphasis this point: when you tell a project manager that the testing effort is going to be 60 man days effort you mean “it’s going to be 60 man days effort”. You didn’t mean “it’s going to be 60 man days effort, but feel free to chop that in half the moment the project schedule looks like it’s going to slip, and the test team can work magic anyway, so we’ll be just fine working 20 hour days to make up for the fact that you failed to listen to us in the first place”.

Hey, if you know best, just half my figure anyway.

Whatever your justification, they know best. Just like my 3 year old.

There’s one trick I’ve learnt now. Took me forty years and the birth of two children to learn it, but then I never said I was a fast learner (more methodical).

I use this with my 3 year old and I’ve started using it with project mangers. That is, I present options rather than just putting one case forward. For some reason when you present options to people they feel they have to choose one of the options.

Bizarrely presenting a few options seems to confound people and distract them from considering anything else.

I’m not saying it’ll work every time. It’s just that for the last 3 months the “you can either go straight to bed” or “you can have a bath, we’ll play trains, then go to bed” has worked every time. I’m not saying it’ll still be working when he’s 16 but for the moment it’s working a treat.

Some years back my wife bought a horse called Murphy (stick with me on this one even if, like me, you’re not keen on horses). With a name like that you won’t be surprised to learn that Murphy came from Ireland. When my wife first saw him he’d been out in the paddock for a year with little food and even less attention. A year previous an Irish dealer had written him off as useless. He couldn’t jump so much as a pole laying on the ground without hitting it.

For reasons that I still don’t quite understand my wife ended up with Murphy. Probably something to do with not having much money and a large dose of naiveté. The first few weeks of riding this horse only confirmed the worse. Too sprightly for a relaxing ride out and too clumsy to compete. This horse didn’t stand a chance of excelling in anything.

None the less he got entered into his first competition with little hope of coming anywhere other than near the bottom. Yet, something quite special happened here. No, he didn’t win it. He didn’t even look that great. Yet he jumped out of his skin. Something about facing jumps far bigger than anyone had ever thought he was capable of, sparked something inside him. He was one of those characters that excelled only when pushed beyond what he, and everyone else, thought possible.

Personally I have little desire to ride horses but this was one horse I could relate too. You see for me, I feel most alive when I feel out of my depth being challenged by something I’m not even sure I’m capable of pulling off. I think deep down most of us feel like this.

How about you and your software testing team? Is there anyone in your software test team that would excel if you gave them a good challenge?

How about YOU? Are there tasks you would be better off delegating because you no longer find them interesting or challenging? Is there something bigger, much bigger, you could take on that would excite you?

Murphy, by the way, went on to win some pretty impressive trophies.

The implementation of screen capture and video capture tools as legitimate software testing tools is an interesting progression to follow. A few years back nobody in the software testing industry was even thinking about using screen capture tools to record defects. Yet now software testing teams are starting to see how simple it is to capture a defect with a video recording. That video recording is then used as a great communication aid to provide the required information to the development team. Yet I see this only as the beginning of a fundamental change in the way we go about testing software. Whilst it may be evolution, rather than revolution, the benefits of video screen capture in software testing can be taken far further than than just recording a defect. Why, for example, are we not using screen capture and video recording as a way of documenting test cases?

Currently we tend to think up a test case, run the test case and document the process (whether that is with Word, Excel or a test case management tool). The very fact that as software testers we are writing the test case down, demonstrates the fact that we expect to re-run the test case. We may also expect someone else in the software test team to re-run the test case at a later date (usually as part of a regression test suite). If, as  software testers, we have the tools at our disposal to capture what we doing as we are executing a test case why don’t we just record it as a video rather than waste pressious time documenting and writing about the test case?

Yes it is useful to document, in writing, the purpose and aims of a test, but why spend ages writing in minute detail about the exact steps you follow to run the test case? Why not just record a video of these steps as you run the test case and then use that video as a documented record of the test steps and expected results?

I suspect that there are two reasons for this test case recording approach not catching on just yet. Firstly the concept of using video and screen capture in the software testing environment hasn’t quite taken hold, even as a simple tool to capture defects. Secondly there is the usual resistance to ‘change’ that affects all of us.  We’ve all been writing test cases down using tools like Word for years, so why change now?

Yet the moment you experience the time and effort savings associated with capturing and recording a software test case with video, over the laborious effort involved with typing a test up, you will be quickly convinced of the benefits. Mainly the benefit that you can spend more time testing than you might do typing up documents. There are big efficiency savings to be made here with such tools. More than that, such an approach leaves the software tester to do what he or she is best at, and that is running software test cases.