Essential Software Testing Tools Blog

A few days back I touched on the fact that I thought UAT was a game of negotiation and compromise. Negotiation and compromise will always be required when discussing UAT issues with a customer.

If you want to remain in control of the UAT process then it’s important that ‘you’ track the issues and write up the issues list. Why?

Well there’s a negotiating gambit known as “write the contract”. In short this just means that you should try to be the one to write any contract during the negotiations. The reason for this is that there are often many details that are not discussed in the verbal negotiations. The person who writes the contract can write the details in his or her favour. It’s then up to the other person to get them changed.

How does this apply to the way you track issues during UAT?

It’s not so much the way in which “you” track them but the way “you and the client” track the issues between you. Establish at the start that you are happy to manage the tracking of the issues.

It may sound like extra work to manage the list yourself, and in the short term it may well be. In the long term you get an easier ride as you write up the details of any issues in your favour.

This approach isn’t going to allow you to remove or make large changes to issues! Your client will quickly lose faith in you if you start doing that.

If, however, you can handle the peripheral issues or small detail, and write these aspects up in your favour then it will make your life easier. I’m not saying you write it up such that the client ends up not getting what they wanted. What I’m saying is that you write it up in your favour. Then during your regular review sessions you cover these points and listen for any objections. If there are strong objections you may well have to change the detail. Often though there won’t be objections and you get the outcome that is best for you.

At the other end of the scale if you let the client control the list then you’ll be the one on the back foot. They get to add issues as and when they feel like it. The customer can even go as far as committing you to a host of embellishments they thought it would be nice to add. Remember, once they’re on the list, they’re on. They don’t come off until a resolution is agreed between all parties.

If you control the list then they have to email you or call you to get it updated. This way you get to vet the issues before they go on the list.

This isn’t about fleecing the customer. It’s just about playing things to your advantage.

It won’t always be possible for you to take personal ownership of the issues list. When the opportunity arises to own it though, take it.

The effort expended managing the list will be repaid many times during the UAT phase. It will be repaid in terms of less rework, less code changes and less testing.

If the development team knew what was going on they might even thank you for it. Then again they might not!

UAT (User Acceptance Testing) is the last hurdle before delivery.

If UAT goes badly it’s fair to say that much of the good work prior to UAT is wasted. Doesn’t matter if you were winning. Didn’t matter that this girl was winning:

Ouch. Bet that hurt!

You can clear every hurdle (requirements capture, project management, development, system test, etc) but get UAT wrong and it will hurt.

Unlike hurdling, when you reach UAT you are entitled to pick the best person for the job.  Select the best tester, test lead or even test manager to help you clear that last hurdle. He or she doesn’t have to be the quickest or even the fittest. They just need to be able to clear the last hurdle.

Always put the best person forward to make sure you clear that last UAT hurdle!

What I’m saying is get the most suitable person in your team to act as the UAT lead and the interface to the customer. This may not be the same test lead that’s been running the testing so far. It may not be the best tester on the team. It may not even be the test manager.

From your side you need to pick someone to lead the UAT who is good at building relationships, good at negotiating and good at reaching compromise.

Yes, sure you want someone who’s good technically, understands the product and understands the UAT test process.  Just as important though is the ability to:

• gently talk the customer round from unnecessary changes.
• admit you need to change a feature that will never work in the real world.
• sense which requests the customer is serious about and which they really could live without.

During UAT it’s all too easy for the customer to start saying “when we said we wanted X we didn’t really mean X”. “We meant X but with bells and whistles”. “And we can’t go live if it doesn’t whistle”.

I agree we’re trying to meet, and exceed, the customer’s expectations. However, in reality UAT is all about reaching compromise. If you’re going to stand any chance of delivering to a reasonable time scale you need compromise.

That is, compromise unless your business is in the habit of losing money. If you want to give the customer free reign to change and modify every part of what you’ve delivered, continually tweaking, then fine. Just don’t lose sight of the fact that delivering the product without making a profit is going to impact on you just as much as the company.

If things get real bad, as a last resort get your sales team involved.

It is, after all, a team effort here. Sales can be much better at negotiating with the customer. They are far less bashful when it comes to saying ‘that will cost you’! The ‘that will cost you’ approach very quickly stops customers asking for too much.

Sometimes it’s the customer that needs to compromise. Sometimes it’s us that needs to compromise. Just make sure you’ve got the right person in place to make those calls.

Whoever you choose, whoever jumps that last hurdle, you need to be certain that they are capable. Capable of building a good relationship as well as being prepared to face up to the customer.

If you fail you can be certain that the customer will be raising the height of that last hurdle whilst you’re not paying attention. And, that will hurt!

A well respected friend of mine compares User Acceptance Testing (UAT) to putting up wallpaper while the plasterer is still plastering the walls.

As a specialist in implementing large CRM (customer relationship management) systems Richard has been through his fair share of user acceptance testing. He says the UAT testing phase usually gets dispensed with when the supplier finds themselves under time or budget pressure. As a result the customer effectively gets landed with the suppliers testing responsibilities, or at worse, ends up testing something that’s effectively still in development. Simply put the test team are wallpapering whilst the developers are still putting the plaster on the walls.

Adding to the pressure is the obvious fact that UAT is the final step before go live. So in many projects everything gets lined up for the live date with little scope to move dates. And guess who bears the brunt of this. The test team yet again who generally end up dispensing with life’s luxuries such as sleep.

All the books say that what is supposed to happen is that development is completed and a rigorously tested system is delivered to UAT. UAT is then just a question of putting a series of ticks in boxes in front of the customer.

What happens though is that a clam tick the box activity is turned into the most stressful phase of the whole project. Deadlines loom. Contract commitments get stretched. Testers and developers work all hours.

The onus is on the test team to act as the thermometer. Is the evidence from your early testing looking bad? The sooner you demonstrate this, the sooner people can consider the implications on the UAT phase. You’re not trying to depress people here. You just need to tell the rest of the project team how it is.

If you don’t think the software is ready for UAT you need evidence to show that it it won’t be. A track record of high priority bugs raised over the duration of the project is a good start. Explaining how embarrassing it would be for the customer to see a bug during UAT tends to hit home harder.

You could just say we’ve got 3 blocker bugs stopping us from going into UAT. Alternatively you could paint a picture. If we go into UAT, and we test feature X in front of the customer, then he’s going to see Y. And that’s going to make us look pretty stupid. Perhaps we should put back the UAT? We need time to resolve this. Then when we run through these UAT tests we’ll look like the professionals we really are. Okay, I’m stretching it a bit here but you get the idea.

Putting back UAT and delivery deadlines is never a comfortable discussion to have with the customer. In my experience you are better off having one difficult discussion before the UAT starts, rather than several very difficult discussions with the customer during UAT.

Maybe we’ll never reach the utopia of completing development and delivering a rigorously tested system to UAT. As testers, part of our remit is to let everyone know how likely we are to have a successful UAT phase. This process of telling it like it is, keeps the whole team focused on what really matters. And what really matters is the customer.

For the customer it’s about finding that balance between delivering the features, to an acceptable level of quality, within the time frame. A sort of ‘the plaster still needs a bit of drying time but we should be okay hanging the wallpaper anyway’ approach.

It would seem that the blog post about treating project managers like 3 year olds hit a few nerves. This response from Joe hit a nerve with me.

“Make it their problem, not yours. The trick here is to turn the tables so the child bears the consequence of their action. So when he writes on the wall, instead of shouting at him and cleaning it up yourself, take a deep breath then show him how to clean it himself.”

The reason it hit a nerve was because I drew on the walls of my parents house when I was 6. Normally I wouldn’t have been shouted at. They we’re in the process of trying to sell the house though. With people due to view in a few hours time, crayon drawings all over the walls wasn’t going to be a strong selling point.

In contrast I also remember finding some screw drivers and starting to take the plug sockets in the house apart. That tester inquisitiveness came out at an early age. In this instance the reaction from my mum was swift but firm. She wasn’t going to let me make that mistake. Fortunate really. Otherwise that tester inquisitive would have ended quite quickly.

It does highlight the fact that the best test managers are always the ones that allow you to draw on the walls. More than that when you know things aren’t looking so good (perhaps your drawings weren’t that good) the good test managers are so approachable that you always feel supported whilst making those mistakes. An essential part of learning.

The worst test managers are the ones that let you stick screw drivers in plug sockets.

A number of very good test managers have really impressed over the years. To the extent that I always thought, when I’m in that position I’ll make sure I do it that way too. Like a lot of things in life, when someone makes it look simple, it’s usually because they’ve spent ages learning and perfecting.

Whilst I’d like to think I’m good at letting people draw on the walls, I know I’ve still got some way to go to perfect this. Usually when the team is up against it with a looming deadline, mistakes always seem much larger than they really are. The temptation to take over and correct the mistake, in order to keep things on track, can be very strong. Resisting that desire to put it right is far from easy.

The consequences, and I know this from experience, of putting others mistakes right are usually far worse. Stepping in to take over something will, 99% of the time, result in de-motivated and de-moralised staff. When people have stepped in, and taken over something I’ve been doing, I know I’ve switch off immediately. I find myself thinking, if you want to do this better then I’ll leave you too it.

So whilst the temptation to step in and clear up is difficult to resist, the messages is simple.

Let them draw on the walls. Just don’t let them stick screw drivers in the plug sockets.

I’ve never had much time for people that play mind games. Yet I find myself playing mind games with my 3 year old son as he grows up.

I hate the thought of people playing mind games with me. So why I find it acceptable to play mind games with my son I’m not sure? Still, anything for a quiet life.

I don’t play mind games because I enjoy it. I do it because, well, I want him to do what I tell him to do. With two small children about the house, a quite life every now and again is something to be cherished. I mean it’s rare these days.

So when I think I can pull it off I’ll stop at nothing. Not even mind games.

I think its just that getting a 3 year old to do what you want them to do can leave you with an immense feeling of frustration. The same feeling of frustration you get when you try to get project managers to do what you want them to do. Like put realistic dates in the plan.

Now that’s not meant to be an insult to projects managers. I’m just trying to emphasis this point: when you tell a project manager that the testing effort is going to be 60 man days effort you mean “it’s going to be 60 man days effort”. You didn’t mean “it’s going to be 60 man days effort, but feel free to chop that in half the moment the project schedule looks like it’s going to slip, and the test team can work magic anyway, so we’ll be just fine working 20 hour days to make up for the fact that you failed to listen to us in the first place”.

Hey, if you know best, just half my figure anyway.

Whatever your justification, they know best. Just like my 3 year old.

There’s one trick I’ve learnt now. Took me forty years and the birth of two children to learn it, but then I never said I was a fast learner (more methodical).

I use this with my 3 year old and I’ve started using it with project mangers. That is, I present options rather than just putting one case forward. For some reason when you present options to people they feel they have to choose one of the options.

Bizarrely presenting a few options seems to confound people and distract them from considering anything else.

I’m not saying it’ll work every time. It’s just that for the last 3 months the “you can either go straight to bed” or “you can have a bath, we’ll play trains, then go to bed” has worked every time. I’m not saying it’ll still be working when he’s 16 but for the moment it’s working a treat.