Essential Software Testing Tools Blog

One of the biggest benefits to the tester from using database comparison tools is the ability to compare databases as part of a product upgrade test. Most, if not all, products with a database backend need to be upgraded at some point. When I say upgrade I mean that the end user will want to upgrade from an old version to the latest version of the product you are testing. Such an upgrade can involved updating the structure or data within the product’s database.

To this extent the software tester is going to be responsible for testing that product upgrade. Now you can approach this with some black box testing and completely ignore what is going on behind the user interface with the database. Or you can supplement this black box testing, get your hands dirty, and carry out some white box testing. For this you are going to need a little understanding of SQL and you may also need to know a little bit about the architecture of the database your application uses.

I can guarantee you that if you put some effort into this white box testing approach you’ll find more bugs. And with a failure during a database upgrade likely to cause some form of data corruption for the end users, finding more bugs has to be a good thing here. So as testers we’re faced with a difficult and technically demanding area of testing yet the consequences of getting this testing right are likely to prevent significant issues impacting the users (that is the potential for the corruption of their data that they store in your application).

The basic approach to running a comparison test like this consists of…

1. Start with the old version of the product running with the old database structure and data
2. Back up or copy the old database structure and data *
3. Upgrade your application
4. Back up or copy the upgraded database structure and data *
5. Install a clean version of the application
6. Back up or copy the clean database structure and data *

* – It’s usually easier to take two independent backups here. One backup with just the database structure and a second backup that covers the database structure and the database data.

Once you’ve completed this you should have the test artefacts listed below. If you’ve used a tool like mysqldump then these artefacts will essentially just be text files that can be compared with a standard text difference tool (like windiff):

1. Old database structure
2. Old database data
3. Upgraded database structure
4. Upgraded database data
5. New database structure
6. New database data

Once you’ve completed this there are a number of comparison tests you can employ now. If you’ve used a tool (like mysqldump) that creates the database backup/dump in text format then you can now compare these artefacts with a text difference tool, or you could employ specific database comparison tools (see below).

The easiest, and probably most useful, comparison test is the database structure comparison. However, all of the following will give you an important insight in to the success (or failure) of the upgrade process…

Upgraded database structure comparison
Take upgraded database structure and compare this structure against the new database structure. In 99% of cases these should be identical. If an upgraded database structure is different to the structure of a database from a cleanly installed new database then you’ve got questions to ask your development team

New database data comparison
With the data from the old database compare against the data from the upgraded database. You will expect to see differences here but you may find some that you wouldn’t expect to see. This can be a difficult comparison to make, depending on the complexity of the database, so it might be worth working with the development team whilst running this test.

New database structure comparison
This comparison should compare old database structure with the new database structure. This will show up the expected differences that have been implemented for the new database. This list of differences should be checked with the development team to ensure only expected changes are evident in the new database structure

The above tests can be executed with freely available text based comparison tools (like windiff). However, if your database structure has any degree of complexity to it, it is usually advisable to employ a dedicated SQL difference tool like SQL Delta, SQL Data Examiner, dbForge Data Compare, AdeptSQL Diff or SQL Data Compare . Products that have the capability to compare two different database (including both data and database structure) give the tester an easy way of checking changes between databases.

The other big advantage of employing SQL comparison tools is that they are very helpful when it comes to keeping test environment mirroring a live environment. That’s a discussion for another day though.

Testing prototype software applications, that are going to be demonstrated to potential customers, presents some interesting challenges to the tester. You know that the software application isn’t supposed to be a fully working application and you expect it to contain numerous defects. It’s also common with prototypes for there to be very little in the way of well defined requirements. Yet somehow we’re expected to work with the development team to produce something that is going to be good enough to convince a customer to invest in this software product. This isn’t a time to work against the software development team but it is a time for the software testing team to work closely with development as a single team to deliver a product that may keep you employed over the next 12 months.

Gathering requirements is one challenge. Deciding which areas of functionality are likely to be demonstrated is another. Working out the restrictions on the amount of test data you need is yet another. Many many aspects of testing are turned on their head when you are testing prototypes. After all you’re working towards delivering a product that, in many ways, isn’t supposed to work.

There are no easy answers on how best to approach this but I would recommend you consider the following carefully:

Talk – if requirements are limited or non-existent then you have no choice but to talk to all the stake holders. If you have contacts in the customers company then talk to them and find out about what they expect. Talk to the developers and see how they’ve interpreted the requirements. Talk to sales, or who ever is planning to stand up and run the demonstration. In this scenario it is usually the tester who pulls together the available information from all the interested parties, and as a result, ends up pulling the whole team together to deliver a great demo product.

Use your judgment – if you’ve spent any amount of time in software testing then you probably already have a good feel for what is important and what is not important in a software application. Use this judgment to to help you focus on testing only the parts of the application that matter. Extensive field validation testing doesn’t matter in a demonstration if the person running the demo knows exactly what he/she is going to enter in that field!

Stand back – use your imagination and put yourself in the position of the potential customer that this prototype will be demonstrated to. If they are senior management (no disrespect meant to senior managers here) then concentrate on testing aspects that support the look and feel of the application. If they are real tech’ies then think more about testing and demonstrating areas of the application that are likely to stimulate the most interest.

Focus on the interesting bits – I did a demo to a newly appointed board member from Sun Microsystems some years back. I won’t go into the detail but the demo revolved around a number of Solaris command prompt shells. At the end of the demo this board member’s first comment was “this would look much better with a graphical interface”. The room went silent. He completely missed the technology behind the application we demonstrated that day. It went to prove though, that in most peoples eyes, what looks good is what matters. Presentation is everything. Make sure you focus on the best looking bits.

There are no hard and fast rules for working on projects that aim to deliver a product for a demonstration. If there’s one bit of advice I’d give here though it’s learn to put yourself in the shoes of the person you are going to demonstrate this too. Use your imagination to work out what is important and use your instinct to help direct the rest of the team to focus on what matters. And don’t forget that 99% of the time people will focus on what it looks like, and nothing else.