Come back for more, have you? For those viewers tuning in late, this is the second of four articles dealing with setting up and running a basic software usability testing session.
Why test the usability of your software? To keep your users from punching their computers in frustration. Or, alternately: to keep your customers happy, which is always good business.
Basically, you want to watch some typical users using your software in a typical fashion. Through observation you can determine the sources of potential frustrations, both large and small, so you can later alleviate these frustrations. This column deals with possibly the most difficult part of running your session: finding appropriate test users.
First you must decide what sort of person you need. Check with your marketing department — you want people from your target market segmentation – for example, the subset of the population that you’re hoping will buy your software. This theoretical profile likely includes: how much experience the given people have with the type of software you’re testing, what experience they have with previous or similar versions, and the depth of their general computer experience.
But where to look? Good places to advertise include newsgroups that deal specifically with your product or a related topic, and local user groups (for example, many major cities have user groups devoted to specific products and environments, such as a Java User’s Group). If you are testing an upgrade of an existing product, try getting the list of people who filled out the warranty card for the current version. You can even try calling some temp and placement services, and ask if they have anyone on file who might be interested. (Be warned, such places often want money for this information.)
Avoid the temptation to use your own employees, as they will generally be biased: they may be unwilling to admit there are problems, they may know too much about it (which gives them an advantage over a novice), and so on.
Once you’ve located the appropriate people, you have to convince them to come in — most companies simply offer cash incentives. Low-budget alternatives include: free copies of some of your existing software, an eventual copy of this software, T-shirts, mugs, vinyl record albums, or anything else you can think of. Also, always offer to feed your participants — no real computer geek will turn down free food. When you call or e-mail your potential participants you should be clear that this isn’t a sales pitch, that no people will come to their homes, and e-mail addresses will not be sold to one of those scum-sucking spamming advertising services.
A little bonus cover-your-backside advice: check with your legal department (or at least your brother-in-law who’s finished his first year of law school) about Non-Disclosure Agreements, and Agreements for the Exchange of Confidential Information. These are two fairly simple and standard forms which you want your participants to sign before the session. They state that a) the users can’t tell anyone about your software-in-progress, and b) you don’t ever have to pay your participants for any of their ideas or suggestions that you might use. Additional clauses can include warnings about mattress tags, but remember: there ain’t no Sanity Clause.
Finally, how many participants do you need? Not enough warm bodies may mean your data is suspect, but each one costs time and money. Remember the law of diminishing returns — after a while, you’re spending money and not learning anything new. IMHO, between two and four sessions is usually about right.
Once you have people to observe while they work with the software, all you need to do is set-up the room and equipment and collect the data.
That’s peanuts. We’ll cover all that in the next two articles.
Blau is a software designer working in the User-Centered Design area at the IBM Toronto Lab. He can be reached at [email protected].
