March 2, 2011, 8:17 a.m.
posted by mage
Techniques for Knowing Your Customers
We have described what it means to understand the elements of every design: people, tasks, technology, and social issues. Here we describe specific techniques that you can use to gain this understanding. Techniques such as task and customer analysis, observations, interviews, surveys, focus groups, and Web site evaluations help you characterize target customers and their needs. Some of these techniques are good for qualitative information, others for quantitative. The key is to use a mixture of techniques to get a more complete picture of who your customers are and what they need.
The word need here is important. One of the major problems with traditional software engineering methodologies is that they have focused on what clients say they want. The difference between what clients ask for and what customers need has led to many project failures in the past. Customers themselves cannot easily express what they need. The methods we present here focus on finding out what these needs are.
One of the problems you will repeatedly face is finding your target customers, and getting them to help out. Are they too busy? Perhaps you can buy their time by offering T-shirts, coffee mugs, or gift certificates. Are they still too busy? See if there is an alternative but similar audience. For example, medical doctors are often too occupied to take surveys or to participate in Web site evaluations, but first-year medical students might help out instead. Although students may not be the exact target customers, they are a pretty good approximation.
What if you have no idea who your potential customers are? This is where traditional market research techniques come to bear. Running focus groups and surveys, by telephone or online, with different types of potential customers, can help your team focus on the kinds of people who will be attracted to your Web site. This type of research should be conducted before you start designing the site.
Run a pilot test before showing your site to potential customers. Have some friends first try out your survey, focus group, or Web site evaluation to work out any kinks in the wording or procedure. Analyze the pilot test data to make sure that the data you're collecting is the data you want. This will help minimize the problems you will encounter when you collect and analyze information for real.
Start a Task Analysis
One of the first steps, before doing any kind of design work or implementation, is a task and customer analysis. A task analysis will help you understand what your customers do now and how they do it, and it will provide ideas for what your customers could do with your Web site. The key to task analysis is to first identify the target customer population, find people representative of that population, and then find out what they do.
When starting a task analysis, use your intuition and experience, as well as informal interviews with task experts, to answer questions that characterize the target audience. Later you can use other techniques, such as observations, surveys, and evaluation of competitors' Web sites to answer the questions in more detail. If you are revising an existing Web site, you can also evaluate it. Successful design teams often use a combination of these techniques to develop a meaningful understanding of customers and their needs.
The sections that follow describe some sample task analysis questions. As you might have expected, they are organized into four categories: people, tasks, technology, and social issues.
Who are the customers? What are their interests? What are their ages? Are they children, young adults, adults, senior citizens, or a combination? What level of education do they have? What kind of vocabulary do they use? What kind of computer skills do they have? Are they expert computer users? Are they novices? What is their income range? What is their reading ability? Do they have any physical constraints, such as poor vision or poor hearing? What is important in their lives?
What are your customers' current offline tasks? What tasks do they do on other Web sites? What do they come to your current Web site to do? What specific tasks do they want to do there? How are the tasks learned? Are the tasks things they will do many times, or just a few times? What tools and information do they need to accomplish their tasks? How often do they do their tasks? Are there time constraints? Do the tasks need to be done within a certain period of time? What happens if they do not complete a task? What do they do for help if they cannot complete a task? In what ways can they recover?
What kind of equipment and tools do your customers have? What kind of Web browsers do they use? What kind of plug-ins do they have? What other kinds of software do they have and use? What monitor sizes do they have? How fast are their network connections?
What kind of social or organizational factors affect your customers? Where will they do their tasks? In what environment do they do their tasks? Is it noisy or quiet? Is it a stressful environment? Is it an office environment? Do they work at home? Do they use a public kiosk or a shared computer? Is security an issue? Do they work late at night? Do they work during peak Internet traffic hours? What is the relationship between the customer and the data? Is it public data? Is it highly sensitive private data? Is the data shared with coworkers? Is it shared with family members?
Experts might ask how you can answer these questions without first doing an in-depth field study, in which you watch customers in action. We think these techniques work best used in tandem. Your task analysis can inform your field observations and interviews, and your interviews and observations can inform your task analysis. You will always have some assumptions going into a field study, and it is advantageous to make those explicit, as the task analysis lets you do. On the other hand, it is unwise to invest too much time in a task analysis before studying real customers. You might become too committed to your initial analysis and have a hard time letting evidence from the field overturn your assumptions.
Quick-and-dirty task analysis before interviewing helps you focus the field investigation. Usually time and resources do not realistically permit an unstructured "let's go in and see what we see" study, so you have to figure out which customers do the tasks you want to focus on.
After your initial task analysis, create scenarios illustrating what people would use your Web site for. Scenarios are stories rich in context that focus more on what people will do than on how. (If your background is software engineering, you may be more familiar with the term use cases.) Here's an example of a scenario for a hypothetical Web site called ebirthdayz.com that specializes in helping customers shop for gifts:
A scenario tells us something about customers and their characteristics, the tasks they want to accomplish, and the context of their use of the site (in this case Victoria's sister's sixteenth birthday, which is important to Victoria).
It is useful to create many different scenarios for each of the several types of customers that you expect to come to your Web site. These detailed illustrative customers are often referred to as personas. Get lots of detail about your personas: name, background, what they do, where they live, and so on. Make these details as real as possible. You can put some of these details right in the scenarios, or you might put some only in a document where you describe your personas. Having real people in mind is even better because you can get more details later, when you need them.
Refer to the details when deciding between different ways of carrying out a design. You might ask, "Would Victoria use this feature for sending business gifts to colleagues? No, she is a college student and probably does not have a need for business gifts. This feature would be irrelevant to this type of customer." Reuse scenarios throughout the design process as a check, to see whether your design decisions still make sense in relation to the scenarios.
Sometimes scenarios include photographs or sketched storyboards. A storyboard is a sequence of Web pages you create to give a rough idea of how a person would accomplish a given task. The storyboard in Figure shows some rough cuts of how people would select different musical genres on a PDA-based music Web site. Although you might be tempted to use software tools to make nice-looking storyboards, there are many good reasons to defer doing it at this stage (see Section 4.4, Rapid Prototyping, in Chapter 4—Involving Customers with Iterative Design, for more information).
3. This sketched storyboard shows how a customer would accomplish one task using the design of a music site targeted at PDAusers.
Again, note that scenarios do not say much about how things are accomplished. The first of the two preceding scenarios does not say where the gift recommendation feature is located, nor does it give specifics of how the gift recommendations are organized. At this early stage it is more important to determine whether the gift recommendation feature is a good idea at all before getting too detailed. Have the design team "walk through" a scenario to see if it makes sense. Is it a compelling story? Does it feel useful? Does it have a good UP-FRONT VALUE PROPOSITION (C2)? Are there any obvious problems with it?
Rich scenarios help you try out design ideas before even building software. They can also provide an idea of which genre patterns and other high-level patterns might be appropriate. Our scenario with Victoria, for example, would tell us that we should look at the PERSONALIZED RECOMMENDATION (G3) and GIFT GIVING (G6) patterns in Pattern Group G—Advanced E-Commerce. These patterns would be necessary, in addition to the patterns in Pattern Group F—Basic E-Commerce, to support the customer goals in the Victoria birthday scenario. Having several personas and scenarios will help you determine which other patterns might also apply.
Scenarios are also useful for describing to clients and customers what a Web site will offer. They tell us about particular customers, describing their situations and what they're trying to accomplish.
We have already mentioned that you can create several scenarios that illustrate your personas accomplishing a variety of goals or tasks. How do you choose these tasks, and what should they look like? These tasks will come from your initial task analysis and will be enriched by later observations and interviews with real customers. The tasks in your scenarios should be detailed, providing specifics about the customers and the situations. Remember that a task description does not say how it is accomplished.
The tasks in your scenarios should also be representative; that is, they should be real tasks that customers or prospective customers currently or eventually want to accomplish. You might say, "I'm inventing something new; nobody has ever done the tasks my site will allow!" Your site might allow someone to do things in a new way, but it is quite rare to invent something entirely new. For example, before sites like evite.com, you could not use a Web site to invite your friends to a party and check on their RSVPs. But the task itself is not new. People had parties before the Web was created, and they needed to invite people and see who was coming. Instead of a Web site, they used letters, phone calls, and even e-mail to accomplish the same task.
The tasks you choose for your scenarios should also be common or important. Common tasks are those that will be done frequently. For an invitation site, for example, creating a new invitation and sending it out will be a common task. Important tasks are those that must be done correctly or there will be unfortunate consequences. Again, on our invitation site it is important that customers have the ability to create an account with their name and e-mail address registered correctly, so that people they invite will know who is sending the invitation. Getting this wrong would not make the site very useful.
Finally, make sure that the tasks you use describe a complete activity—that they are entire tasks, not subtasks or pieces of a task. Thinking about complete activities forces us to consider how features will work together, which is important because the tasks in your scenarios will become the basis for the site design and for the tasks used in future customer tests of the site.
Imagine that you're creating a Web-based banking site. You might decide that there are three different scenarios, each with a different task: (1) checking a savings account balance, (2) checking a checking account balance, and (3) transferring funds between savings and checking accounts. If you develop these three features independently and then later test them, they might work just fine. Unfortunately, you might also end up with an awkward design and not know it until the site launches.
A more realistic, complete task is a combination of the three subtasks just described to achieve a common customer goal: "Make sure that I have enough money in my checking account to cover the last check I wrote." This task would require first a verification that the checking account had sufficient funds. If not, the customer would next check the balance in savings and then move some money from savings to checking. If your banking site were designed to support only the three subtasks, using them in combination might be tedious. For instance, the design might require going back to a main menu to select a new operation. Only by knowing about complete, realistic tasks in advance would you be able to smoothly support these tasks in your design.
The task analysis and sample scenarios generated so far are based on your knowledge and intuition, as well as on any interviews and observations you might have carried out with prospective customers. In the next section we describe in more detail the techniques you can use to get feedback from customers. In addition to helping you complete your initial analysis, these techniques can help you see if the analysis and scenarios are correct.
Observe and Interview Customers
Techniques for observing and interviewing customers can be quick and informal, consisting of conversations over coffee, or having customers show you what they do now in their homes or at work. We have found that most people are willing to help, especially if you explain that you're using the information to improve your site for them. And paying for coffee or lunch doesn't hurt.
Ethnographic Approaches Can Be Used to Observe
Ethnography is a more formal technique used in sociology and anthropology to observe and interact with people. Ethnographers study people in their normal environments. The advantage that ethnography has over techniques such as interviews and surveys is that you can watch what people actually do, as opposed to what they say they do. You can also ask them questions while they show you what they do, to verify your inferences. You can see the people with whom they communicate, the tools they use, and the kinds of things they create—things that may be difficult for participants to remember or explain when taken out of context. Although a rigorous ethnographic observation can be difficult and time-consuming, a more informal and "ethnographically inspired" field study can be fast and still yield valuable information that can drive the design.
For example, if you are building a banking site, your ethnographic research might include visiting a bank for a day and studying all the different types of transactions that customers perform with the teller. Because you are also extending banking capabilities into the home, you might also study the financial activities people do at home, such as paying bills, checking balances, and transferring money. Ethnographically inspired observation is easier than you might think. First recruit some participants and ask if you can follow them around for a day or two and watch what they do. If they don't mind, use a digital camera to take pictures of their workplace or home. These pictures will make it easier to describe what you learned to your clients and to the rest of your design team.
In addition, see what kinds of Web sites people visit. Ask them to take you through your Web site or through a competitor's Web site. Ask them what they like and dislike about the sites. Look at the kinds of tools they use and note the kinds of information they use to make decisions. Ask questions to make sure you understand what they're doing. Run your interpretations by the customers to see if you're right. Look for any sign of disagreement, even signs as subtle as "Huh?," "Umm," and "Yes, but …" Customers will feel uncomfortable until you phrase the question correctly, but they may be hesitant to come right out and say so.
Follow Up with Informal Interviews
Tell any customers you interview what the Web site is supposed to do, and ask them what kinds of things they would like to do. Ask if they have any ideas about how they would organize and structure parts of the Web site. Show them sketches and scenarios and ask what they think about these conceptualizations (do not show these sketches too early in the observation or interview because doing so might bias the customers). Although they cannot develop the design for you, customers can certainly provide a lot of useful information.
When talking to customers, phrase your questions carefully. Do not lead people toward a certain answer. Questions that do not lead to a simple yes or no answer are preferable. For example, questions like "Would you like this feature?" do not work well because most people will just say, "Yeah, sure, why not?" You want to ask questions that get people talking so that you can hear what they're thinking.
A better way of asking a question is to show two alternatives and ask people what they think. It is difficult to judge something by itself. It is easier to compare the differences between two approaches.
Another way of phrasing the question is to give people a list of features and ask them to state how important each feature is on a scale from 1 to 7, with 1 being "not important" and 7 being "very important."
Recording observations and interviews using an audio recorder can make your job much easier and the data you collect more reliable. It's hard to keep up with taking notes while you're watching someone in action and speaking to them. Use your notes to record only the most important things and remember to write down the time that these events occur. This information will make it easier to find the corresponding place in the audiotape later. Transcribing your audiotapes can be tedious and time-consuming, but having a transcript is valuable for noticing subtle issues that you might have missed and helping you confirm inferences you might have made. There are several services that will transcribe audiotapes for a modest price. We recommend that you take advantage of such a service if you can.
Organize the Information You Discover
Your observations and interviews will result in a lot of data. Organize and make sense of that information. In affinity diagramming, for example, you arrange all the individual points and concepts you have gathered on a wall-sized, hierarchical diagram (see Figure). Write each concept on a Post-it note. Group related concepts together, and draw lines between related concepts in different groups. Use different colors to denote groups and even groups of groups, creating a hierarchy. The affinity diagram gives your team a visual explanation of the customer's problems and needs, all in one place. Affinity diagrams can eventually become the basis for your initial information architecture, and they are good starting points for scenarios and storyboards.
4. An affinity diagram organizes the information resulting from your interviews. Over time, this diagram can result in a site map representing the site's information architecture.
Card sorting is another technique for determining the best site organization that is easy to carry out. It helps you understand how to group items so that people will be able to find what they're looking for by recognizing the groups. It also helps you find and fix terminology that would be hard for customers to understand.
To understand card sorting, imagine you wanted to organize a deck of cards. You could organize the cards by suit, separating them into clubs, spades, hearts, and diamonds. However, the cards could also be organized validly by number, that is, grouping all four kings together, all four queens together, and so on. Alternatively, you could organize the cards by color, separating the red cards from the black cards. Because these would be relatively large groups, it might make sense to further subdivide the red cards, by suit or by number.
Likewise, there are many ways of organizing Web pages. The point is that there are many valid ways of organizing content, and it all depends on what you need. You group pages in a way that makes sense, and then later you name the resulting categories. Variations of the card-sorting method include sorting the categories into subcategories, or even asking customers to carry out the card sorting.
Card sorting can be a useful exercise if you need to create or validate the organization of a site. For example, suppose your site starts with the following content:
Depending on your target customer, your sort might come out differently. If these were categories for a grocery site, you might sort them as follows:
If customers were particularly concerned about freshly picked, locally grown fruit, you might sort the cards in this way:
If customers were concerned about pesticide use, you might sort the cards like this:
Just for the sake of argument, a botanist would probably sort the cards completely differently, maybe like this:
If you carry out a card-sorting activity with several customers or other team members, cluster the resultant groupings to understand where they agree and where they do not. Pay attention to items about which consensus could not be reached. Would renaming the item improve things? Some items might fit into several categories. An easy way to visualize the data is to use IBM's EZSort or NIST's WebCAT (see the Resources section later in the book for details on downloading these tools). Asking these questions about where to put each content item and what to call the categories will help create the most customer-friendly content structure possible.
Know the Limits of These Techniques
Sometimes it is useful to observe how people work for at least a few days. The amount of information you learn about your customers during this time will be tremendous. However, it is difficult to capture some kinds of information in such a short time. For example, how do the environment or priorities change for families during events such as holiday seasons or childbirth? If you observed a family for just one or two days, you would probably miss things like this.
Unfortunately, few of us have the time to carry out interviews and observations at this level of depth. This is one of the trade-offs you will have to consider, depending on your situation and the type of site you're attempting to design. Extended fieldwork might not be as useful for a site that customers occasionally come to for e-commerce transactions, but it may be crucial to spend lots of time on this type of field study for a Web application that someone will use for much of the day, every day.
Survey Your Customers
Traditionally used in market research, surveys are another useful way of finding out about your customers and helping you confirm who should be your target audience. Surveys are used to gather a great deal of information from lots of people. If you are planning to revise your Web site, consider adding a survey on the existing Web site to get feedback from current customers about what they like and do not like. Use a combination of multiple-choice and free-form questions. Multiple-choice questions make your data analysis task easier and allow the respondent to move through your survey faster. Free-form responses let people write at length about what is right (and what is wrong) with the existing Web site.
Surveys can be delivered in several different ways. You can survey people in person. One especially effective technique is to approach people coming into a shopping mall and ask them whether they would like to help improve a Web site. Mailing surveys to your target audience or using a market research firm to ask survey questions over the phone are also common techniques. When mailing surveys, be careful to get responses from a representative sample of customers. This problem largely goes away with telephone sampling. All three of these techniques can be expensive and take a lot of time to get results.
The Web opens up the opportunity to carry out survey-based research online. Several firms offer this service, or you can have your development team build a simple survey tool for you. Web-based surveys can be delivered via e-mail to research participants (from your customer list or from a list you buy from a market research firm), in pop-up windows to a randomly sampled set of visitors to your site, or as a link on a feedback page. This flexibility offers you a powerful way to get survey results about your customers quickly.
Convincing people to participate in a survey usually requires some enticement. Offer potential participants a chance to win a prize in a drawing, a T-shirt, a coffee mug, or cash. The reward you offer will depend on how much time the survey takes to complete. The longer it is, the more you have to "pay." What you pay might range anywhere from $15 to $100 (U.S.). This, along with the fact that people will simply stop participating in a survey that is too long, is one reason to make your survey take less than 15 minutes to complete. This is especially true for Web-based surveys, where people will give up even more quickly. The reward will also depend on the type of participant you are trying to recruit. It is not unusual to offer executive-level participants cash compensation or a donation of $200 or more to their favorite charity in exchange for 45 minutes of their time.
Surveys can be tricky, though. At best, you can get a lot of data and make conclusions that are based on quantitative results. This can be helpful for convincing others in your organization about the usefulness of the results. Unfortunately, when you want to make quantitative conclusions, you need to be sure that those conclusions make sense from a statistical perspective. Surveys have to be designed properly to give you reliable data—results that would be found consistently if you ran the survey over and over with the same type of audience under the same conditions. Also make sure you have enough participants, and get a high enough response rate so that you achieve statistical validity—results that are highly likely to be right.
A lot of the survey work we have seen on the Web simply falls down on one or both of these issues. We recommend that you work with a firm that has expertise in this area, as well as read a good book on survey design to understand the full impact of drawing conclusions on poorly collected data.
Another issue to watch out for is that surveys report on what people say, not on what people do. A lot of research has found that what people say does not always correspond to what they do. This is especially true when people have to reconstruct specific details about what they do on the Web. People can remember things at a high level, but they tend to quickly forget the details. Despite these shortcomings, surveys have been proven very effective over and over for determining the target market, product and concept feasibility, price elasticity, attitudinal and brand image, general opinions, and preferences. Ultimately, however, if you want to know what people really do, or are going to do, you should watch them in action.
Run Focus Groups
Focus groups are commonly used by market researchers to find out about customers and their opinions. In a focus group, a handful of people (6–12) who are representative of target customers are brought into a meeting as a group. They may or may not know each other beforehand. As in the interviewing process already described, people are asked questions about competitors' Web sites and about the proposed Web site. If you are revising your Web site, you might ask the focus group what they like and dislike about the site. If you are creating a new Web site, ask them the same questions about your competitors' Web sites. Get their feedback on the proposed Web site by showing them sketches or pictures of how it will work. It is also common to present scenarios of future use to see how these ideas resonate with the group.
Just like the Boy Scouts, focus groups have the motto "Be prepared!" Do not go in blindly and hope you will find useful information. Identify what you want to find out. Have an idea of what you're looking for, and make sure that all of the questions you ask will help you learn whether you're going in the right direction. Also be ready for criticism. Although it may sting a little in the short run, it will result in higher-quality designs in the end. Other members of the development team or management can sit in on these meetings. Hearing comments directly from customers is much more convincing than reading reports. Be sure to keep the number of these insiders low so that you don't overwhelm the focus group.
Focus groups are difficult to run well. Often the moderator can be too controlling and drive the group to conclusions that he or she would like to see. Another common problem is that an individual in the group dominates and causes groupthink to emerge as the other members defer. You will have to get the dominating person to quiet down so that you can draw other members out. Find moderators who have experience running a focus group because they will be familiar with these problems and know how to handle them gracefully.
Note also that you may get different results, depending on the chemistry of the people in the group. Sometimes your results will be positive, sometimes negative, and very negative—all in response to the same questions and the same examples. For this reason it is usually a good idea to run a focus group several times, with different types of people, in different geographic locations. Also be careful in your recruiting to avoid professional respondents, focus group members who make money on the side by going from group to group. You want to get people who are representative of your customers, not people who are just conveniently available.
One caveat about focus groups is that, like surveys, you can learn only what people say, but not necessarily what they actually do. In other words, you can learn a lot about their attitudes and their perceptions, but not much about what they might do in practice. This is why focus groups are more useful for the early stages of design, when you are more interested in finding out about your customers than in trying to evaluate what they will do with a Web interface that you have not yet built or even prototyped. This kind of information is still valuable, but it should be supplemented with the other techniques described here.
Analyze Existing Web Sites
Another way of getting information about potential customers and their needs is to ask them to evaluate existing Web sites. Use your existing Web site or a competitor's Web site to get a feel for what's right and what's wrong. Recruit some representative customers, and observe what they say they want to do on the Web site, what they actually do, and what steps they take to do it. Make note of the kinds of mistakes they make, and pay special attention to what they say they like and don't like. You might also want to have a questionnaire that they can fill out, to learn more about their demographic information and their interests and subjective ratings. In Chapter 4—Involving Customers with Iterative Design, we discuss in more detail how to do this.
Start your analysis by finding all the people who sent you e-mail about your Web site. Whether they suggested a new feature or criticized a feature that did not make sense, these are the customers who cared enough to make a comment about your site in the first place. If they live close enough, consider asking if you can visit them for an interview, or if they can come to your office to help evaluate the Web site.
But do not rely only on this type of customer. Because they have voluntarily sent a complaint or suggestion, such customers have already been self-selected as having a certain type of personality or level of expertise that may differ from that of the rest of your customers. Make an effort to find a wide range of people who are representative of your overall customer base.