BACK TO BLOG

UX Design: The most comprehensive guide to user testing

User research is an essential part of the Design Thinking and User Experience (UX) design process. The application of user research is to 

• Unearth new consumer behaviors  (Design Thinking)

Discover innovative ideas by observing what users do

• Usability testing (UX Design)

Close the gap between designers’ and users’ mental models

By getting feedback from those who will use our product (or service) regularly, we want to make sure that the design will delight and help them to get things done adequately.

At Bonanza Design, we offer two packages:

• Design Thinking Package — a four-week process

By which we discover new ideas and design user-centered solutions. At this phase, we conduct extensive user research to study users’ needs and discover new patterns of consumption

• UX Design Package — a three-week process

By which we design pixel perfect applications. At this phase, we rely heavily on usability testing to make sure the finished product is usable and engaging

In this article, we offer you our hard-earned experiences doing user research and usability testing for years. We categorized the insights into four segments

• A: Before testing
• B: During the testing
• C: After testing
• D: Interview questions

TIPS FOR USABILITY TESTING

PART A: BEFORE TESTING

— How to recruit users

• Cheap Option – Fast but NOT reliable

Tap into communities across social media such as Facebook groups, Linkedin groups, Reddit channels, etc. You can source testers within a few hours. 

This is a great option for testing cases that deal with consumer products. The only caveat is that testers could cancel on you easily. And it happened to us (a lot!)

TIP: If you need to source 5 testers, aim to book at least 8.

In this case, you have to create the NDA, book the time with the testers, and guide them into the facility. All on your own. It’s resource-consuming for sure.

• Expensive Option – Slower BUT reliable

Through platforms such as Testing time, you can book testers. Platforms as such offer options to filter your search. This is a great option for testing cases that deal with Business to Business (B2B) products.

Bear in mind that, by adding a few filters, the total price of sourcing 5 testers might easily go over 1000€. Use this option when you deal with a niche market in which finding the right person is strenuous.

In this case, you don’t need to do anything. The platform books the testers and makes sure they find their way to the doorstep of your office. You just need to open the door and say hello.

— Prepare your interview script (testing plan) carefully

Your interview script is the guideline by which the team aligns on the purpose of the project and interviewer(s) performs their job. Sometimes you have to bring a professional from the outside of the organization to get the job done. Hence you want to make sure the interview script is clear and includes all the sections below:

• Intro about the project

You don’t need to go in-depth explaining the project but it should offer enough information so both the interviewer and testers understand the purpose of the project.

• Testing objectives

You want to explain the objectives of the testing. The clearer the objectives are, the more promising the results of the testing.

The next chapters are about the questions you want to ask. Try to be brief and ask only questions you need to answer. Your time is limited; hence your questions.

• Background questions (mostly demographics)

This is the part to write down the general questions you have about the users including where they live, what they do and their age etc.

• Open-ended questions about their habits

Here you get more specific about their habits when it comes to the topic of the testing. For example, if you’re developing an app to track users’ diets, then the open-ended questions relate to how users deal with food daily.

• Tasks to perform

Here to write down the tasks you’d like the testers to perform in case you have developed a prototype.

— Understand the kinds of task you want testers to perform

• Get their Impression  

For example, you’re developing a landing page for a new product, and you’d like to know how testers feel about the information architecture and branding. 

To get the testers’ impression, try the Five Second Test exercise first. Show the design for only five seconds. And then ask them about their impression. And then show the design again but this time with more time, and record what’s changed in their impression. 

• Perform a task

This is for when you’re developing a prototype of an app that performs certain functions. Then you want to test those functions with users like signing up or booking an appointment.

IMPORTANT: Often we only design tasks to test how usable the application is. Seldom do we design tasks to test whether testers are willing to contribute to the growth of the business and user base.

A user can contribute to the growth of the business in two ways

• By subscribing to the application 
• By inviting their friends to sign up for the application

When you design a testing plan, it has to include a few tasks to test whether a user is willing to contribute to the growth of the application. How?

• Pull up an overlay at the end of the testing which asks testers to subscribe to the app. It should feel sudden and random. Try to surprise them here. 

You want to get testers off-guard so their reaction would be genuine and honest.

• Pull up an overlay screen and ask testers to invite their friends to the app. Again try to get testers off-guard so their reaction would be authentic.

— Allocate someone unbiased to lead the usability testing

Do not let the designers, product managers, or developers, who are invested in the project, to be the ones who interview testers. This is especially important if you have a prototype. Find someone outside of the team. 

— Double-check the wireframes again with the designers the midday (not evening) before the testing day

Midday is important! In case there are things to improve, you’d have enough time to implement changes.

— Try to meet some of the users in their own space

Meet users where they are. Especially important if you’re conducting user research. For usability testing, it’s okay to invite them to your office.

— Only show the final designs to the testers

Don’t show testers low-fidelity prototypes. Testers won’t take the testing session seriously. When you have low-fidelity prototypes, then test them internally so colleagues can spot the ice-berg-type usability problems. Then finalize the design and test them with real users.

— For a proper user interview, you need 45 minutes

We’ve tried 15, 30, 45 and 60-minute formats, none of them worked better than a 45-minute format. It’s just the sweet spot.

— You have to at least interview 5 testers

Study shows that interviewing five testers reveals about 80% of all the potential issues with your design. Also, our experience proves this claim right. On top of that, it’s only possible to test five users in a day.

— Use Rainbow Sheet to capture interview outputs

To capture interview observations, notes, and scores for each task, use the rainbow sheet. Read about it here and look at it here.

— For a proper interview, you need two rooms

Try to have two rooms. One room for the interview. The other room for your team to observe the interview via a TV and take notes. The Interviewer does only lead the session; the team does the note-taking. 

— Schedule the interviews properly

Only do user testing in the morning and early afternoon. Leave the rest of the afternoon for the reflection and sorting insights

PART B: DURING THE USER TESTING

— Look in their eyes, and introduce yourself and your role clearly

Hi there! My name is Claire and today I’ll be guiding you through the process. On behalf of my team, I’d like to thank you for being here with us. Your feedback means so much to our team.

— We don’t test you; there’s no right or wrong

Remind testers that we don’t test them and we’d like to get their reactions to certain things we’re working on. Remind them that their critical eyes could help so much to improve the experience.

— Don’t jump into the conversation right away

Make testers feel comfortable first. Use an ice-breaker. Find a relevant topic to talk about e.g. today weather.

— Observe the reactions to the task

Your task as a testing team is to observe user behaviors. As the interviewer, stay in the background and don’t disturb the tester flow. 

— Ask testers to sign the consent form before hitting on the record button

You want to record the testing session but before testers need to give their consent. Just google user testing consent form, and you get what you need.

— Press the record button before you begin the interview

Don’t forget.

— Record how long it takes to perform each task

Don’t help testers during the task performance. Let them deal with the task themselves. If they’re getting frustrated, don’t interrupt. Observe their situation and encourage them to talk about their frustration.

— Remain unbiased towards design

Be unbiased towards each screen and try not to point their attention to certain parts of the design.

— Encourage testers to think out loud

Encourage testers to think out loud constantly. Ask them questions when they’re silent for a while. Silence yields more silence. Be careful. It’s a fine line though. Don’t keep interrupting them constantly. Give them enough time to express their thoughts seamlessly before you ask them a question.

— Try to criticize the design yourself

I said you should be unbiased. However, here’s a trick. Early on, try to make a light-hearted joke about the design yourself e.g. Oops our designers forget to change this font again! 

Doing that, it encourages users to express their judgment without the fear of being judged, wrong, or impolite. They see you as their friend. And that’s important. 

— Answer questions with questions

Do not help testers to figure out things. Period.

— Wrap up and reflection

Allow testers to reflect and summarize their experience and add the last points if they desire. Do not skip this part. Sometimes testers need time to conclude their thoughts and give their final feedback.

PART C: AFTER THE TESTING

— What if the testing results were super negative?

Great. Then iterate. Don’t be jaded by the results of the test. There’s no failure. Feedbacks are there for you to improve your work. Happy iterating!

— What if the testing results were super positive?

Great. Then continue with the detailed design and preparing the assets for the development.

— What if the testing results were somewhat positive?

Reflect. Derive clear action points from studying the results of the tests, improve the prototype, and TEST again. 

IMPORTANT: Do not continue without testing the improved version with the testers again. You want to proceed only if you got super positive results from the follow-up user testing session.

PART D: TYPES OF INTERVIEW QUESTIONS

We have an amazing Instagram post that visualizes this section. Take a look here 

This section is taken from the design thinking playbook. It’s a fantastic book and you should have it on your bookshelf. 

• Tracing behaviors

  • Who taught you that and how do you know it works?
  • Why do you smile when you say that?
  • If this was a difficult task, why was it difficult for you?

• Gaining clarity

  • What exactly do you mean by this?
  • How would you describe it in your own words?

• Asking about a sequence

  • Tell me more about the first memory of…
  • and what happened before or after the time you…?
  • How did you do this before? How did you improve?

• Asking for examples

  • What software are you using?
  • What was the last app you downloaded? With whom did you discuss it?

• Exploring exceptions

  • Did you have problems with this before?
  • When and why didn’t this work, then?

• Understanding connections and relations

  • How do you communicate with your boss?
  • From whom did you hear that? And how did you hear of it?
  • Who helped you with that?

• Informing outsiders

  • If you had to explain it to an exchange student, what would you say?
  • How would you explain this to your grandparents, or even describe it to a small child?

• Comparing processes

  • What is the difference between your home and that of your friend?
  • What is the difference when you do this on the road instead of at home?

We hope that you find this article helpful. As always in case you have feedback to share reach out to us at contact@v2.bonanza.design