Our reality has changed overnight. Due to the COVID-19 epidemic many restrictions came into our professional lives. At Symetria we asked ourselves: how to effectively conduct UX research in times of transport limitations and inability to meet with the user face to face? We came to the conclusion, that, surprisingly, the world of UX research and design won’t change that much. Many previous research methods were based on remote communication. What’s more remote usability tests itself came to life 24 years ago. We have now gathered Symetria’s experience and knowledge in the field of UX remote research methodology. As a result, we created an article in which you will learn how to conduct remote usability test step by step. We will talk about the tools, the effect they will bring and what to expect when conducting such research.
Step 1 – Where to start?
Imagine: you start a new project and qualitative research is an important part of it. You’ve done plenty of usability tests in the past, but this time you find out you can’t conduct the test in the lab. Perhaps you wonder where to go from here. The first stage of your project remains the same: you need to figure out what would you like to explore and choose a suitable method. In case of remote test you have the following choice:
- Remote usability test via Zoom, Whereby, User Testing, Loop11 and UserZoom
- Complementary qualitative research with RespoTeam ™
Step 2 – Conducting the test
Let’s assume that your project concerns an online banking process. In addition, you want to learn about experiences of the users with a similar product and explore their perception of banking as such. You decide on the first method: remote usability testing.
Afterwards you have to make another decision. The test can be conducted in a similar way to those taking place in the lab: the user and moderator meet, however it happens online. These are called remote moderated usability tests. However, it is possible to conduct it without the participation of the moderator. Softwares that are currently available allow you to invite the user for a test, which automatically guides them through the tasks and questions you designed. Which option should you choose?
Considering that the study will contain elements of an interview, moderated usability testing combined with IDI will definitely work best. In the next part you will learn: what will happen during a remote session, what actions the moderator must perform and what the user will do.
Connecting with the user
First, you need to connect with the participant of your study. Video-conference programs such as Zoom and Whereby come to the rescue. These softwares have some limitations in the free versions, but for a small amount you can significantly improve the comfort of conducting the research. Moderated tests can also be conducted using the User Testing or UserZoom platform.
During the test
After successfully connecting with the user, you have to share the prototype with them. You can do it by sending a link to the prototype or the website using the chat option. Then the user shares their screen view with you. This function is very easy to use and is included in the free version of both mentioned platforms. Our favorite for now is Whereby, for a simple reason – it does not require the user to install any app on the device. Installation can be problematic for two reasons: firstly, not all users know how to do it; and secondly, they can simply be reluctant of installing an unknown software on their device.
We also want to record the session and enable a live stream for the client to observe. Not much changes in this matter. You only need to record the session with the video-conference program. Both Zoom and Whereby offer this function. Live stream can be provided in Zoom by connecting it to your Youtube channel.
You may be wondering what challenges could arise when conducting such study. You can certainly recall a being on a conference call where someone was barely audible and communication was almost impossible. The reason is usually the equipment: poor microphone or slow internet connection. In our experience, this is a rare problem during the one-on-one sessions. However, if you want to minimize such obstacles, you can always make one of the recruiting criteria having an access to high-speed internet.
What about confidentiality? Perhaps you would like to test a prototype that presents innovative solutions that are not yet on the market. Is the confidentiality agreement sufficient? We should definitely trust our respondents. But it’s better to be safe than sorry. Prototypes can be protected with a password, which we change after each test so that the user cannot open it later.
Testing on mobile devices
We have discussed the test of a desktop version so far. Is it possible to run the same tests on mobile device? In this matter we have some limitations. In Whereby, the main obstacle is the lack of share the screen function on mobile, but Zoom does have it. However the best alternative are platforms, that we discuss in the next section.
Step 3 – What if we have less time?
Moderated tests are not all there is. Unmoderated usability tests can be a great alternative for your study. Their advantage is that the data will be collected much faster, because the tests are not limited by the availability of the moderator. Also the users can adjust the time of the test to their schedule. In addition, many people can take the test in the same time. Their potential disadvantage is that we cannot ask the user during the test about problems that we didn’t foresee. It also means that we need to think carefully about the test scenario, so that everything is understandable from the perspective of the participant. In the next part you will learn how to design such usability test.
Choosing a testing platform
There are many providers on the market today, but we will briefly discuss three of them: User Testing, Loop11 and UserZoom. What criteria should you take into account when making a decision? You need to answer the following questions: Do you want to recruit participants? Will you have a small or large research sample? What is you budget?
- This platform will be most useful if you need a small sample and can’t recruit participants on our own.
- User Testing can be bought only as a bundle: recruitment plus the test. Your task is only to design the test; no link to the test is provided. The good news is that you won’t need to worry about recruitment. The downside, however, is that the platform focuses on users from English-speaking countries.
- It is best for a small number of participants, for both trial period and annual plan. During the 14-day trial period using the platform costs 49$ per participant. Then you can switch to one of the paid plans. They are paid annually and the basic one costs over 1000$ per month. In addition, the limit of people participating in one study is 5, although the number of tests is not limited.
- This platform will be most useful if you plan to carry out tests on larger samples and have the possibility to recruit on your own.
- Loop11 offers an unlimited number of tests, but without recruitment. Unlike User Testing, it allows you to generate a link, that you can send to the participants. The recruitment service is offered but is paid additionally.
- Due to the increased demand for online research during the epidemic, Loop11 extended the trial period from 14 to 25 days. After this time, new users can count on the promotional price of the monthly plan as well, which has been reduced from 249$ to 199$.
- This platform also offers both options: with and without recruitment service.
- User Zoom provides the opportunity to run unlimited tests, on an unlimited sample size.
- The platform provides a support system – experts who will help to code and also provide research support.
- The one-year license includes training and support, all to ensure your success. The cost of an annual license starts at 25,000$.
Writing a test protocol
Let’s say you decided to try out User Testing. In order to start, you will have to do two things: determine participant criteria and write the test protocol. That doesn’t seem to be different then what you’ve done preparing to the regular test, does it? The only difference is that you will do both on the User Testing website. Once you log in, you will choose whether you would like to test the prototype, website or mobile app. Then fill out the form describing the participant criteria. Afterwards you will start planning the test. First paste the link of the prototype or website to be tested. In the section Scenario, you will describe the imagined situation in order to give context to the study – the user will see this message first. This is also a good place to remind the user about the think aloud protocol. In the Test plan section place the tiles from the right panel, simply using drag&drop. It will constitute tasks, questions and questionnaires in your test. All that’s left now is to test the scenario with the Preview test button. After the payment is done, you can relax and wait for the recordings to appear on your account, where you can download, forward and analyze them.
Step 4 – How to stay in touch with the users?
Great! You’ve finished all you remote sessions. Regardless of whether you decided to go with moderated or unmoderated tests, after analyzing collected data you might come to a conclusion that some issues are not entirely clear, or maybe even new questions have arisen. Qualitative research conducted with RespoTeam ™ tool will nicely compliment your study. It’s a platform that allows to create a virtual research room and ask clients or users some questions. You can raise various topics: from perception of your brand, to understanding a specific element on the interface. It is a very useful combination of moderated and unmoderated tests.
Setting up a RespoTeam room
In order to participate in the study users do not have to log in anywhere. All they need is a Facebook account. Users can also activate the option of receiving notifications via Messenger, that will send a reminder each time you ask a new question in your room. The answer can be written at a time suitable for the user. Respondents may or may not see each other’s responses and engage in a discussions. Perhaps you are concerned that the questions will be misunderstood by some participants, since they will receive them in a written form. There is a solution to that: RespoTeam provides the opportunity to ask individual participants follow-up questions. Also if you’re not sure how to word your questions in order to obtain reliable results, the RespoTeam can provide support of a research team. Furthermore either you or RespoTeam can recruit the participants. This toll has a great advantage, because the answers are gathered very quickly. We have used this tool many times for clients in various industries: from the largest Polish banks to household appliances & electronics manufacturers.
Step 5 – Analysis and report
Once you collected all the data, you can proceed to analysis and writing a report. In this part, the research process will no longer be any different from the one you go through after a regular research.
What will the nearest future bring?
One thing that we can be sure of is, there will be a lot more research done remotely. That means our skills and tools will be improved in this matter over a short period of time. In addition, users will be eager to participate in the study without leaving their homes, simply because they have more time on their hands, and they will feel safer without being exposed to personal contact.
The costs are also reduced, as there is no need to travel. In a word, remote testing is a time and money saver. Our everyday life, although will definitely look differently in the near future, in terms of UX research won’t go through a revolution. Available tools allow to replicate conditions held in lab, and thus collect the same, valuable data.
A huge advantage of remote tests, for users as well as the research, is the authenticity of the situation. The user is actually at home and viewing the website in their “natural environment”. The test can be therefore extremely similar to the real-life situation, when the user is sitting, in the evening in their favorite spot on the couch.
Alicja Janowska, UX Specialist, Symetria, Poland