Validating ideas fast with remote user research

Remote user research – using technology to carry out user experience research without being present – is a key part of our agile approach to UX design. Covering everything from self-moderated usability testing and feature validation to card sorting, in-depth interviews and tree-tests, it basically means doing the same research we’ve always done, but taking advantage of the latest technology to ‘crowd-source’ our results, improving quality and speed and adding value for our clients. At Evolve we’ve carried out hundreds of hours of remote research on a variety of projects, and we’ve learned a few things along the way.

This article answers some questions for anyone considering remote research, like the tools should you be using and designing research for remote users.

Why go remote?

Remote user testing has some big advantages over traditional face-to-face tests. Firstly, it’s quicker to set up, and quicker to get results. It’s possible to access a large pool of participants, screen for suitability and start remote user testing, all very quickly. Remote user testing also makes it possible for you to stretch your resources further. You can run multiple tests at the same time so you get more out of a project’s lifespan and budget. And testing sessions don’t use up your time personally because you don’t need to be present. Other advantages come from the nature of remote testing itself. Because you’re not physically there when your participants complete the test, it’s a cleaner test environment. Users are in their natural habitat, meaning they’re more relaxed.

There’s no temptation to intervene, and no social pressure on the person taking the test. That means you get the unedited truth! Remote testing is also more logistically flexible than face-to-face testing; participants can be recruited easily from anywhere in the world, and you save costs on travel because they can take part from anywhere.

Is there a downside?

Of course, like any research method, remote testing has its disadvantages that need to be mitigated for. Without being able to see the wider context of a participant, and not being face-to-face with them, it can be harder to build relationships, although using video connectivity could help to bridge that gap. Some drawbacks are technical, especially when you’re working globally; for example, local infrastructure or a participant’s access to technology may impact on connection speeds.

Using the right tools for the job

Several easily accessible online tools can be used for remote testing. Survey tools like Google Forms and Survey Monkey and video calling and conferencing platforms like Zoom, Facetime, Skype and meetonVC are widely available and often free. There are also a range of qualitative tools, like Optimal Workshop, Notable and User Zoom, which offer functions including card sorting, tree testing and click testing. With these platforms, you can upload a prototype design and get participants to annotate it, conduct card sorts or see what participants remember. For qualitative research, such as self-moderated remote usability testing popular tools include, WhatUsersDo, Validately and Loop11. UserZoom also offers a similar service alongside their qualitative features.

Choosing the service that’s right for your project will depend on budget, but also on the type of research. If you want to access a particular demographic, certain price plans on the dedicated platforms allow you to do your own recruitment, for example. You may also need to consider what software users will need to install to participate, as this could potentially be a barrier to taking part. Tools offering native Android or iOS apps may be the best option.

Creating effective tests

The fundamental question you should be asking when designing a test remains the same whether you’re remote or face-to-face; what are we testing? Start by defining your objectives, and then write the tasks to those objectives. That way, your results will be relevant and useful. Beyond this, with remote testing, the key consideration is that you won’t be present when the participant carries out the test. This means you have to:

  • Be succinct. Make sure tasks can be written on a post-it note. Any longer and you risk users forgetting what they’re meant to be doing.
  • Avoid ambiguity. You won’t be there to explain what you’ve written, so you need to make sure users can grasp it quickly.
  • Establish a rhythm. Users need to know what to expect in a task, and if anything is going to behave differently to the rhythm, it needs to be explained, so they don’t assume something has gone wrong.
  • Avoid leading questions. This is true with any research, but when you’re not there to monitor responses it’s particularly important.

It’s often best to recruit an extra participant to validate that your tasks work the way you intend before you start.

Testing problems

It’s important to hold your research to the same standards of proof required of traditional research methods; remote testing might be easier and faster, but that shouldn’t make it less accurate. This means showing your working out – often possible through video evidence on many of the self-moderated solutions. Also bear in mind that some specialist platforms don’t limit the number of tests a participant can do, with the risk that users can become over-familiar with the process, making them non-representative of a normal user.

Try before you buy

You may encounter resistance to the idea of remote user testing. To overcome this, many of the remote user research platforms offer free trials, a great way of demonstrating how well it works to colleagues before dedicating budgets. As platforms become more sophisticated and an agile approach to UX design becomes mainstream, remote user research will become increasingly popular. The question you should be asking yourself right now is not, what do we gain from remote research, but rather, what do we gain from still doing it face-to-face?