Remote user research in an online world
In such challenging times, we believe the collective and growing challenge to engage, understand, collaborate with and serve citizens, communities, businesses and government in crisis situations (e.g. COVID-19 pandemic, bush fires, droughts, floods and other disasters) is one that is best met through digital service innovation. Richard Phillips, Lead UCD Practitioner at Informed Solutions discusses user research in an increasingly online world.
Since March 2020, we have been living in challenging times. For the past five months the COVID-19 pandemic has forced a huge variety of professions to pivot and scramble in order to cope with the fast-changing and worrying health and economic landscapes that we now live in. The relaxation in social distancing and drivers to get economies back on their feet again are taking place at different rates across the world, depending on a country’s individual circumstances as it relates to the decline of infection and mortality rates.
Travelling through such difficult times, we believe the collective and growing challenge to engage, understand, collaborate with and better serve citizens, communities, businesses and government in crisis situations (e.g. COVID-19 pandemic, bush fires, droughts, floods and other disasters) is one that is best met through digital service innovation.
As an international digital transformation practice with experience in designing and delivering national programmes for both Australian and UK Governments, Informed Solutions’ team of user researchers are well versed in executing remote research including; user interviews and moderated and unmoderated usability testing, with teams on both sides of the world connecting and engaging with people in remote locations and different time zones.
With remote and colocation hybrid working patterns set to continue for the foreseeable future, Informed Solutions Lead UCD Practitioner, Richard Phillips shares his thoughts on the critical, early stages of service design around successful remote user and stakeholder engagement as a platform for successful digital service design and delivery.
Executing research purely online, the researcher has little control over a user’s environment except through the online conferencing tools that are available and accepted. Just coming out of lock-down in Australia, the Informed Solutions team was part of a user group undertaking a combination of guerrilla and remote user testing to better understand the user experience of citizens using digital identity to apply online for government welfare during COVID-19 instead of physically proving their identity by visiting a welfare services shopfront.
Taking a mobile app for the purposes of usability testing into the public domain and asking passers-by for feedback during this time had its challenges, however we identified the geographic areas for the project ahead of time and prepared for our different approach and the difference in acceptance of passers-by in both city centre and more remote community locations. Additionally, although the testing was carried out with full compliance to social distancing and sanitisation guidance, peoples’ attitudes to risk differed and so the speed and efficiency of feedback capture varied too.
The guerrilla testing moved quickly to remote user research as full lock-down was confirmed, which introduced both challenges and benefits with regard to the ease and effectiveness of conducting research solely through digital channels.
For remote user research, it’s really important to be interested and engaged with your participant for any research, but especially around sensitive and challenging COVID-related issues. When you’re on a video call it can be challenging to pick up on the micro expressions and social cues you would normally and relatively easily identify in face-to-face settings. You’ll also find you have to be more engaged than usual to make an authentic connection. It can be difficult to empathize with people over an online connection, so actually seeing the person on the other side of a remote research call has a significant impact on your ability to connect with people, gain a decent measure of engagement and secure meaningful feedback on service design.
From experience, participants will give more thoughtful responses and be open to conversations that delve deeper into issues if they can see who they are speaking with. Added to this, with people working from home, some may be more open with us if they feel more relaxed and abstracted from the workplace and the pressures that brings.
When cameras are off, the act of capturing accurate feedback and responses becomes even more challenging: User research depends not only on what users say, but how they say it combined with a physical, body language-related reaction. If you can’t observe someone’s facial expressions in order to understand their emotional response to a question or a particular piece of information, user feedback can become somewhat of a guessing game – judging silence or pauses in a dialogue against thoughts or emotions that might vary all the way from confusion to discomfort.
Another interesting challenge for remote user testing relates to observing a user’s experience navigating a digital service through a variety of devices to fully understand the end-to-end user experience through a particular web service or device.
In face-to-face settings, user researchers are armed with task cards that can help the flow of a session. In remote, online settings, researchers have to prepare digital task cards and work out timings in order to best present when needed. Added to this, the researcher will have to take screen size and screen real estate into consideration for a user to be able to view both the test prototype and the task card. The tech should not get in the way of the user experience.
Outside of channel shifting, we allowed extra time to test the tech and conduct dry runs to ensure everything ran smoothly, including order and timing for feedback completion as well as screen sharing and control functions that allow a user to navigate a prototype system effectively.
Alongside the different challenges associated with online user research there are also benefits including; the ability to recruit for a wider geographic spread of participants – particularly those living remotely who often have different user needs to those that are easy to access. This change to ‘all remote’ research really drives you to have higher engagement with this group, whereas in ‘normal’ times there may be a tendency to concentrate on those users that are easier to recruit – another positive amid the challenges.
Additionally, when you’re working with an agile team, remote research allows you to fit many user sessions into a single day and iterate quickly. It can be an extremely valuable addition to your research tool belt. Although a positive, it presents the researcher with an interesting balancing act as researchers might be expected to speak with more people, however you still need to spend the same amount of time doing analysis and downloads of findings. Even though you might be able to fit more slots in, you often can’t actually engage with a huge increase of people if the requirement is to derive meaningful insights from each session – something that a user researcher always needs to be aware of in terms of structuring a round of research properly.
There are many tools out there to help you conduct remote user research. But collecting data is the easy part, what you do with it is vital, especially when it relates to the collection of information concerning citizen welfare. Research alone provides data, not insights. The key to making any piece of research a success is how you use that data to generate actionable, prioritised recommendations that produce effective online services
Capturing this type of data allows us to accurately articulate the complex problems that require solving and most importantly work collaboratively with project stakeholders to align user and organisational needs. This alignment and putting people first is what helps ensure that a digital design succeeds the first time around.
As the winner of a Queen’s Award for Innovation, we work with some of the world’s largest and most respected organisations, delivering digital services that are used by millions of people every day. The critical first steps outlined in this piece provide a reliable foundation for our digital transformation projects. They allow us to progress onto designing, building, testing and delivering optimal solutions that users are satisfied with and continue to use with ease. Our thinking is always shared across Informed Solutions’ Communities of Practice allowing us to share, shape and refine best practice for remote user research tools and techniques across our international operation.
Originally posted here
