Choosing software for remote usability testing

One of the usability evaluation methods we are planning to use soon (alongside face-to-face usability testing) is remote moderated usability testing. There are of course limitations to using this method, and some of these have been listed in the Gov UK Service Manual, such as one could miss out on important contextual or non-verbal information (this is why this method is usually used in combination with other methods). It is also not suitable to use with participants who have low digital literacy.

The advantages are also listed in the manual, which include saving time and costs, fitting participants in when best suits them (especially if you have users like ours who are busy during the working day and tend to have with caring responsibilities in the evening), and including participants based in remote parts of the country. Furthermore, when compared to lab-based usability testing, ethnographers especially argue that remote usability testing has i) less contextual bias, as it takes place in a physical environment natural to the user and with familiar technology, and ii) affirmation bias (an eagerness to please the researcher) may also be reduced by the physical absence of the researcher (further more here).

As the remote testing we plan is synchronous, I have spent some time exploring seven different video conferencing and remote sharing application tools available on the market:

1. GoToMeeting (Pro) – This video conferencing software was the one that many in Government appear to use for remote usability testing. It allows the participant to share their screen, records everything, allows multiple people to join to observe, and does not require the user to create an account. It works on Internet Explorer and on smartphones too once you download an app. It also provides an option to transcribe recordings which I am yet to test. It ticks all the boxes in relation to functionality but does not appear to have the friendliest user interface. It also requires the participant to download a plugin on their browser for it to work. Cost: Pro plan – £24 p/month on monthly subscription or £19 p/month on yearly subscription.
2. Zoom – This software on its Business Plan has all the features listed above for GoToMeeting Pro and appears to have a much more user friendly interface. However, you must purchase it for 10 licensed users on the business plan, which is not an option for us but could be for larger departments. On the Pro plan with one licensed user only, the main limitation is that recording is limited to 1GB of MP4, which does not work for us.  Updated: Zoom (on cloud) now has an automated transcribing function for cloud recordings, which is a huge bonus! Cost: £15.99 p/month (Business plan); £11.99 (Pro plan)
3. Google Hangout (on G-Suite Business) – This piece of web-based software from Google has the features we are looking for, such as the ability for the participant to share their screen and for multiple people to join the video call to observe. It is also one of the cheapest from the options provided here, and does not require the researcher or participant to download any plugin or software. But it does require both the researcher and the participant (and all observers) to have a Google account, which may be a burden or barrier for the participant especially. Whilst testing, I found that with multiple users observing, connection regularly dropped for one of us, which was frustrating. The session can be recorded on the G Suite edition of Google but not on the free version. However, the free version could be used with another recording software, such as Snagit/Camtasia/Nimbus. A fellow user researcher has also shared her experience of using Hangout for contextual research here. Cost: £6.60 p/month for G Suite Business Edition
4. Appear.in – This software was recommended by a fellow user researcher but it does not work on IE. I therefore successfully tested it on Firefox. Unlike GoToMeeting and Zoom, it does not require one to download a plugin so that is a huge plus. However, one of the things that could be frustrating (and perhaps even off putting for participants) is that it forces you to share your webcam at launch, even though you can switch it off once you are inside. On my Android phone, it does not require one to download an app (which is always a plus) but you do have to ask the participant to rotate their screen to see the full screen. The interface is really easy and simple to use but in the free trial, I could not test recording the session nor the participant sharing their screen, and it was not clear in the description whether that is an option in the paid version.  Cost: $9.99 p/month
5. Morae – There is an option to download TeamViewer and run it alongside Morae (see instructions here on how to do that). However, some researchers appear to be concerned that the software records the video of the user when sharing his/her screen, but does not appear to record the audio. Others argue that connectivity can be poor when the observer is not on the same local network. It also feels a bit clunky to use with the need to run two or three pieces of software at one time.  Cost: £1817.05 (one off cost)
6. Microsoft Teams – This web conferencing and collaborative software allows one to communicate with those inside and outside an organisation. However, both the researcher and participant need a Microsoft account and need to download Microsoft Teams to share and record the screen. This does not work for us as access to Teams has been blocked on our work laptops through Windows Firewall. However, it appears to be a neat solution if you need to collaborate with the participant too as well as conduct testing. Cost: Office 365 Business Essentials – $5.00 user/?month
7. Skype for Business – This software has to also be downloaded by both researcher and participant, and again allows one to share screen, record the session and have multiple observers in the session. Skype is a well-known software and some participants may feel more confident downloading it on their devices. We already use Skype for business in our department, but access has been restricted, which means we cannot dial out using Skype video to anyone outside our organisation nor share screens. Therefore, unfortunately it is not an option we can consider.  Cost: Office 365 Business Premium-£9.40/?month
8. Lookback.io – I haven’t personally tested this piece of software as it is blocked by our organisation. However, a colleague demo’ed this software to me, and it is particularly useful if you are a solo researcher and note taker, as this timestamps your notes against the recording. 

So, what did we choose?

For our team, it seems GoToMeeting (Pro) is the best option for now. Having said that, the free version of Appear.in is a really user-friendly tool to share screen and audio, and could be a good solution for remote Show and Tells. Furthermore, unlike the free version of Zoom, it does not have a 40-minute limit on group meetings and does not require the user to download a plugin or app, which I am hoping means will not be blocked by users’ firewalls. I know some colleagues have tried Google Hangout for remote show and tells, but that requires creating a Google account, which may not suit all participants.

If you do decide to use GoToMeeting, there is a moderator template guide which you may find useful to adapt for your sessions. There is also a useful guide on conducting remote usability testing in the Gov UK Service Manual.

Update: Zoom now has a new automated transcribing function for cloud recordings, which is a huge bonus. We may consider trying that for a while.

So, which piece of software have you used to conduct remote usability testing, and how has your experience been?