How to recruit (lots of) users for online user testing

Reading Time: 6 minutes

This article is about how to recruit users for online user testing.

Tube station with lots of people

A couple of people have asked me recently, how did you test so many users in your Human Centred Systems Master’s project?

Luckily, I’m in the process of writing up my paper for publication which means that I have the answers! I hope that by writing this post, I can help anyone who needs to find users for their online tests.

FAIL TO PLAN, PLAN TO …

It all starts with planning, followed by design, experimentation and then iteration – sort of like the User Centred Design process.

The first step to getting users is defining who your target audience is and then designing what you want to say to them. Ideally, the end result that is that: you’ve targeted the right people, they are interested in your message and they decide to part in your study.

For my project, I wanted users from a “first world country”, my message communicated that it was quick (only 5 minutes), it was easy (you have a video tutorial),  you get entered into a prize draw (£30 Amazon voucher) in return for taking part in my online study.

DESIGN YOUR MESSAGE

Next up is designing the message. This means structuring your message in a clear way, using familiar language and being persuasive.

After I drafted my first message, I figured that the easiest way to recruit people to do my test was to ask my friends via Facebook. I quickly found out that unless they were currently studying at University, the phrases “experimental study” or “research project” didn’t mean much to them.

I probed them about the wording and they told me that the word “study” reminded them of books and research project sounds like it’s something tedious or boring. I had to make my message more user-friendly so I experimented with different words to replace “study”. It was a bit like doing keyword research for SEO.

Eventually, I realised that people love “experiments” because they sound exciting, fun and there is a sense of mystery to it.

After asking my friends and then acquaintances on Facebook, I looked to reddit.

EXPERIMENT AND ITERATE

I modified the message I used on Facebook to include an introduction and the monetary incentive (£30 Amazon Voucher prize draw). I headed over to the /r/london and /r/london_forhire sub-reddits and read a couple of the top posts. This let me understand what people generally discuss and what sort of language they use.

For these two sub-reddits, I decided to start with a “text post”:

Title: [Help-a-student] Short, fun 5 minute dissertation experiment and a prize draw for £30 Amazon Voucher


Hi,

I’m a Human Computer Interaction (HCI) Masters student and I go by the name Will. I’ve just finished creating the experiment for my Masters research project: www.uxstudent.co.uk/research/ and I’m really excited to share it with you.

My research is about online testing methods for websites. I hope that my research will contribute to improving digital products so that they are easier to use. It would be so helpful if you could take part in the study – I promise it will not take more than 5 minutes as there are only 3 questions.

A short instructional video that will explain how the test works and at the end of the test, you can opt in to a £30 prize draw for an Amazon voucher! Thanks so much for all your help guys/gals, I really appreciate it. Hope you have a lovely week, Will.

As you can see, I tried to make my message as short and concise as possible. I began by introducing myself and my project before moving on to explain what my project was about and how long it would take. Hopefully, for anyone scanning the text they can least see that it’s “5 minutes” and “only 3 questions”. I close by explaining that they will be guided via a video because everybody likes videos. And then of course there is a prize draw to seal the deal.

I couldn’t reveal too much information about the experiment itself so I used this to my advantage. I tried to create a sense of mystery and excitement. I wanted readers to think “What kind of experiment is this?”, “It’s only 5 minutes”, “This should be easy”.

After a day and a half, I managed to read all of the comments left by redditors, respond to them and improve my message even more.

Next up was the /r/beermoney (a sub where people do online tasks for money) and /r/usability sub-reddit.

Title: Spend 5 minutes doing my experiment with 3 questions and enter a prize draw for £30 ($47) Amazon gift vouchers /r/beermoney

Title: [Help-a-student] Short, fun 5 minute dissertation experiment and a prize draw for £30 Amazon Voucher for /r/usability


Hi,

I’m a Human Computer Interaction (HCI) Masters student and I go by the name Will.

I’ve just finished creating the experiment for my Masters research project: www.uxstudent.co.uk/research/ and I’m really excited to share it with you.

My research is about online testing methods for websites. I hope that my research will contribute to improving digital products so that they are easier to use.

It would be so helpful if you could take part in the study – I promise it will not take more than 5 minutes as there are only 3 questions.

A short instructional video that will explain how the test works and at the end of the test, you can opt in to a £30 prize draw for an Amazon voucher!

Thanks so much for all your help guys/gals, I really appreciate it.

Hope you have a lovely week, Will.

The second message has been refined from my first message so that its even easier to scan. I took out the paragraphs, made the sentences shorter and used more line breaks. Note how I catered the titles of my post so that I complied with the rules of the sub-reddit and made the message more enticing.

After doing these “text posts”, I experimented with doing just “link posts”. For these posts, reddit users (redditors) only see the title of my post and then decide whether or not they wanted to click through.

I tried various titles and tried to optimise for the types of users in each subreddit:

HCI/UX Dissertation Experiment: Prize draw for £30 Amazon Voucher, short & fun 5 minute study (Please help and take part) /r/webdesign

UX Dissertation Experiment: £30 Amazon Voucher, short, fun 5 minute dissertation experiment. Please help out. /r/glasgow

MSC Dissertation Experiment: £30 Amazon Voucher, short, fun 5 minute dissertation experiment. /r/manchester

[Help-a-student] Short, fun 5 minute UX/Marketing dissertation experiment and a prize draw for £30 Amazon Voucher /r/web_marketing /r/UCL /r/psychology/

HCI/UX Dissertation Experiment: Prize draw for £30 Amazon Voucher, short & fun 5 minute study (Please help and take part) /r/userexperience /r/SEO

Web Design MSC Dissertation Experiment: £30 Amazon Vouchers, short & fun 5 minute experiment. (Please help take part!) /r/brum /r/design_critiques /r/webdesign

Fun Dissertation Experiment for you to try. It only takes 5 mins and has 3 questions + prize draw for £30/$47 Amazon Voucher. /r/australia /r/newzealand /r/wales /r/ireland

[TASK] Spend 5 minutes doing my experiment with 3 questions and enter a prize draw for £30 ($47) Amazon gift vouchers /r/slavelabour

[Academic]5 minute test about investigating online testing of websites (Westerners) /r/samplesize

Your success have depends on how much experimentation and iteration you do. Every day I would read the comments the redditors left and see their reactions. Not only did this help me optimise the titles/copy for my next post but it helped inform me about people’s reactions to my study. Sometimes, redditors would even suggest useful information which then ended up in the “Other findings” section of my report.

I tried to advertise on other social channels as well but this was not as successful. I began with writing a short article on Medium to explain what I’m experimenting with and why in an attempt to persuade readers to help me out.

I pinned the imagery on the article to Pinterest and then tweeted the articled to #ux, #design and #dissertations. Looking back at it, if I had persisted with these channels I might have had more success but I was too busy reading and responding to people in forums and on reddit!

RESULTS

Results in a pie chart/

 

My online test was open for roughly ten days and over this period a total of 880 people took part in the tests. Of these, 54 participants reported that they had either taken the test before or were under the age of 18 and were therefore omitted. From the remaining 826 participants, 366 were from the “US, Canada or a Third World Country” so their responses had to be omitted.

Finally, it was found that 114 participants did not fully complete the tests so I was left with the data from 346 users to use for analysis. The lesson here was clear. For online testing, you need to aim for a high quantity of data because quality is going to be low and you are going to have to remove some of the data collected.

What I have described here might be stuff that only worked for me but I hope that someone might be able to get some ideas from some of the things that worked for me, and perhaps adapt them to their own circumstances.

If you have any questions at all, please do ask. I would be delighted to do whatever I can to help someone else succeed in their UX related studies.

PS: If you know of any other places to recruit users for online user testing, please email me – I am will@uxstudent.co.uk.

OTHER SITES I USED

http://www.postgraduateforum.com/
https://www.findaphd.com/
http://www.thestudentroom.co.uk/
http://www.mumsnet.com/

Thank you to Nisha Kotecha for helping to proofread my work.

How to recruit (lots of) users for online user testing