I’d rather talk to two people than test with ten

Hot take: Most unmoderated usability testing these days is pointless.

Not because we shouldn’t test with users, but because it’s become more about ‘crossing the t’s and dotting the i’s’ than learning anything useful. There are two reasons:

First, according to User Interviews’ State of Research Strategy 2025, 71% of respondents said their organisation has people who do research but aren’t dedicated researchers. Rather than build research as a function within a product org, the responsibility lies with designers and product managers who have a whole-ass other job to do, on top of conducting research.

When time is a factor, testing quickly drops down the priority list to a mere box-ticking exercise, to confirm that yes, you did show this to users before it went into build.

And in order to tick those boxes visibly, we default to questions like ‘did you like this?’ or ‘would you use this?’ - useless in practice, but they give us a nice clear stat at the end for our eager stakeholders.

Second, the tools themselves. Some do have rich filtering and screening options for participants, but trying to find your ideal user in a pool of thousands of people around the world can be incredibly tough. Case in point: I once tried to find company directors of SMEs (£1m-£100m revenue) to test a prototype with, but the only option to select was ‘company director’. Result? Sole traders and side-hustlers.

Study participation is a side hustle in its own right these days - a quick look at r/beermoney and the like will often flag the larger platforms as ways to make spare cash. These participants aren’t there to give you insight. They tell you what you want to hear, with just enough detail to get a good rating.

I saw this exact pattern when testing my application journey work for YouLend. I got some useful feedback around labelling input fields, which was great, but two of the eight results were people speed-running the entire application in under two minutes while vocalising how great and clear everything was.

Moderated testing is more of a commitment, but it’s worth it to see first-hand how users react to your product. There can still be a profit motive on their end, but it’s much harder to bullshit your way through it when there’s someone on the call asking you the questions.

I know how hard it can be to get users to speak to you when UXR isn’t your full-time job, so I’m not gonna say ‘Just do it moderated from now on’. But if you’re a designer/PM who regularly needs to test things (i.e. all of you) then try to spend some more time with them where you can. If you can’t, perhaps spend an hour this week building a shortlist of five customers who’ve agreed to a call next time.

Personally, I’d rather have a proper conversation about an idea with two people than a whole bunch of nothing from ten.