Unpicking the myths and misconceptions around user research.

Max Moir , Product Strategy and Research Director
23 May 2022

The past few years have seen the growing recognition of user centricity and the importance of user experience as a cornerstone of the development of successful digital products that increase conversion, loyalty and generate revenue. And yet, there is seemingly still a reluctance in business to commit investment into building effective user research into the creation of new propositions and the delivery of digital experiences. 

At Somo, we are often confronted with common anxieties or misconceptions around the value of user research within the product development lifecycle, so we thought we’d take a moment to unpick some of the feedback we get when recommending qualitative user research.

We don’t need to because…

I can get all the insights I need from quantitative research - Broadly speaking, user research practice is formed of two approaches: Quantitative research and Qualitative research.

Quantitative research involves gaining statistically significant insight through the measurement of activity or responses across larger numbers of participants – things like sending out surveys, conducting A/B or multivariate tests, interpreting analytics data. Quantitative research is very good at telling you the “what” of a situation, e.g. what type of devices people prefer to use when booking a holiday or how many people are exiting a journey early. 

On the other hand, qualitative research is typically engaged in answering the question “why”. For example, why do users prefer to use laptops when booking a holiday? What is making users decide not to complete a purchase? As a result, qualitative research tends to focus on fewer participants, engaged in deeper activities that examine individual reason and purpose, with methodologies such as depth interviews, moderated user testing, diary studies.

Both approaches are essential to understanding the effectiveness of a product or service as experienced by its users, however, on their own, they will only help you understand half the story. Knowing that users are exiting a purchase journey without converting tells you that there is a problem, but not why the problem exists. Conversely, one user’s opinion on the design of your checkout doesn’t prove that most users will struggle to convert.

Qualitative and quantitative approaches should be deployed together with one feeding into the other and vice versa: quantitative insight should be used to inform qualitative hypotheses – these are the problems we need to solve, what is causing them? In turn, qualitative insights should be validated by quantitative testing to verify their broader importance. In other words, while it’s good to know that you have a 10% abandonment rate, it’s far better to also understand why, so you can iterate and improve. 

I already have some user research from a different project, I can just re-use that - While the use of existing research can be an excellent jumping off point for initiating an understanding of users’ general attitudes and behaviours when beginning a project, in some cases, extreme caution is required when re-using insights – and, in many cases, it should be thrown out completely in favour of new fieldwork focussed on the specific problems we are trying to solve. Consider the following:

  1. Context: is the old research relevant to the customer or business problems you are now trying to solve?

  2. Audience: does the old research consider the same types of users or personas that you need to engage for this project?

  3. Age: How old is the research? Were experiences around markets and competition, product interaction, technology usage or general user behaviours when the prior research was undertaken the same as they are today?

If the answer to any of these questions is ‘no’, then you might pose a greater risk to the success of the new project by trying to fit square insights into round user journeys. 

We don’t have enough time…

User research will take too long / Research doesn't play well with Agile delivery - We often come across anxiety from clients that research will hold up the delivery of a final product to market while teams wait on detailed findings from lengthy field work undertaken by researchers.

Secondarily, we often come across Delivery Managers fearful that the introduction of user research will knock their highly efficient agile teams off the tracks with long-term waterfall research projects that take weeks to report, and which interrupt the velocity and rhythm of project sprints. 

Put frankly, these fears are not unfounded. User research can often be planned and executed outside of delivery teams, particularly when commissioned with external agencies, causing unnecessary delays. Likewise, research fieldwork can be poorly integrated into the sprint cadence of a project. Both outcomes cause delays and challenges in planning and refinement.

Ensuring an efficient and well-oiled framework for conducting testing is in place is essential to delivering research efficiently. At Somo, we’ve worked hard to create a rapid and proven approach that integrates research expertise into scrum teams and develop processes around fieldwork that can exist within the agile framework. Developing UX runways that allow for brief-to-report to take place within a 2 week sprint and working fast and focused to deliver insights at pace. Embedded researchers have a greater understanding of the project as a whole, can internalise user needs more effectively, and consult directly with team members to represent the voice of the user.

It’s going to cost too much…

User research is too expensive / I don’t have the budget - It’s true that research can seem costly, especially when conducting complex or lengthy pieces of fieldwork across multiple locations, but in many cases it really doesn’t need to cost the earth, or anything at all. There are plenty of imaginative ways of gathering customer insights without spending anything more than time: guerrilla testing of concepts and journeys, contextual observation of customers in-store, reaching out to get responses across social channels – all are free and invaluable in guiding product teams down the right path. The fact is that some validation is infinitely better than no validation at all and fixing an unnoticed issue ahead of time and before it comes at commercial cost saves a business money.

Moreover, the cost of research should be viewed as relative. With a typical development team costing in the region of £XX per sprint, Product owners should be informed and confident that they are taking the right decisions before committing costly resource. User research is a low-cost way to validate these decisions and reduce project risk.

I can just get my colleagues to do the research for me and save on recruitment and set- up costs - This might seem like a good idea on its face, and there can be some benefits in leveraging the professional knowledge and expertise of colleagues in identifying opportunity areas or pain points with a product or service. However, as an approach to understanding the attitudes or behaviours of users it is beset with issues.

Colleagues will be experienced in the product or journey already, meaning testing won’t reveal issues effectively. They are also likely to have their own specific agendas or ambitions for the product that are disconnected from genuine user needs. Moreover, your colleagues might simply not be at all like your typical user. The CEO of Mattel no doubt has strong opinions about his company’s pre-school products, but I doubt they align particularly well against the needs of the products’ users.

The same issue holds true across all research in fact – research with inappropriate users is likely to be worse than no research at all, as it stands a good chance of sending UX teams in the wrong direction, creating journeys that meet phantom needs.

It’s more trouble than it’s worth

User research is an academic pursuit which will not provide me with anything actionable - There might be some historical truth to this particular myth around the academic focus of research. Research is necessarily focussed around academic rigour as researchers are typically trained to ensure robustness in their approach to projects. However, at Somo we recognise that the development of digital products requires pace and flexibility to fit within agile structures alongside that rigor. As such, our approach is to ensure that projects are aligned around agreed hypotheses, developed collaboratively, in order to conduct fieldwork that is tightly focussed rather than overly broad.

Further, researchers are integrated into delivery teams and fieldwork is observed by team members responsible for the design or development of end products, meaning that collaborative analysis sessions focus on the creation of actionable insights that can be applied to user stories and feature backlogs.

Researchers don't or won't understand my product properly - Some products or industries are specialised and require being able to understand complex terminology or processes. Researchers need to be fluent in both the language of the business as well as the needs of the end user – and for that reason, when preparing fieldwork, collaboration and support from subject matter experts is often key to ensuring that nothing is lost in translation. Having worked across a plethora of projects in different verticals and specialist business functions, Research and UX teams at Somo are experienced in gaining an understanding of businesses and products in order to deliver successful product insight and – as a result – are extremely capable at quickly unpacking complexity.  

However, and more importantly, the researcher’s role is often to act as a bridge between business and end users, to connect and translate user needs and pain points to business goals and requirements in order to help user experience teams design clear and easy-to-use digital journeys. If a product is difficult to understand for a researcher, it’s almost certainly going to be challenging for users to grasp, and those challenges will emerge as pain points through testing.

User feedback will make me look bad in front of management - Constructive criticism or even negative feedback early in the development of a product might make your management team look unfavourably at your product, but just imagine what kind of reaction you’d receive launching the same product to your users live and untested after investing in months of feature development.