Why trust is – once again – the key ingredient to having quality user feedback.

“Everybody lies” is a New York Times bestseller, proposing that social science approaches are basically worthless. Some arguments may even strengthen the theory by author Seth Stephens-Davidowitz. For example, people won’t always tell the truth because the answer contradicts social desirability.

Journalism is no stranger to this behaviour. Surveys often conclude that the audience wants longer, in-depth reporting while data shows otherwise. Of course, you’ve to read those surveys always with a grain of salt.

However, is data the “truth serum” Stephens-Davidowitz claims it to be? When it comes to design improvements, it’s probably a viable solution to observe how people use a product rather than to ask actively to avoid a distorted image. This process results in constant iterations of improving and testing. That’s a perfectly fine use of data.

Now, I see two significant pitfalls with the sole reliance on data. First and most obvious: Data is also subject to confirmation bias as soon as it gets analyzed by someone.

Source: Farnam Street, fs.blog
Source: Farnam Street, fs.blog

Second, data may not give you a deep understanding of people’s needs because you can only observe existing products. A simple example: You don’t know if users want a search functionality by looking at the data if there isn’t a search functionality in the first place.

If we think about asking people about our products, there’s a straightforward rule: Don’t ask stupid questions.

The better you ask, the better answers you get. Also, what I especially believe: You have to make an effort. If you don’t have a good relationship with your users, they will lie to you, especially if you spread some uninspired online survey. Meet with people, talk to them in a relaxed environment, not in an interrogation-like setting. Trust is once again, the key ingredient to having qualitative results.

At best, data-driven research and qualitative interviews are combined to create some amazing products as BBC News Lab did for Gen Z.

Prototyping New Story Formats for News
Prototyping New Story Formats for News

Don't ask stupid questions