Don't ask stupid questions
Why trust is the key ingredient to having quality user feedback.
“Everybody lies” is a New York Times bestseller, proposing that social science approaches are basically worthless. Some arguments may even strengthen the theory by author Seth Stephens-Davidowitz. For example, people won’t always tell the truth because the answer contradicts social desirability.
Journalism is no stranger to this behaviour. Surveys often conclude that the audience wants longer, in-depth reporting while data shows otherwise. Of course, you’ve to read those surveys always with a grain of salt.
However, is data the “truth serum” Stephens-Davidowitz claims it to be? When it comes to design improvements, it’s probably a viable solution to observe how people use a product rather than to ask actively to avoid a distorted image. This process results in constant iterations of improving and testing. That’s a perfectly fine use of data.
Now, I see two significant pitfalls with the sole reliance on data. First and most obvious: Data is also subject to confirmation bias as soon as it gets analyzed by someone.