There was an article on the BBC news recently entitled ‘British public wrong about nearly everything, survey shows.’
I thought you might be interested in it for what it is, as it says a lot about social issues, the government, and the British public.
But it also makes an important point about research.
What this demonstrates very neatly is that research doesn’t uncover the truth, it uncovers people’s perceptions of the truth.
When you ask people a question they (usually) tell you what they believe to be the truth, the truth as they understand it. People have their own truths. They are not (usually) lying when they get something ‘wrong’, they are giving you a fair interpretation of the world as they see it. But they might be technically ‘wrong’.
Don’t get me started on the nature of truth, I took an honours course on ‘cognition and its social embedding’ and could go on ad nauseum…
This has implications for the way that research evidence is used. As you know I am an enthusiastic advocate for evidence-based decision making and the role that primary research can play in that.
But when you look at evidence you need to do so within the context that respondents are telling you ‘their truth’ and not the ‘absolute truth’.
No problem if you are asking them what they had for breakfast or whether they bought sugar last week. They probably know that, you can be confident that when you ask about certain kinds of recent behaviour you’d get reasonably accurate results, although a degree of forgetfulness and ‘don’t care’ has to be accounted for.
Did they really see an advert for NSPCC? Or was it actually an advert for Save the Children?
Did they really see something on the news about museums last week? Or was it actually something about art galleries?
People give you their account of what happened, but if you had actually followed them around and noted down what happened you may have experienced a different story. They’re not lying, they are just telling you what they think happened and the memory can play tricks.
When you’re asking about something more woolly things can get even more hazy for people.
If you get down to gauging a respondent’s understanding of a given topic… well they can explain their understanding to you, and it is true that their understanding is what they believe it to be, but it can be… well actually wrong.
Have they actually got any idea about what percentage of benefits are fraudulently claimed?
Have they actually got any idea how much Edinburgh’s tram system was budgeted to cost?
Sometimes people are pretty sure they know something, and talk about it down the pub with a great deal of confidence. But sometimes they are wrong. Why? Many reasons. Ignorance, misinformation, media bias, incoherent news, confusion, forgetfulness, Chinese whispers.
They still think they are right. And that’s interesting. That’s their truth. And it is important to understand the gap between their truth and the actual truth.
So long as you recognise that gap is there.
This government survey is a great example of that. It doesn’t say that Brits are idiots, it says that the government needs to educate Brits in certain areas. It tells us what those areas are.
For us mere mortals, well if you’re sending out a survey it is vital to remember that context is key and interpretation is key. Think about what you are really asking and what level of ‘truthfulness’ you can expect from the results.