Does your questionnaire pass the toddler test?
The Health Visitor took out a picture of a jungle and asked me “Can you name any of the animals in this picture?”
I answered…. “yes”.
Cue silence, and a smirking mother.
That wasn’t what the Health Visitor expected me to say (rather, I should have answered “lion, tiger, giraffe…”) but it was a perfectly reasonable and correct answer to the question that she asked.
Ask a closed question, get a closed answer.
It’s like I was destined to be a market researcher.
And yes, I am still that literal. It is really quite annoying for my husband.
This is very relevant in qualitative research, because in a focus group or depth interview you want to get people talking. You don’t want to give people the excuse to give one word answers! But, in a conversational setting it is possible to follow up an unintended closed question with an open question to explore the detail. I’m sure the Health Visitor did that all those years ago to get me to name some animals, probably asking “Which ones?” or “What is this one?” or similar. Not ideal – and as an experienced qualitative researcher you learn to speak in open questions – but an OK contingency plan.
It isn’t so easy with a questionnaire. In a questionnaire you put the questions out there and get the answers back, and that’s the extent to the dialogue. So you’d better hope you got the question right the first time!
If I wanted to find out which animals 100 respondents could name from a picture and I asked…
Q Can you name any of the animals in this picture? (Please write in)
… and 100 people wrote in “yes”, I’d be pretty disappointed.
But what if I worked for a restaurant and I wanted to know who my competitors were, so I asked 100 respondents…
Q Can you name any restaurants in Edinburgh? (Please write in)
… and 100 people wrote in “yes”. Again, I’d be pretty disappointed.
In research, the strength of the findings relies on the strength of your questions so it is essential that every word in every question is precisely tailored to ensure that you are asking what you mean to ask. No room for mistakes or confusion or ambiguity.
You shouldn’t be patronising when you are writing a questionnaire… but… think of the three year old me. Does each question pass the toddler test?
Rubbish questions lead to rubbish answers which lead to rubbish findings. And that is just a waste of everyone’s time.