Keep your open ended questions for special occasions
Back in the day when I worked at a research agency, we used to include a maximum of two open ended (free text / write in) questions in each questionnaire. And possibly a small handful of ‘other specify’ options as well if absolutely necessary. But not many, anyway, we deliberately rationed them.
One thing I have noticed of late is that when organisations write their own questionnaires they put in many many many open ended questions.
Now there are good reasons to use open ended questions in a questionnaire, and these include:
- Wanting to elicit a spontaneous response in the own words of the respondent
- Wanting to collect quotes that can later be used verbatim
- Wanting the respondent to feel that they have had their say
However, I can assure you that this can be achieved in two questions out of every thirty. Pretty much all concepts can be addressed via closed questions (i.e. tick box style) if you want to – and if substantially more detail is required then a qualitative methodology such as a focus group or depth interview should be used instead.
But a really good reason to minimise the number of open ended questions that you use is efficiency in the analysis process. As I have said before, the holistic method of considering your questionnaire/analysis/reporting together is something that non research specialists generally do not think about when they are writing their questionnaires.
If you collect data you really ought to do something with it, but imagine just how much information that could be.
What if you asked ten open ended questions and respondents wrote two paragraphs for each? That’s twenty paragraphs per respondent. What if you got 200 responses to your survey? That’s 4,000 paragraphs.
Someone has to read them, and pick out the interesting stuff, and maybe code them up into themes or categories, and maybe convert these themes back into percentages as if they had been closed questions in the first place. Maybe someone even has to type it all into a spreadsheet if you did a pen-and-paper survey. It takes a lot of time, and this may be time that you could be better using for something else. And from what people tell me this task often comes as a bit of a surprise.
I’m not saying avoid open ended questions entirely, but I am saying that you can get results that are just as useful and far easier to interpret if you keep your open ended questions for special occasions.