Skip to content

Sugging, frugging, and other ways to write a bad questionnaire

April 27, 2011

Every now and again I get solicited or unsolicited correspondence from organisations using a questionnaire-style format like this:

Dear Cat lover

Do you like innocent baby kittens that are all fluffy and cute?  (Yes / No)

Did you know that every day innocent baby kittens are needlessly slaughtered by evil villains?  (Yes / No)

Do you think it is acceptable that that every day innocent baby kittens are needlessly slaughtered by evil villains?  (Yes / No)

Would you like to stop the needless slaughter of innocent baby kittens?  (Yes / No)

Did you know that ‘Charity X’ works tirelessly to stop the needless slaughter of innocent baby kittens?  (Yes / No)

Do you support the work that ‘Charity X’ does tirelessly to stop the needless slaughter of innocent baby kittens?  (Yes / No)

Did you know that if ‘Charity X’ does not receive enough donations, more innocent baby kittens will be slaughtered?  (Yes / No)

Can we have just £10 to support our essential work?    (Yes / No)

→ Note: Entirely made up by me, not based on actual correspondence

I exaggerate.  But only a little.  I’m sure you’ve all experienced something like this:

Do you have five minutes to help us with a survey about products in your home?

How long have you lived in your home?

When is your home insurance due for renewal?

When can we call you to tell you about our home insurance products?

Note: Entirely made up by me, not based on actual correspondence

Neither of these questionnaires is designed to genuinely try and find something out, they are simply a vehicle to lead you to make a decision you might not have made in other circumstances.  I consider this to be deceptive and dishonest: it is not genuine research because genuine research would look to measure your views rather than change your views.  Genuine research would never look to ‘sell’ you a product or concept.  This is pseudo research known as frugging / sugging (fundraising under the guise of research / selling under the guise of research).

So what does my industry have to say about this?

Sugging is by definition a deceitful activity. It’s damaging to the interests of market researchers and ultimately consumers. It hits response rates and costs, and it makes people doubt the legitimacy of market research. The MRS has tried many times to put a stop to this odious practice and will continue to argue for action to be taken against it.

 Market Research Society chairman, Eamonn Santr, 2005

Sugging and frugging are not the only ways to ‘use’ the questionnaire format.  Here is a ‘survey’ that I received through my door the other week.

Do you agree with the SNP that council tax should be kept frozen to protect families, pensioners and others who are less well off in our society?  (Yes / No)

Do you agree that the Tory/Lib Dem public service cuts are too deep and quick threatening a double dip recession?  (Yes / No)

Do you agree with the SNP that the Tram project has been a complete waste of money?  (Yes / No)

SNP, Edinburgh, 2011

This is not as overt as the kitten murdering example above, but still biased, emotive, and designed to scare/shame you to agree with SNP policies. 

As an experienced researcher it would be easy for me to write a biased, emotive, and leading questionnaire to elicit the kind of responses that would make my clients very happy.  But the point I would like to make is that I wouldn’t do it, and in my years working with market research agencies I have never met anyone else who would do it.  Quite the opposite, I have seen colleagues go out of their way to maintain their research ethics and the reputation of the agency / industry.  It is better to lose a client than to compromise the integrity of the research, and where clients ‘insist’ on inappropriate lines of questioning (and they do) or misleading use of results as a PR opportunity (and they do) a professional researcher just says ‘no’. 

It does not matter who the client is: how prominent they are, how much they paid, what they plan to do with the results.  I can absolutely guarantee that any survey conducted by a reputable research agency will have gone through extensive checks and balances at every stage to ensure that the integrity of the research is upheld.  A professional research agency will ensure that any research that they conduct is credible.  Otherwise, it is worthless.  Otherwise, the research agency doesn’t have a business.

The sort of questionnaires highlighted above damage the research industry by devaluing genuine research and encouraging the view that all research is biased.  These surveys were not prepared by experts and they are pointless in research terms – you can’t do anything useful with the findings because the findings are meaningless. 

Please don’t judge my industry against suggers and fruggers.  Spare a thought for the credible and methodologically meticulous work that agencies do all the time – work that genuinely helps organisations large and small to make evidence-based decisions.

Advertisements

Comments are closed.

%d bloggers like this: