I have recently written a guest blog for Salient Point called Will people buy my product? so please do stop by to read my article and browse the rest of the posts which I think anyone who works in a business will find interesting.

So I’d like to explain why I can.

First a diversion into the way that the market research industry is set up.  As is the case with many industries, market research is not a single job function in itself but a collection of job functions under a wider umbrella. 

Job functions in market research include:

• Methodology (planning programmes of research, designing questionnaires and discussion guides, ‘sampling’ who to interview, setting quotas for who to interview)
• Scripting (setting up questionnaires into formats appropriate for web, phone or in-person, testing scripts)
• Field (recruitment for focus groups, telephone or in-person interviewing, supervising teams of interviewers locally, booking and managing whole surveys UK-wide, ensuring quotas are met)
• Qualitative fieldwork (depth interviewing, moderating focus groups)
• Data entry (entering questionnaires into analysis programmes, transcription of interviews)
• Data analysis (cross-tabulating data, statistical analysis, ‘coding’ responses into categories)
• Insight (interpreting data, reporting, recommendations)
• Client management (project level and account level)
• Plus all of the usual admin, finance and HR functions

Each of these areas has a variety of levels of seniority within it, posts from semi-skilled through graduate level, and its own separate career path where you can start at the bottom and work up.

The big market research agencies have departments dedicated to each of these elements, and if you gave them a contract to undertake research on your behalf your project would usually be co-ordinated by one or two research consultants.  This is the function that I have played (again, with increasing levels of seniority) throughout my career.

The role of the research consultant is to plan and manage the whole project, ensure client aims and objectives are met, and make the whole thing appear seamless.  They are the ones that you will brief, that will work with you to plan a methodology, that will update you on progress, and that will interpret the findings and present back to you.    

The other elements of the project are essentially sub-contracted internally to appropriate experts under each of the job functions outlined above.  Any given research project is likely to involve at least one (and probably more) person from each of the job functions, and the research consultant selects these from their global colleagues, briefs them, and negotiates around who needs to do what and when to meet the client requirements.

Although I am an independent researcher, I can still provide a full service because I can arrange to sub-contract any of these elements of the process in exactly the same way that I did when I worked at a global research agency.  As the client you get exactly the same service as you would if I worked at a research agency, because I play the negotiating and co-ordinating role, and I am your key point of contact for the project.  I’m the same person, with the same skills and experience, doing the same job in the same way that I always did, but I can pick and choose the partners that I work with to ensure that they are fit for purpose and competitively priced. 

So let me know if you’d like the full service!

I just wanted to let you know that the article was syndicated by Research Live (the website for Research Magazine which is the official publication of the Market Research Society) and the Huffington Post which was very exciting indeed!

Do you agree that Scotland should be an independent country?  (Yes/No)

I’ll go out on a limb here and say it’s not the worst.  And I thought it would be worst.  But it’s not great either.

Lets start with a good point.  Alex Salmond has said the question is ”short, straightforward and clear” and he’s right, it is, I think anybody would be able to understand it.  As we are talking about a referendum, it is necessary to plan for every adult in the country to be able to read and understand the question and I think that has been achieved.  The question seems to comply with the following element of the Market Research Society code of conduct (our professional quality standard):

B.14 Members must take reasonable steps to ensure that the design and content of the data collection process or instrument is appropriate for the audience being researched.

However, and most importantly to me, as a member of the Market Research Society I wouldn’t write this question myself and use it in a questionnaire.  The main reason is that it is not a balanced question.  The MRS code of conduct states that:

B.14 Members must take reasonable steps to ensure that Respondents are not led towards a particular point of view.

The question in no way meets this quality standard.  Because it asks us whether we agree that Scotland should be an independent country, we are led towards the agree response.  The way that the question is framed assumes a particular value position.  It is saying ‘Do you agree (with us) that Scotland should be an independent country?  Eliminating bias is questionnaire design 101, and not doing so is really not acceptable.  Questions should be written to be balanced by assuming no value position, for example:

• Do you agree or disagree that Scotland should be an independent country?
• Should Scotland be an independent country?

But lets consider the content of the question and intended use of the results.  Thinking about what the SNP want to achieve this is a good question for the SNP to ask, in that it is probably the best way for them to drum up a bit of national pride.  They ask us whether Scotland should be an independent country.  Of course many will simply agree, but at the margins this wording has the potential to elicit an idealistic agree response from anyone with any leaning towards patriotism (and there are plenty of them in Scotland) particularly when the ballot form is tarted up with ethnic imagery such as saltires (sorry, I mean the Scottish Government logo).  We are not asked to judge when independence should happen, or whether it will work.  We are just asked to say if, in an ideal world, it should be the case.  And that’s a legitimate question. 

There’s a caveat to that though.  We all know that a ‘yes’ vote will lead to independence, so those in Scotland who do agree that Scotland should be an independent country but don’t agree that it will work in practice will be forced to lie with a no vote.  So given the potential use of the result it is unclear whether the question meets the following MRS quality standard:

B.14 Members must take reasonable steps to ensure that Respondents are able to provide information in a way that reflects the view they want to express

Linked to this, when the SNP get the results they will take the percentage yes vote and if it is high enough they will use it to start the process of independence.  But is that appropriate based on the question?  Not really.  The answer to this question is not hugely practical, it just tells you what the nation thinks might be good if we lived in a vacuum.  Don’t forget this is only one of a big pile of subtly different questions that the SNP could have asked.   I bet if they asked Should Scotland become an independent nation next year? or Could Scotland function as an independent nation? they’d get an entirely different response.  It is therefore unclear whether the question meets the following MRS quality standard:

B.14 Members must take reasonable steps to ensure that responses are capable of being interpreted in an unambiguous way

By the way, this is the original referendum question that the SNP were proposing in 2007.  Much more practical and balanced.  Interesting that five years on they feel the need to go the other way…

The Scottish Parliament has decided to consult people in Scotland on the Scottish Government’s proposal to negotiate with the Government of the United Kingdom to achieve independence for Scotland:

Put a cross (X) in the appropriate box

I AGREE that the Scottish Government should negotiate a settlement with
the Government of the United Kingdom so that Scotland becomes an
independent state.

OR

I DO NOT AGREE that the Scottish Government should negotiate a
settlement with the Government of the United Kingdom so that Scotland
becomes an independent state.

So is the current question the right question?  Ach, well that depends on what you are trying to achieve.  Would I have asked it that way as a professional researcher?  No.  Is the proposed approach ethical?  Questionable.  But if you are trying to drum up support for Scottish independence it is pretty well framed, so good on the SNP for being a bit sneaky there - I have a certain respect for their cunning.  And if the no vote wins, that implies a no to all the other more ethical less idealistic questions too.  So this is them giving it their best shot…

Or is it?  I absolutely loved this alternative referendum question that I sourced via Facebook.  Much more appropriate than the official version!

9.30am – Wake up to admin

I usually come to my desk in the spare room around 9.30am, having got up at… erm… well 9.30am.  I never pretended to be a morning person, and working for myself (and usually from home) means I am rarely under any pressure to get up too early.  I spend the first half hour doing my admin – reading and responding to emails, having a quick look at Twitter and LinkedIn, checking my bank account against any expected invoicing, and sorting out any equipment I might need for the day.  Today I have a depth interview at 10am so I needed to ensure I had a (working) digital recorder to hand and a print out of my interview schedule in advance of that.

10 to 10.45am – Depth interview

I have had a depth interview scheduled in to undertake at 10am, which was very kindly set up for me several weeks ago and so was a fixed point in my diary.  This project involves me and some associates conducting 30 interviews by telephone with fairly senior people around Scotland.  This is my fourth, and I have another scheduled in for next week and some more as yet unscheduled.  The interviews are semi-structured meaning that at the start of the project I designed an interview schedule which is a list of questions to cover during the interviews.  Preparing an interview schedule is not as straightforward as it looks, as it involves taking all of your client’s random thoughts and putting them into an order which will appear seamless and logical to the respondent.  The idea is that the sessions feel fluid and natural to the respondents and are flexible enough to explore new issues as they arise, but because we have a list of questions there will be a certain amount of consistency across the 30 interviews and all of the client’s key needs will be addressed.  There is a certain skill to conducting depth interviews, these being conversations in which you do not actually take part – glossing over the fact that you have a list of questions, and not joining in with your own opinions.  In fact, the less you say the better.  If you plan it properly, and with a bit of experience, the respondent will never notice that it isn’t a ‘real’ conversation. The actual interview today took 38 minutes (which is typical for this project) but there is always a bit of faff at the start (speaking to secretary and getting put through, introducing the project and getting consent to record the session) and more at the end (answering questions about when the report will come out, politely closing the conversation).

10.45am to 12 noon – Hate listening to myself on tape

This project requires me to submit an overview of each interview I conduct to my client, and from these my client will collate all the themes into a report for his client.  That’s unusual, I usually do the whole thing myself, but I this case I am being employed as an expert adviser and interviewer rather than a full service consultant.  It is much quicker and easier to write an overview such as this if it is done by the person who conducted the interview, immediately after it was conducted.  Preparing the overview involves downloading the recording onto my laptop and listening to the interview and pulling out key themes and verbatim comments under headings of each of the client’s objectives.  This takes longer than the original interview because there is a lot of starting and stopping of the recording involved, and interviews tend to roam about a bit so notes need to be reordered thematically rather than reported chronologically. This respondent also used quite a few acronyms that I had to look up on the internet.

12 noon to 12.20pm – New Business Development

Working for yourself you always have to have your eye on the next thing, and I have had a list of people that I wanted to catch up with and chat about future opportunities.  No point doing so in December as people are busy and less focused, so I’ve saved them up for January.  Spent a bit of time today firing out emails to all the relevant people to set up lunch or coffee meetings later in the month.

12.20pm to 12.40pm – More New Business Development

I also keep my eye on opportunities to competitively tender, so that I can put in proposals for projects where my skills are relevant.  I check the internet for opportunities every few days as there tends to be a fairly quick turnaround of clients wanting proposals back in a week or two so it is good to find them early.  I have a favourited list of sites that tend to feature relevant ‘invitations to tender’, and I also have some standard advanced search terms that I use on Google.  When something comes up, I download the documents and consider whether I think it is worth the effort of putting a proposal in.  Is it interesting?  Do I have the right skills and experience?  Is the client asking for something reasonable within the level of funding offered?  There was quite a few maybes today (but no definite yeses), so I put them aside to look at another day.

12.40pm to 1pm – Non-research work

I have my finger in a few pies and I am corresponding with a branding expert who is working on logos and straplines and so on for one of my other enterprises.  He’s sent some thoughts through, so I considered his ideas and sent back my response by email.

1pm to 2pm – Break for lunch

I Sky+ myself a lot of social commentary style documentaries and I like to watch these while I have my lunch.  Working as a researcher and particularly working in the voluntary sector I think it is important to expose myself to all of the different lifestyles that the world has to offer.  I think it makes me a better researcher to know what’s out there – both strategically (when designing research) and operationally (treating people respectfully).

2pm to 4pm – Work on a proposal

Just before Christmas I identified a tender that I would like to respond to, that was due on 6^th Jan (tomorrow).  I therefore planned to prepare it this week, and had already done a lot of it yesterday.  This included designing the methodology and budget by dividing the project up into tasks, estimating how long each would take, adding these up against a daily rate, and messing about with it until it came in under budget. This proposal requires me to write 1,000 words for the approach and methodology (and I sneaked in two pages of appendices).  That’s not a lot at all.  It is much easier to write a 30 page document so cutting it down was the challenging bit.  Finished it up, and will leave it until tomorrow for a proof read.  I like to put in a bit of distance between finishing up a document and giving it a last look over.  You can get too close to something you have been writing for days and this makes you miss important points.

4pm to 4.15pm – Update client

I have been working on a project in December and it is due to complete at the end of January.  My client is fairly hands off, but I wanted to make sure he was updated and informed so I prepared and sent a quick email detailing progress and plans against what we had originally agreed.

4.15pm to 5.30pm – Work on report

As I mentioned earlier I like to put a bit of distance between myself and a document I had prepared, and final reports are another example of this.  I have been working on a project for a year now, collecting lots and lots of data and writing it up – and the report is due in January.  I aimed to (and achieved) drafting the 60 page report before Christmas, so I looked at it again today for a final edit.  As some changes were required, I will look at it again tomorrow and see if I’m ready to submit it.

So that’s my day.  A reasonable representation of my working life, although some of the other things I might do include phoning or meeting with clients, designing research tools and methodologies, planning and moderating focus groups, data analysis and networking.  Amongst other things.

Today I finished around 5.30pm which you will note means I did a standard working day.  Although it is quite typical that I worked for seven hours it isn’t usually so clear cut.  Today I’m going out to the pub at 6pm so I needed to fit the working day in before then.  Usually I might go on until 7pm or so having got up later, gone out for a jog, taken a long lunch, or gone out on personal errands.  Some days I will have meetings outside the office, and I might go for a swim or to the shops on my way home and work on a bit later.  Unlike most people I seem to work better in the evenings and will happily work through til 8pm if I don’t have anything better to do.  Self-employment means you can work in a way that suits you, and I like that a lot.

A person who is writing a survey would generally begin with a list of themes to be covered, flesh the themes out into rough questions, and then tighten up the rough questions ensure they make sense. 

But questionnaire design should not stop here!   A person who is writing a questionnaire should think about the respondent at all times and go the extra mile to make their questionnaire a pleasant experience for the respondent.

Do not underestimate the power of writing a respondent-friendly questionnaire.  If the respondent is happy this will achieve:

• Less confused or irritated respondents.
• Fewer drop-outs / higher response rates.
• Higher quality data (based on more responses and coherent thought processes).
• More meaningful results / decisions based on results.
• Good feeling towards the sender of the questionnaire (which could be your company!)
• Good feeling towards future questionnaires / the research industry.

So how can a researcher go about writing a respondent-friendly questionnaire?

Firstly, get the questionnaire order right by thinking about the ‘story’ you are trying to outline.  If you want to find out about a sequence of events your respondent went through (such as a transaction or purchasing decision), a good way to present it is to re-take them through the experience chronologically.  This means the respondent will recall everything in a logical order and therefore give more coherent responses.  Ask respondents about behaviour before attitude:  it is easier for people to answer questions about what they did (i.e. what did you have for breakfast today?) rather than what they think (i.e. what are the relative merits of toast as compared to eggs?).  Asking easy questions first warms respondents up to the harder parts of the questionnaire meaning that they give fuller and more considered responses.

Secondly make your questionnaire look nice and welcoming rather than scary and intimidating.  Use an accessible font and colour scheme along with your branding and logo, and start with providing a realistic estimate of how long the questionnaire will take to complete.

On a hard copy survey:

• Keep the survey down to a page or two if possible.
• Do not allow questions to run over a page break.
• Make sure there is plenty of white space on a page.
• Keeping routing (i.e. Skip to Q4) clear and easy to follow.

On a web survey:

• Put one or two questions on each page (rather than all of them on one page) so that no scrolling down is required. 
• Include a status bar so respondents can see how much of the survey is left to complete.

Thirdly, be courteous – include a pleasant and welcoming intro at the start of the survey explaining who you are and what you are doing, and at the end of the survey thank the respondent for their time.  Be polite, and let them know you appreciate their input. 

In previous blog posts I have written about the importance of thinking about what you want to get out of your survey, but this shouldn’t be at the expense of being human.

As I’ve said in a previous blog post, “Researchers collect demographic information (population characteristics) from respondents to check that their sample is representative of a given population, and to segment the findings to see whether different groups of people react differently to questions than others.  This means that any demographic information that you give a researcher will be analysed along with 50+ others (usually many more!) and will not be looked at separately.  Refusing to provide demographic information limits the value of a person’s input into a survey because their responses cannot be segmented.”

Although it sounds a bit controversial when you write a whole blog post about it, social class is just one of many demographic segments (along with age / gender / ethnicity / whatever) that a researcher might look at to identify useful and interesting patterns in the data.  

When you’re doing a survey you can’t just ask someone what their class is though, or say ‘How posh would you say you are on a scale of 1 to 10?’.  You’d get a very subjective response and when you’re dealing with hundreds of respondents you need a system that is as consistent as possible in order to be able to get a useful result.  So back in the day the National Readership Survey invented the SEG system based on occupation and it is still widely used as a practical proxy for class / wealth / poshness / lifestyle.

So how does it work?  In total there are six SEGs – A, B, C1, C2, D and E.  These are defined as follows (or see this link for more detail):

• A - Higher managerial, administrative, professional e.g. Chief executive, senior civil servant, surgeon
• B – Intermediate managerial, administrative, professional e.g. bank manager, teacher
• C1- Supervisory, clerical, junior managerial e.g. shop floor supervisor, bank clerk, sales person
• C2 – Skilled manual workers e.g. electrician, carpenter
• D - Semi-skilled and unskilled manual workers e.g. assembly line worker, refuse collector, messenger
• E - Casual labourers, pensioners, unemployed e.g. pensioners without private pensions and anyone living on basic benefits

Researchers find out which of these you are by asking you a complicated series of questions at the end of a questionnaire, based on the occupation of the main income earner in your household. The questions usually go something like this:

What is the occupation of the main income earner in your household?  

What is the industry / type of company that they work in?

What is their position / rank / grade in the organisation?

What qualifications do they have? (i.e. degree / apprenticeship)

How many staff are they responsible for?

NOTE TO INTERVIEWER: CIRCLE SOCIAL GRADE  – A B C1 C2 D E

This can usually only be done in a face-to-face or telephone interview to get the correct level of detail and as it is a bit of a judgement call, and interviewers are trained in covertly allocating SEG based on your answers.  The interviewer has to ask a lot of questions about the occupation of the main income earner because some job titles can be very vague and span several SEGs.  For example:

• A Consultant could be a Doctor, a business advisor, or a mobile phone salesperson.
• An Engineer could be chartered, skilled or unskilled. 
• A Chief Executive could be the boss of a multi-national organisation, the boss of a local charity, or a sole trader.

So that interviewers can get the SEG right there is even a book that interviewers can buy called ‘Occupation Groupings: A Job Dictionary’ which lists pretty much every job title with a SEG against it.

So what do we do when we have allocated you to a SEG?  Well your SEG will be tacked on to the end of the data you have provided, meaning that it can be used to see whether people in different SEGs answered the questions differently.  This helps us to interpret the findings and make recommendations about what our client should do next.   Researchers don’t look at individuals and questionnaires though, they look at data tables and sub-groups containing many tens or hundreds of respondents.  They don’t even necessarily look at each SEG individually.  When doing the analysis researchers might group SEGs in a few different ways, for example:

• AB, C1, C2, DE (generally ends up a fairly even split across the population)
• ABC1, C2DE (allows you to look at the responses of the well off against the not so well off)
• BC1C2 (allows you to look at the responses of ‘ordinary’ people)

As a result we researchers can answer important questions such as who is more likely to buy a product or who is more worried about an issue.  We don’t have any interest in looking at your questionnaire and saying “Oh look, Ruth Stevenson from Edinburgh is a Social Grade B”, but instead use the information to build up a picture of a target market so we can tell our client something practical like “Aim your advertising at young BC1 Males in urban areas”.

For example a questionnaire could be a feedback form at an event you attended, a survey to gauge awareness of an advertising campaign, an application for a credit card, a phone call from someone selling you double glazing, an exam, or an assessment for taking your medical history.  Questionnaires can assist you in offering your opinion to improve things in your life and allow you to gain access to products and services.  However if you ‘fail’ some kinds of questionnaires the questionnaire can act as a barrier to accessing something you want or need, such as insurance, healthcare, or entrance to a job or university.

Recently an article entitled Warning: questionnaires may damage your health caught my eye.  A very interesting article about the experience of reassessment for state benefits from a person with mental health problems.

I love questionnaires, I really do.  But then I like to think I am a champion for good and ethical research.  Unfortunately sometimes questionnaires are used as inappropriate decision-making tools in inappropriate circumstances. In the case of the linked article, the questionnaire is a scapegoat for the real issue of benefits reassessment but (as an expert in research methodologies and a previous exployee of a mental health charity) I am happy to concur that I would not consider a questionnaire an appropriate assessment tool in these circumstances.

For most people a questionnaire is a questionnaire whether it is a happy fluffy ethical survey or whether it is a twenty page compulsory interrogation upon which the livelihood of a mentally ill person is decided. 

I’ve blogged several times before about the difference between good and bad research, and the reasons why taking part in genuine market research is (or should be) an ethical and ideally pleasant experience.

It upsets me when the tools of my trade are used in a way that causes stress and pain for people.

Those of you who subscribe to IJMR can look out for the full publication (Volume 53, issue 6).  Alternatively you can find the article reproduced on my website.

Well quite by chance I came across some information about a question asked in ye olde census which made me laugh and shudder at its sheer awfulness.     

In the 1871 Census people were asked whether anyone in their household was:

1. Deaf and dumb
2. Blind
3. Imbecile or idiot
4. Lunatic

Respondents were invited to “write the respective infirmities against the name of the afflicted person”.  This continued through several cycles of the census, and in 1911 respondents could also select ‘feeble minded’ (an important distinction?) in addition to those above.  After that, these questions on ‘infirmities’ did not continue.

Which is good, obviously, because by today’s standards much of this terminology is entirely inappropriate and sticking any of this in the census is ethically and methodologically questionable too.  Thankfully this is not the way that we think about disability anymore.  Even in 1881 the powers that be recognised that their methods were not really working, saying “how very incomplete are the returns which relate to these afflictions, and more especially those which relate to idiocy and imbecility.”  It seems people were not keen to put down on paper that a member of their household was an ‘imbecile’.  Funny that.  It didn’t stop them asking though, the question wasn’t taken out until 1921.

The Census is snapshot of a place in time and it is interesting to see how the Census represents the times as they were and the world as people saw it.  However the world changes rapidly, and research needs to change with it to keep up.  I often wonder what we do or say today that will be considered bigoted and inappropriate in the future.  I cringe for myself in advance.

But as researchers we have to do the best we can to respectfully capture the world today the way the respondents of today see it.