A while ago I mentioned that an anthropologist had written a blog post in response to my blog post, and that got me reminiscing.

When I took my undergraduate degree at Edinburgh University you were required to take a number of outside courses, and I took Social Anthropology 1.  A lot of people saw Social Anthropology 1 as their ‘easy’ subject (even though at the time you needed higher grades to get in than you needed for medicine) and it was full of loud private school kids who had spent the summer digging wells in Africa and continued to wear the hat that had been woven from goats wool by an elderly blind woman to thank them for their diligence.  I loved it.  It was all about people and society and how things are the same and different across cultures around the globe.

It all came together for me by chance in the last essay of the year, which was actually one I picked from the list at random.

For the essay I was required to research the controversy/debate of the conflicting work of Margaret Mead and Derek Freeman.  Margaret Mead was a graduate student who went to Samoa to study the people, living with a Western naval family and concentrating on the subject of adolescence.  She used observation and interviews as her methodology and concluded that the Samoans sailed through adolescence and were a sexually free group of people.  Freeman, on the other hand, went across and threw himself into the culture and language, living together with the Samoan people.  Afterwards, he claimed that Mead had been hoaxed by the Samoan people and that her conclusions were consequently not valid.  He said that because she was too far removed from the culture she was unable to tell when the Samoans that she interviewed were joking with her.  I’m simplifying, of course, and the debate goes on.

This stuff was so fascinating for me.  I had learned about research methodologies before, of course, and about choosing the right methodology for the task at hand.  But what I hadn’t considered was the difference that it could make to use two alternate methodologies to look at exactly the same thing.  How you could get a completely different outcome, and what a big hoo ha doing so could cause.

It’s funny, when I am asked in job interviews (or the pub) why I got into research I usually say it is because I am nosy interested in people.  But I guess that’s not the whole story.  I first became captivated by research when I discovered the power of methodology and considered the delicate balancing act that research actually is.  I wanted to get in on that.

And Margaret Mead made me do it.

Well sometimes they pick me and sometimes they don’t.

And often when they don’t, they tell me it is because I’m a sole trader and that choosing a sole trader would be risky:

Real example 1:  “Slight risk of non-completion as sole trader.”

Real example 2:  “The only factor which scored lower was around contingency arrangements and project risk.  The panel had specific discussion around the small business model and in terms of the contractor being incapacitated the panel did have some concerns.”

For a client, the worry is that if something goes wrong with me there will be no-one there to pick up the pieces.  They worry that my personal life will intrude into my work and their project won’t get done.  They’ll be stuck with no outputs, and a mess to clear up.

Well they are right in the sense that things do happen and I do have to deal with them myself.  As a sole trader I can’t just call in sick, I can’t just forget about my responsibilities, I don’t have a colleague to take on my tasks.  I need to man up and rearrange my own appointments from my death bed, and the fallout of being away is all still there for me to deal with when I come back to my desk.

And I know, because I have had personal stuff that I’ve chosen to or been forced to fit around my work.  I’ve had a few days of sickness, I’ve had a few days when my computer was down.  Some of it has been unexpected, some of it has been quite awkward to rejig.  Some of it has been really really important to me personally but of no relevance to my working life.  I’ve been on holiday…  I’ve taken a degree…  My cat became sick, and required constant care for a time, then died…  I have had some time consuming and inconvenient medical issues to deal with…  I’ve received bad news…  This has all happened, and in the vast majority of cases my clients don’t know.  They generally don’t need to know.  They don’t need to know because I have excellent time management and prioritisation skills, I plan ahead for the unexpected, and I am 100% committed to presenting my clients with a professional service and delivering on my promises.

To start with, I only get involved with a project in the first place if I am as certain as I can be that I can complete the project on time.  I never over-promise.  If there is any doubt I don’t put the tender in, I don’t get involved in the conversation, or I decline the work.

When the project comes around, I make a plan which has plenty of contingency time built into it, particularly around risky areas.  Risky areas are large tasks, or times when I will need to get something from someone else, whether that is corresponding with members of the public or getting a sign off from a client.  If I find myself with a bit of unexpected time, I use it to get ahead.  Maybe I brainstorm some ideas, or start drafting a document, or set up a template or some charts for a report.  And I always give myself an internal deadline of completing tasks one day earlier than the official deadline.

This means it is really easy to accommodate short-term absence of up to a week or so, especially if I have a bit of notice.  If I’m sick, well yes I might have to rearrange things if I’ve got fixed points (meetings, appointments) but it would be very unusual for this to actually disrupt the critical path of the project.

If I was running uncomfortably close to a deadline (and I don’t recall this happening in two and a half years of self employment), I could work long hours or pull an all-nighter.  If I was sick I could take a meeting remotely and then go straight back to bed.  I’ve done that – I’ve pitched for work by Skype on a day that I was technically off sick with a dress over my PJs, and I won the work.  As a sole trader and a home worker it is relatively easy to do that sort of thing.  And you can always take the day off afterwards to recover.

Consequently, all Ruthless Research projects to date have been delivered on time.  Some have been delivered early, and many have been delivered on a tight turnaround.

I will concur that if I was run over by a bus or (god forbid) actually killed there would be a bit of a problem.

But even in that case, I have systems in place to ensure that the appropriate parties would be notified and that my immaculately maintained and backed up files would allow someone else to easily pick the project up and run with it.  I have expert colleagues who could do this on an Associate basis and I have made arrangements with my next of kin and a specific trusted Associate just in case.

Is this enough?  No.  Some people will never pick a sole trader.

Which is a shame, because I think the benefits that I can provide in terms of flexibility and expertise and seniority and price and a guaranteed personal service may well be worth the risk.

I have recently written a guest blog for Salient Point called There’s market research and then there’s market research so please do stop by to read the article and browse the rest of the posts.

“Whether it’s focus group frolics, sexual intrigue in the office, arousing in-depth interviews, or simply the red hot antics of the agency’s employees, you’ll be coming back and coming again (and again!) for more research romps. What’s more, the staff aren’t just misbehaving with each other: they take their client and supplier relationships seriously as well…in the bedroom, office, hotel room, car lay-by….the list goes on!”

Well I’ve read one of the short stories – Mirror Mirror – (the things I do for you!) and I can’t really go into specifics here because it is explicit stuff.  About 90% of it is sex in a viewing studio whilst a focus group is happening on the other side of the two-way mirror.  As our hero wonders at the end, “Why weren’t all focus group sessions like this?”  Indeed, mine never have been… But he also points out (correctly!) that us researchers are “professional voyeurs” so maybe Amelia Bryant has got it right and research and erotica are a match made in heaven!

Amelia Bryant is a pen name, and whoever she (?) is, she knows her market research too.  The non-sex 10% is entirely true to life so she must be in the industry.  I wonder if I’ve met her at a networking event?!

By externally influencing factors I am talking about the conditions in society that could have an impact on how people view the world and respond to questions in a survey.  And specifically in the case of surveys that run over time, I am talking about situations where the conditions in society are different each time you ask your questions and thus introduce inconsistency between waves (and screw with your results). Yes, it is a massive and woolly issue.  So I’ll give you a couple of practical examples.

• Imagine you wanted to run a survey about attitudes towards gun ownership. If you asked a set of questions in November 2012 it is likely that you would get a difference response to if you asked the same set of questions on 15^th December 2012 – the day after 20 children and 6 adults were killed by a gunman at a primary school in Connecticut.   
• Or imagine you wanted to run a survey about celebrities doing charity work. If you asked a set of questions in early 2012 it is likely that you would get a difference response to if you asked the same set of questions in early 2013 – following allegations of the abuse of more than 400 individuals, including children as young as 8, by a TV presenter who has raised an estimated £40 million for charities – Jimmy Savile, OBE.   

Occurrences such as these are likely to cause a spike in your tracking data as public opinion changes in response to the external event.  Most times, this change in attitude is transient.  It generally takes a long time to change attitudes permanently so running a survey in these circumstances can cause anomalous results.    

But simply being aware of external factors is not enough.  Also throw in the fact that any external influence could impact on your findings by making your trends go up, down or stay the same (and it does not always go the way that you expect) – and that you can’t know how they would have been without the influence happening.  Oh, and in some cases you might not even know that there has been an influence at all.  Yes, it’s complicated.  And it usually only comes up if there is a very very obvious linked public event (much like how they hold off film releases if the content would be considered tactless in the circumstances) or if you notice a spike in your data and you don’t know what has caused it.

It is also important to be aware that these nebulous external factors could creep in and artificially inflate or deflate the findings of a single wave research project too.  Or even within a single wave.  When I used to manage an omnibus survey we kept the fieldwork period for a thousand responses down to three days to minimise the chance for anything occurring in the news during the fieldwork period meaning that the later responses would come out different to the earlier responses.  Yes, we took it that seriously.

So if you are running a research project and you want to keep on top of this you need to look out for externally influencing factors, and if you spot anything going on in the world that might influence your survey you need to think about what, if anything, you are going to do about that.

If your survey is aligned but not closely linked with the external event you could consider holding off your fieldwork for a bit until things quieten down.  If the influence is likely to have caused permanent attitude change, or it is the attitude that you are tracking, well, you will need to keep this in mind when comparing your findings over time as a potential explanation for any variation in your data.  As I have said before, the most important thing is to understand and acknowledge any limitations of a survey that you run and make your conclusions accordingly.

Charities you can trust and that provide public benefit.

Trust, eh?  That’s an interesting one.  Many members of the public do indeed question the trustworthiness of charities – and rightly so.  They want to know that public money / grant money / their own money [delete as appropriate] is being well spent.

Through Ruthless Research I work with a lot of charities, and I am a big believer in the role that good quality research can play in helping the public to trust charities, and in helping charities to demonstrate their trustworthiness to the public.

How so?

Evidence.

What better way for you – the person in the street – to judge whether you should place your trust in a charity?  If you can examine some evidence and be sure that a charity is acting as you would like it to act or doing what you would like it to do, then you can receive some assurance that the charity is trustworthy.

Research (my line of work) plays a major role in this, as there are many ways in which what I do can contribute to the generation of evidence for charities.  For example:

• Planning strategy or projects
• Exploring the need or demand for charitable projects
• Understanding what their stakeholders think
• Weighing up risks
• Testing out ideas
• Evaluating the effectiveness and impact of projects
• Developing good governance

For a charity to collect this evidence is not enough to gain the public trust though.  Transparency is also important – the charity has a responsibility to make this evidence available and to tell people what they are doing, why they are doing it, and what they based their decisions upon.

The public also need to be able to judge whether the evidence at hand is worth listening to.  I play another role here, as the external expert.  As an accredited researcher who has no stake in the organisations that I work with I provide robust, ethically sound and independent research which both the charity and the wider public can be assured is robust and unbiased.

I don’t always tell charities the good news that they want to hear, but I do work closely with them to enable them to make evidence-based decisions about ways to take their practice forward based on the best evidence that they have available.  I’m glad to be doing my bit.

It has been quite a different – but exciting – one for me, gathering stories about the caring relationship to contribute to the development of a play called Couldn’t Care Less.

So, please do have a look at my guest blog for Couldn’t Care Less called The art of gathering stories and while you’re there read a bit more about this interesting new play.

I have recently written a guest blog for Salient Point called Enterprising women so please do stop by to read the article and browse the rest of the posts.

I’ve had two ‘market research’ callers recently whose first question is to ask me what age group I am part of and, when I tell them, they say ‘Oh, we already have enough people from that age group, we don’t need to talk to you now’. What are they up to?

The answer to this is ‘quotas’ but in order to talk about quotas first of all we need to talk about sampling.

Research is all about asking the right people the right questions, and the process of selecting the right people to participate in a research project is called sampling.  The idea behind sampling is that you don’t need to talk to a whole population to find out what that whole population thinks.  Which is handy, because speaking to ‘all women’ or ‘all coca cola customers’ (etc etc) would be very expensive / time consuming / impractical.

Someone wise once explained sampling to me using a soup-based analogy.  If you get a bowl of soup, add salt, and stir it up, you do not need to drink the whole bowl of soup to know whether it tastes right.  You can tell whether it is correctly seasoned by tasting a spoonful because all of the spoonfuls taste the same.

So, I’m saying that if you talk to a subset of a population you will more than likely get the same answers as if you asked everyone, as long as you are sure that the characteristics of the subset match the characteristics of the whole population.

When a researcher sets up a research project, they think about the population that they need to talk to and they find out a bit about the kind of people that make it up. In practice they might think about tens of different characteristics (gender, age, socio economic group, location, purchasing behaviour…) but I will give you a straightforward example.

If a survey was being carried out with members of a sports centre, maybe the researcher will find out that the membership totals 10,000 people of which 60% are male and 40% are female and of which 25% are aged under 30 and 75% are aged over 30.

Say the researcher decides to conduct interviews with 200 sports centre members.  They will want to try to make that sample of 200 members as similar as possible to the original 10,000.  So they will say I want 60% of the interviews to be with men, 40% with women, 25% aged under 30 75% aged over 30.  They will work out how many individual interviews need to fall into each category:

• TOTAL INTERVIEWS: 200
• Male: 120 interviews
• Female: 80 interviews
• Aged under 30: 50 interviews
• Aged over 30: 150 interviews

These are the quotas.  What this means is that as the interviews are being conducted (and that could be in person, by phone or online – or indeed for recruitment for focus groups) the characteristics of the respondents are recorded against these targets in real time.  As the quotas ‘fill up’ the researcher will be keeping an eye on things and once one is full (i.e. if 50 interviews with under 30s have been completed) then it will be closed (i.e. no more under 30s will be interviewed).  This keeps going until 200 interviews are completed and all of the quotas have been filled.

However, in practice, it can sometimes happen that you will be invited to participate in a research project but when they find out a bit about you it will turn out that you are ‘out of quota’ – that you do not fulfil the characteristics that they are looking for to ensure their sample is accurate.  In the case of this example, perhaps they have completed 195 interviews and they are just looking to interview five women aged over 30.  So they keep ringing round the list of members until they find some!

As an aside, there will be other cases where the survey will be fairly open but there will be a screener question to make sure that only certain types of person can fill it in.  You will notice this when you go to fill in a survey and get knocked back at the first question, which happens because the researcher or client only has an interest in collecting responses from people with a particular characteristic and you do not fit that characteristic.

I recently conducted a web survey with the screener question:

Q1       Approximately how many dives have you done in the last twelve months?

1)      0  [CLOSE]

2)      1-10

3)      11-20

4)      21-50

5)      50+

 

Close screen:  At this time we are looking for active scuba divers to complete the survey, but thank you for your interest.

A few people were disgruntled that they could not participate because they had not dived in the last 12 months, but the questions were only relevant to recent divers and it was important to the legitimacy of my findings that only recent divers completed it.

Unfortunately when quotas and screener questions are used some people will end up being willing but ineligible to participate, and I sincerely hope that those people would be properly thanked for their time and their interest.

Read it here.