Anecdotes are not data. Or are they?

I’m a sucker for small, offbeat museums.  I’ve been to an Accordion Museum, a Corn Art Museum, and even a Penis Museum (yes, you read that right).  It’s in Iceland, and it is awesome.

Here is a picture of me getting a reading with a phrenology machine at the Museum of Questionable Medical Devices in Minneapolis.  Phrenology, in case you don’t follow 19th century science and medicine, was a highly developed “science” that made predictions and assertions about your intellect and behavior, based on the bumps on your head.  This was considered a very legitimate and mainstream practice back in the day.    Nowadays, it has been banished to odd little museums in the Midwest where we smugly cluck to ourselves about how naïve people were a few generations ago.

Now here is the thing about Phrenology machines.  They will give you a very accurate reading about the bumps on your head.  If it’s data you are looking for, the phrenology machine is a pretty good instrument to turn to.  The problem arises when you try to draw conclusions from that data.  Turns out that determining personality traits from the bumps on your head is about as reliable as basing it on where the planets were at the time of your birth (but perhaps that’s a conversation for another time).

As we now know, in our modern wisdom, asking someone a few questions will give you a much better indication of their personality traits than feeling up their scalp.

However, when it comes to customer research, many organizations are reluctant to simply talk to their customers.  “That’s just anecdotes” they say.  “Anecdotes are not data.  We need data”.

But here is an underappreciated fact about data:  If it’s true, it’s data.

When people say “Anecdotes are not data”, what they really mean is “be wary about the conclusions you draw from anecdotes”.  This is a valid caveat, especially in fields like medicine where you are trying to find small statistical effects.  If your cousin says “My cold went away after I chewed on these pine cones”, you will need more information than that before the FDA will allow you to prescribe pine cones to cure colds.

However, anecdotes can be extremely useful, especially when you are starting out from a place of relative ignorance.   Don’t get me wrong.  I love me some data, and I use it often in my work with clients.  However, anecdotes are an underutilized technique for “getting the lay of the land”.

Let’s say you were living in bunker for a few years and you hadn’t seen the outside for quite some time.   You could send someone outside to check things out.  If they come back and say “it’s cold outside and there is white stuff on the ground” they are giving you an anecdote.  But with that story, you can start to develop a hypothesis that it is currently winter. 

Now that hypothesis could be wrong.  Maybe it’s really June and there was a freak snowstorm (I live in Canada, this happens sometimes).  Maybe there was a nuclear holocaust and the planet is plunged into permanent winter.  Nonetheless, it’s a pretty reasonable hypothesis that it is winter, which you can then start to build on by collecting more data.

Many organizations are living in a bunker when it comes to contact with the their customers.  They have only the vaguest idea of who they are, how happy they are, and what they want.   If this describes your relationship with your customers, I suggest that you consider stepping outside of your bunker and talking to a few of them.