18 Feb

In search of "the right question"


I made a promise to myself years ago that I would never work in a Dilbert-like cube farm, yet here I was.

In my defense, I was there on a special mission.

I blinked under the fluorescent lights and shifted uncomfortably in a chair that had a tendency to slowly sink downwards at the slightest provocation.   I was talking to some very nice, but somewhat bored older ladies who had been designing forms for 25 years.  They had started out using light tables, and valiantly learned new technology as it came out.  Now their department was investing in some big, scary, expensive new forms business process engine .  It was clear that they were not sold on this new thing-a-majiggy.  My special mission was to figure out what they needed this product to do so that the technology could help them do their jobs, rather than hinder them.

You might think that this would be an extremely boring interview but it wasn’t.

Whenever I do product requirement interviews, I always try to get to the heart of what their job is about.  I don’t ask a lot of questions about features.  This is because people use products because it helps them get stuff done.  They care about their goals, not the product features.

I always design my interview protocol to stay focused on the underlying goals that frame their day-to-day interactions with the product.  Sometimes however, being extremely tuned in to your participant and going off-script can give you the best gold nuggets.

I talked with these ladies about how they designed the forms, the tools they used and what they thought about.  I noticed that they were intimately familiar with their forms.  The differences between a B-4627a and a FZ-231 (not their real names) was like night and day.

On impulse I asked them “Do you have a favorite form?”  And their eyes lit up.  “Oh yes!  The B8-WEN definitely.  That’s the maternity leave form.  I always know that people are happy when they are filling out those forms.”  Then they started talking about how the forms all had a different personality.  “This one is filled out by farmers.  Farmers have big hands and they are filling them out on the back of their truck.  You have to give them lots of space.  Also, you can’t just give them a block that says “address”.  You need to spell it out for them, because they might end up giving you a weird answer.”

Then it hit me.  I had let stereotypes of older female civil servants get in the way of what was now plain to see.  These women were not adverse to the change just because they didn’t like change.  They were not afraid of the technology as I had initially presumed.

Their concerns were that the technology would not let them make the kinds of customizations they felt were essential.  While I saw government forms as a way to feed the insatiable maw of faceless bureaucracy, they saw them as completely the opposite.  They saw forms as the way that ordinary citizens interact with their government.  Because there are many reasons to interact with their department, the forms had to reflect each unique purpose, and be understandable to the people using them.

Here is the key point.  Understanding the goals and even the world-views of the people using your products will help you build better products for them.  When you talk to customers, make sure that you make room in your conversation to find out about those things.

03 Feb

Start With the Question: lessons from my 2nd year physics lab


Second year physics was like getting thrown into the deep end of the pool and being told to figure out how to swim.  You weren’t given everything you need for an experiment, and a description of how to perform it.  Oh no.  That stuff was for babies (otherwise known as first year students).

In second year, you were given a lab full of equipment, and a list of questions.

You picked one of the questions and figured out how to use the equipment to help you answer that question.

You were expected to not only use the physical equipment completely at your discretion, but also to pull from your arsenal of mathematical tools (differential equations, statistics etc)  You also had to know how accurately you needed the answer because that would affect how you approached the problem.

Notice the order of that process:

  1. Start with a question
  2. Pick the right tools
  3. Get the data
  4. Answer the question with the right level of accuracy

My lab partner Marco and I decided to measure the speed of light.  Mostly because it let us play with lasers.

We set up a laser at one end of the lab, and a mirror on the other end.  We measured how long it took for the light to travel from the starting laser, across the room and back.  We needed to figure out the most accurate way to measure the distance from one end of the lab to the other.  In that process, we discovered that the floor tiles were laid out with extreme precision.  In the end, we were getting a more accurate measure of the distance through counting tiles than any other technique we could think of in that crowded, messy lab.

I learned two things from this experience.

  1. Man, whoever laid those tiles had serious OCD.
  2. Physics professors don’t appreciate having the speed of light reported in tiles per second.

But the main point is that you had to figure out which equipment was the right one to use for any particular question.  That was a skill that was actively taught in the physics curriculum.

But what does this walk down memory lane have to do with customer research?

When I started working in customer research, I was really confused by the fact that most of the time, people jumped to the method first.

  • “We need to do a survey!”
  • “We should run a focus group!”
  • “I just read an article in Harvard Business Review about <new method fleezleLips>.  Let’s do that!”
  • “We need BIG DATA!”


This would be the equivalent of someone in a physics lab saying:  “We need lasers!”  no matter what.  Even if the question was something like: “What is the humidity of the air?”   Well ok, there were some people like that in the lab, but I tried not to hang out with those guys – they were a little weird.

Market research is not as “hard” a science as physics, but that’s no excuse for sloppy thinking people.

All the market research methods out there are great methods.  They are all useful some of the time, but none of them are the best tool in the toolbox all the time.   They each have their time and their place.  The true expert researcher knows when to employ the right method for any given problem.

You need to start with the question first.

image source – Creative Commons license