The Vacuum
A while ago I remember reading a post about The “Dumbness of Crowds”, by Kathy Sierra.
It was actually quite provocative – as they usually are – and it makes for a very interesting read if you haven’t been there before. One of the most intriguing concepts it brings up is that “crowd” here is not a committee, nor a panel of experts. It is, instead, a set of individuals – individually.
Remember physics class? I remember them quite vividly. I remember that every test result I’d get to would only be true if the environment of the experiment was a perfect vacuum. The same happens with feedback gathering and decision making.
If you are looking to gather valid feedback on a problem of any sort from various people during the design stage of a product, the worst thing you can do is to gather a crowd around and try to get it all at once. As soon as someone starts talking, it’s like lighting the fuse for a bomb. In a matter of minutes the feedback fountain will have dried up, and you’ll be left with converging opinions – or maybe strong supporters of different opinions who weren’t able to settle on a certain point. And that’s what you’ll have to work on.
This kind of approach has the following drawbacks:
- What you’ll frequently get is a standardized opinion: not the solution that everyone wished for, but the one that fewer people disagreed on. While sometimes this “good enough” solution is actually what you are looking for, if you are looking for a more daring solution, you might want to keep away from getting feedback from a crowd at first.
- You’ll rarely be able to gather information about the “idea incubation” stage. During that process, every person’s mind was a different sandbox that was frequently being fed new information as soon as someone else said something that seemed useful. They would process it, and afterwards blurt out another idea, which will in turn be someone else’s idea-fodder.
- Usually, you will not be able to gather new useful feedback from these people after the first interview because they will have their minds set on a particular solution. Since the different opinions have converged (that is, the distinct feedback count is lower than the group head count), people will more frequently reason their way into “the group feedback was the optimal feedback” instead of thinking their way through it by themselves. Unless these people are as committed to achieving the best solution for your problem as you are, the effort of clearing their mindset to have another go at the same problem is an extra mile that few are willing – and able – to go, no matter how well-intentioned they are.
Though this kind of session is great if you’re on a tight deadline or reviewing a proposed solution – for which you are ready to back all the design decisions with valid arguments as well as open to question them if pertinent – the truth is that for a design phase it’s best to gather feedback individually, in the “vacuum”. This way:
- The fact that you asked each person individually will most likely get her more involved with the problem at stake. She’ll feel that you value her opinion enough to listen to her individually, and as such she’ll feel more compelled to help you out.
- Everything you’ll gather will be – for the great majority – completely uninfluenced. As such, if you gather similar feedback, it will be because it makes more sense for each person, instead of having more people settling for a suggestion without thinking their way through any other.
- You will be able to gather the whole “idea incubation” stage as feedback. The more you ask the other person to think through the ideas out loud, more insight you’ll get on what a particular statement is based on. When time comes for you to question conflicting ideas, you will have more arguments to back each of them, and whatever decision you end up taking, you’ll most likely be more informed to make it than if you had just gathered the final result of the think loop.
- Since everyone has a distinct opinion and individual arguments, it will be more easier for you to question previous assumptions with the correct people, and “milk” every single idea as much as you can.
I’ve been trying this approach for a while now, and it has been giving good results. Sure, it requires more time and effort in the beginning, but I believe that it is worth it in the long run.
