Product Development & Management

Don’t Tell. Don’t Ask. Data Beats Opinions.

By |

The following guest post is the third in a series of four articles by innovation agitator Alberto Savoia (find the first here). Throughout his career as a serial entrepreneur and Google employee, Alberto has experienced great market successes–along with a few inevitable failures. In this series he’ll share his knowledge about why products fail and provide recommendations for beating the odds. 

________________________________________________

In my previous two articles (found here and here) I introduced The Law of Market Failure which states: “Most new products will fail in the market, even if they are competently executed,” and identified failure in premise (i.e. building the wrong product to start with) as the most common, and hardest to recover from, reason for failure. In this article, we begin to look at how we can test a new product’s premise before we invest too much in it.

All new products begin with an idea. If all you have is the idea, however, the most you can do with it is to solicit opinions based on it; and if you do that, you are likely to run into a couple of problems that may seriously derail your decision-making process:

The Lost in Translation Problem: An idea is an abstraction–and a subjective one at that; it’s something that you imagine or picture in your head. The moment you try to communicate what you see in your mind’s eye to someone else you run into a challenging translation problem–especially if your idea is new and different from anything else they’ve seen. The way you imagine the new product and its uses may be completely different from the way they imagine it.

The Prediction Problem: Even if your audience’s abstract understanding of your idea is a close match to your original intention, people are notoriously bad at predicting whether they would actually want or like something they have not yet experienced, or if and how they would actually use it.

In my book, Pretotype It, I introduced the concept of Thoughtland, a fictional place where unrealized and untested new product ideas live. In Thoughtland, every idea can be a winner or a loser–depending on whom you ask … and how you ask it. This can lead to two dangerous outcomes:

False Positive: People like your idea, they think they understand it and can see themselves using it. They give you thumbs-up and tell you to go ahead: “Go for it! If you build it, we’ll definitely buy it and use it.” Fueled by such positive feedback you proceed to implement the idea. But after you launch it, all that enthusiasm and all those plans to buy and use your product are nowhere to be seen.

False Negative: People just don’t get your idea, or it makes no sense to them. They don’t see what you see. When you talk about your vision, they think you are hallucinating: “Have random people follow you as you write 140 character blurbs? You’ve got to be kidding!” As a result, you drop any plans of implementing your idea and move on. And a few months later another company launches a very similar product to great success.

False positives can lead you to believe that your idea is immune to The Law of Market Failure, so you invest too much too soon in a new product that will eventually flop. False negatives, on the other hand, can scare you away from giving your idea a chance, and you end up prematurely scrapping the next Twitter, or Google, or Tesla.

To minimize your chances of getting false positives or negatives you need to collect something more substantial and objective than opinions–especially when the people who give you those opinions have no skin in the game. And the only way to do that is to transport your idea from Thoughtland to a more concrete environment–let’s call it Actionland.

In Thoughtland you use abstract ideas to ask hypothetical questions and collect opinions.

Thoughtland: Ideas -> Questions -> Opinions 

In Actionland you use artifacts to prompt actions and collect data.

Actionland: Artifacts -> Actions -> Data 

Let’s say, for example, that you have an idea for a computer monitor stand with sensors and motors that automatically moves and adjusts for optimum ergonomics. Let’s call it The Last Stand–as in: “You can now last longer working at your computer.”

If all you have is the idea and, perhaps, a drawing, you are stuck in Thoughtland. You can go around your office, explain the idea to your colleagues and ask them questions such as: “What do you think of this?”, “How much would you pay for it?” You can reach out to other companies and talk to HR departments: “Would you consider buying this for your staff?” “How many would you order?” That’s a lot of hypothetical questions and subjective answers (with no skin in the game)–not a lot to go on.

Of course, you might learn a few useful things in the process. For example, that most people would not use their own money on The Last Stand, but would be happy to try it–if the company pays for it. Before you make an expensive new product decision based on a flimsy combination of ideas, questions and opinions, however, remember The Law of Market Failure.

As I’ve mentioned in my first article: “In criminal law, a person is presumed innocent until proven guilty. When it comes to market law, we should presume a potential new product to be a failure–at least until we’ve collected enough objective evidence to make us believe otherwise.”

And to collect that objective evidence, you need to move from Thoughtland to Actionland.

You can do that by building a proper prototype of The Last Stand (i.e., an actual stand with sensors and motors–and all the software to run it.) But such a prototype will be costly and will take weeks/months to create.

On top of that, working prototypes are great for determining the feasibility of a new product (Can we build it? Will it work?) but are not sufficient for determining if there is a viable market for it.

Wouldn’t it be wise to objectively test and gauge the potential market for The Last Stand–at least a bit–before making a major investment in prototyping it? Are there other artifacts we use can to collect market data that are simpler, cheaper and quicker to build than a prototype? Yes, between abstract ideas and proper prototypes there are pretotypes:

Pretotype, noun: An artifact used to test hypotheses and collect data about the market appeal and actual usage of a potential new product objectively and with a minimal investment of time and money.

Pretotypes help you to “make sure that you are building The Right It before you build It right.”

In the next and final article in this series, Pretotype It, I continue my discussion of pretotypes and show you how simple pretotypes can quickly take you from Thoughtland to Actionland and be used to collect valuable market data. Please share your thoughts about decision-making in building products in the comments below.

Read the fourth and final article in the series: Pretotype It.

About Alberto Savoia

As a serial entrepreneur and an early Google employee (where he led the development and launch of Google’s AdWords), Alberto Savoia has experienced great market successes–along with a few inevitable failures. Most entrepreneurs and innovators respond to failure by licking their wounds and moving on to their next idea. But Alberto decided to first deal with the sting of failure by stinging back. Between 2008 and 2011, while still at Google, Alberto became a serious student of failure in new and innovative products. After reading dozens of studies and analyzing hundreds of new products, he was able to identify the main cause for why most new products fail in the market, and developed a set of techniques, which he called Pretotyping, to minimize such failures.

The concept, techniques, tools and metrics of pretotyping were an instant hit within Google; and soon Alberto found himself teaching pretotyping at Stanford University and at many companies, from startups to Fortune 500. His handbook “Pretotype It–Make sure you are building The Right It before you build It right” has been translated in many languages and has become an invaluable guide for thousands of innovators and entrepreneurs world-wide. Alberto’s work on innovation has been recognized with numerous awards, including The Wall Street Journal Innovator Award. Learn more about Alberto on his website and follow him on Twitter @Pretotyping.

 

Latest posts by asavoia (see all)