ReStartup: Build-Measure-Learn
Yesterday, we began this series on how established businesses can learn from entrepreneurs as they seek to ReStart after COVID-19 lockdowns. We also discussed how leaders need to learn to view everything as a hypothesis until it becomes a proven truth.
There are three steps to moving from hypotheses to truths:
- Develop a set of well defined hypotheses.
- Prioritize them so that you evaluate the most important ones first.
- Run experiments to turn hypotheses into verified truths.
Yesterday we used as an example the hypothesis that “everything will return to normal.” This isn’t a very good hypotheses for a couple of reasons, most importantly it is just too big.
“Everything” implies all aspects of the business: customer desires, operational effectiveness, financial performance, etc. That’s too big of a hypothesis to test at one go, so it’s helpful to break it down into a series of hypotheses that we can test individually. For example, we might have a set of hypotheses around customer behavior, another set around our internal operations, and another set around business economics.
In their book Testing Business Ideas¹, David Bland and Alex Osterwalder identify the characteristics of a good hypothesis. A good hypothesis is testable (it can be validated based on evidence), precise (you can define what success looks like), and discrete (it describes a single testable aspect of the business).
For example, “we believe customers still want our food” isn’t a very good hypothesis, while “we believe that at least 50% of our loyal customers plan to eat in our restaurant this summer” is testable, precise, and discrete.
We then need to prioritize all of these assumptions. Sometimes there are one or two “leap of faith” hypotheses that are so critical that, if they are wrong, then the whole business fails. For example “we believe that we will be able to source enough meat to keep up with customer demand” might be such a “do-or-die” hypothesis. These huge assumptions are the ones you need to test first. After these “leap of faith” hypotheses, move down the prioritized list to less critical, but still important ones. At some point in the process, you’ll get to hypotheses that are interesting but aren’t impactful enough to justify the time and expense to verify.
So how do we move from hypothesis to proven truth anyway?
In The Lean Startup², Eric Ries introduced the Build-Measure-Learn loop. In this approach, you build an experiment, you run the experiment and measure results, and then you evaluate the results to see what you’ve learned. Based on what you’ve learned, you modify your hypothesis and refine the experiment, then you run it again. And again. Until the experiment validates the current hypothesis.
Ries makes two important points about the Build-Measure-Learn loop. The first is that you need to approach it in reverse order. You identify what you want to learn, then you determine what you could possibly measure to answer the critical questions, and then finally you design the right experiment. The second point is that at the end of each loop, you need to ask whether you still have confidence in the general hypothesis and you just need to refine it, or is it time to “pivot” to a different hypothesis altogether.
All of that is fine and good, but how do you design and run an experiment in the first place?
Bland and Osterwalder break hypotheses into three broad categories:
- Feasibility: Can we do this? Can we make it and deliver it?
- Desirability: Do they want this? Will customers buy it?
- Viability: Should we do this? Can we make money?
They then provide 44 different experiments. For each experiment they ballpark the financial and time investment required, the speed of results, the strength of evidence produced, and the broad categories of hypotheses that the experiment is well suited to address.
A few examples from their book might help give a sense for what experiments look like.
Some experiments are similar to traditional forms of corporate learning. For example, customer surveys can be helpful, especially if they are designed to truly test hypotheses and not just confirm the current beliefs.
Other experiments may be new to ReStartups. For example the “Buy a Feature” exercise modifies the traditional focus group meeting to have customers allocate imaginary money amongst potential improvements to your product or service. Another example is the “Wizard of Oz” where you have people “behind the curtain” manually performing tasks that in the future will be automated in order to test and understand the value of that service before investing in the automation.
Selecting and running experiments takes diligence, but the hardest aspect of adopting the Build-Measure-Learn approach into ReStartup (established business) is the change in attitude required. Quickly learning that our current model isn’t going to work anymore is a huge win, even though it feels like a major setback. By uncovering the issue early in the ReStart, we hopefully have time to pivot to a different approach. (Of course, that new approach then needs to be tested and validated.)
Proverbs 1:5 teaches us that “A wise man will hear and increase learning, and a man of understanding will attain wise counsel.”
Demonstrate wisdom by being teachable and adapting to this still unknown new reality.
Sources:
¹Bland, D. J., & Osterwalder, A. (2020). Testing business ideas. Hoboken, NJ: Wiley.
²Ries, Eric. The Lean Startup: How Todays Entrepreneurs Use Continuous Innovation to Create Radically Successful Business. New York: Crown Business, 2011.
