Embracing the uncertainty: Risk management in product planning
My previous post addressed some of the more common anti-patterns that arise when planning for future investments to make on your products. The fundamental problem with this sort of endeavor is that of human biases and our inability to predict the future. Product planning is after all a bet on the future, both in terms of what the market needs and your ability to deliver that, and bets are risky. In short, the anti-patterns that I mentioned in a previous post, could all be mitigated by proper risk management.
There are at least two distinct categories of risk in product planning. One is intrinsic to you, and therefore easier to manage. The other extrinsic and consequently riskier. The intrinsic risk is a function of the resources available to you that can be utilized to deliver on your roadmap. The extrinsic one is the market reaction to the product enhancement that you deliver over the time. You can reduce the bias and inherent risk of both.
Intrinsic risks
Product planning requires you to make predictions on the resources available to you and the time it takes to deliver on each discrete product investment. The two big elements of risk here are resource availability and estimates.
Resources are, broadly speaking, the number of engineers, product managers, designers and all other disciplines that are going to be actively working on building your products. They are your factory. One of the more common pitfalls I have seen when looking at your resources is assuming that they remain static and are somewhat fungible.
The people on your teams aren’t static. You might assume that they are happy and motivated and won’t be leaving, but life has an uncanny way of getting in the way. You have to assume that the people you have today aren’t going to be around for the foreseeable future, at least for the future that you are placing your bets on. Not only should you assume that the team will change, likely to your detriment, but you need to be actively de-risking dire negative outcomes. One of the more severe risks I have encountered here is being overly reliant on one or a very few engineers on some portion of your product and code-base. You never want to be single-threaded on any portion of your code-base.
Some of the mitigating tactics I have applied to counter this risk are rotations and pairing. Rotating your team members across various parts of the product serves two purposes. First, it allows your team members to explore more of the product and cover a broader set of use cases and potentially tech stack. This offers great learning opportunities and minimizes the risk of code ossification. Second, it provides natural coverage for being overly reliant on one or a few engineers who are familiar with a portion of your code base. Pairing is a technique that I am a big fan of. It naturally offers this redundancy, better code quality and a lot more.
Estimating software projects is a notoriously difficult and highly inaccurate endeavor. The topic is probably worthy of a few posts alone, hence I will try to avoid why that is. Previously I wrote about a way to gamify this activity in an attempt to make it more inclusive and reduce the variance in these estimates. The general idea is to divide the people who will be actively involved in delivering your roadmap into teams. Each team should ideally have the same personas like designers, software engineers, product managers and so on. Teams are then given a menu of potential investments that they are asked to estimate. The estimates must be done independently to reduce bias. Once all teams have completed their estimates, you can then assess them for large discordancies. For example, you might find that one team projected a feature to require 6 months, whilst another estimated it at 2. Digging into the assumptions each team made and having an open debate on the discordance helps reduce the variance.
One word of caution. This process isn’t bullet proof either. It reduces the bias and large variances, but doesn’t eliminate it. Estimating software projects is hard to get right.
Extrinsic risks
When placing these product bets, you are also betting that the market will react positively to them. This is a risky endeavor, and can be far riskier than the intrinsic risks I outlined earlier. The negative consequences of getting a negative reaction from the market can be quite severe: your business will suffer, your customers might churn and so on. Hint: The job of a product manager is hard.
The risk of you misreading the market, or delivering products and features that aren’t aligned with the market is a function of time. The further out in the future that these features or products are delivered, the riskier they are. Time comes into play in two manners. The first possibility is the predicted time to build a feature. The longer the time horizon, the riskier the bet. The second manner in which time manifests itself is by pushing out the release of a feature out into the future. This can happen when a feature is of lower priorities than others on the stack and hence is pushed out to some future date. The further out, the riskier it is. Each of these risks could be reduced, although the manner of reduction will differ.
When dealing with a feature, or product, that will require significant time (to me that is > 6 months), the best mitigation I have found is to decompose it into smaller incremental milestones or outcomes. Each discrete outcome or milestone represents a portion of the entire feature that is functional, can serve a subset of the market and therefore can be validated. Validation, and getting early feedback, is the key to reducing this risk. Rather than wait 9 months for your feature to be out in the market, you try and get a subset of it in 3 months, assess and validate before you commit to investing more. The art of this approach is in trying to decompose the entire feature into discrete and valuable deliverables. It’s easy to decompose a feature into technical milestones, but these can be of no value to customers. You must decompose the feature in a way that delivers value to some customers. You need that to validate the utility of the feature.
The manner in which one handles the risk involved in a feature that is far out in the future is not dissimilar than the previous approach: feedback, validation and hypotheses testing. Said otherwise, a roadmap is not a static diagram that you ink at the beginning of your planning cycle and stick to it for 12 or so months. You must continuously revise the assumptions you made, especially as you make progress towards some of the investments you made.
A feature that you recently released might have unlocked a new use case that you didn’t consider. This new use case might be of higher priority than the investments you decided on when you planned your roadmap. You therefore, need to be continuously revisiting your assumptions and the bets you are making.
Perhaps the biggest advantage of adopting this betting mindset is that it allows you to be a bit more creative and riskier in the bets you place. An anti-pattern I have seen over the years is to over index on the “voice of the customer” Your product team goes out and interviews customers and prospects and returns with a sorted list of features that then become your roadmap. After all, how can one argue against the voice of your customers? It turns out that customers don’t necessarily always know what they want, especially when it is concerned with product features. Over relying on what customers say might prevent you from thinking creatively and trying to offer a new and novel way to solve a problem. Over-index on customer problems and pains, not on how they want to solve them. By having risk mitigation actions in place you can be more creative and can take more risks, especially if you are going to be building novel and differentiated features.
It’s all about risks then
I used the work betting a few times throughout this article. It’s no coincidence. It’s a mindset I have embraced after reading Annie Duke’s most excellent book “Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts”
Annie is a former professional poker player and a World Series winner nonetheless. She is also a renowned author in cognitive-behavioral decision science and decision education. I leave you with one of the more impactful quotes and visuals from her book:
As outcomes come our way, figuring out whether those outcomes were caused mainly by luck or whether they were the predictable result of particular decisions we made is a bet of great consequence. If we determine our decisions drove the outcome, we can feed the data we get following those decisions back into belief formation and updating, creating a learning loop:
Duke, Annie. Thinking in Bets (pp. 79-80). Penguin Publishing Group. Kindle Edition.
ICYMI
20 CEO Lessons Learned at HubSpot on the Journey from $0 to $20billion by Brian Halligan
The Infra Red Report by Redpoint
Facebook releases Llama 2 just as ChatGPT’s usage starts to decline