How to Measure Product Success Using OKRs. Part 3
In our past two posts (check out part 1 and part 2 if you haven’t seen them yet), we’ve talked about the fundamentals of product discovery—how it needs to be continuous and customer-focused, with product development teams working in harmony to seek and incorporate customer feedback so they can deliver products they truly want, need and rely upon.
Now we get to dig into how to fine-tune this process. Product discovery should not only be constant but should also strive toward a constant upward trend in which product teams zero in on that target faster and more precisely with each new product or iteration of a product. One way to do that is to incorporate OKRs into your product discovery process: “Objectives and Key Results".
What do OKRs Mean in Product Discovery?
One of the lags that often occurs in product discovery is how a lot of teams rely on hindsight to see if they’ve succeeded in hitting the goal with a particular product. That, of course, takes time to evaluate, which is what creates the lag—a time that Tim Herbig suggests whittling away at by using a focused OKRs process strategy. The idea is that product discovery teams can deliver better products faster with a clearer vision by creating clear objectives and metrics that assess key results throughout the product discovery process, not just after it. It involves tweaking and honing the focus from just deliverables to what those deliverables should achieve, and how well the product impacts the objectives outlined. OKRs is a fairly simple product development strategy that came of age in the earlier days of Intel in the 1970s and spread throughout the tech world and is enjoying a resurgence today. That's because it has helped many companies align their teams around a focused objective, which clarifies what everyone in every team should be doing to create a positive impact for the customer.
How to Use OKRs in Product Development
It’s actually pretty simple. Looking at the needs of your target customers through feedback, you pick an objective and the results aiming at the objective they should achieve. This post from Atlassian drills it down very simply. For example:
- Objective: We solve customer pain point X.
- Key Result: We will be able to measure relief from paint point X by specific metrics, by a certain date.
Felipe Castro, an OKR coach, author, speaker, and evangelist, summarizes the two components of an OKR:
Objectives: are memorable, qualitative descriptions of what you want to achieve. Objectives should be short, inspirational, and engaging. An objective should motivate and challenge the team.
Key results: are a set of metrics that measure your progress towards the objective. For each objective, you should have a set of two to five key results. More than that and no one will remember them. These metrics should be something that can measured on a timely basis.
As simple as it sounds, it’s a very effective way to define and refine the goal ahead of the product discovery team so they can focus on the specific results to achieve along the way toward a customer-focused product that sticks the landing. In order to work, of course, the objective needs to be aligned with your overall company vision. Like SMART goals, the results need to be specific (creating this product or feature will solve the defined paint point), measurable (customer data will show a significant percentage of relief from the defined pain point), attainable (challenging, but still doable), relevant (focused on making life better for the customer) and time-based (quarterly, for example). It’s also important to note that OKRs are a collaborative process. It does not work effectively as a top-down product management tool, but as a team-based product discovery and creation process that is fluid, yet focused.
At Spark Equation, we use Perdoo, a platform for strategy delivery, to successfully execute our product strategies and implement the right OKRs, you can try Perdoo for free. Here's a free Atlassian OKRs template you can also use to get started.
Breaking Down the OKRs Process
It’s best to think of OKRs as a process rather than just events of pieces of a goal-setting strategy. One way to do this is to ask some questions and have the team collaborate to find the answers.
1. Defining the Objective
Relying on what you’ve discovered in customer feedback and discussions (remember, they’re actually part of your discovery team, or should be if they aren’t already), select a pain point or deep aspiration of your target customers. Is there some functionality they wish your current product had? Is there something they want to be able to do better, but they need a tool to help them? What is causing them stress that you could possibly alleviate? Questions like this will help you to land on an objective to create an impactful product or piece of functionality that they really need. The entire team should be involved in creating this aspirational objective.
2. Determining the Key Results
This is where you quantify what achieving the objective should look like in reality, so that you will know the product meets the objective you set out to achieve. Which numbers would change, and in what direction? For example, if you’re creating a feature or product to save the customer time analyzing their data, how many minutes or hours would it save? How would you measure ease of use? With each objective, delineate between two and five key results that have actual number measurements attached to them. This sounds easy on its surface, but the trick comes in knowing how to find the right numbers because they aren't always the ROI or percentages of customer satisfaction you might think of. We'll talk about that in a moment.
Once again, it's helpful to remember that OKRs are a process, not an event. Finding key results in your OKRs process, as Tim Herbig explains, follows a build + measure + learn + adjust + rinse, repeat pattern. And that requires the use of both leading and lagging indicators.
A lot of companies struggle to figure out the key results part of OKRs effectively because it can be a real challenge to define what metrics will provide the most accurate and positive way to measure key results. In part 2 of this series, we discussed how leading and lagging indicators factor into this process, and why it’s important to focus on the leading indicators to prevent lag in your product discovery process. Leading and lagging indicators measure different aspects of progress, and while both can have value, leading indicators accelerate the process and enhance your agility in creating stronger impact by sending better products to market through the OKRs model.
Using Leading and Lagging Indicators to Measure OKRs
Almost everyone already uses lagging indicators because they’re so easily quantifiable—after a product has been in circulation. For example, ROI or customer experience, two very important metrics that we all use to measure whether something succeeded or not. We can only measure those numbers after the fact. You can only measure your customer’s experience with a product after it’s been created and used. Unfortunately, a feature may not work well and cause problems for the customer, so then you have to go back to the drawing board to fix it. During that time, customers have also lost time and patience, and important windows for developing future products and customer relationships may have closed. With ROI, you may spend a lot of money on a project only to find out later that it didn’t generate much return. While these metrics can help you avoid making the same mistakes in future endeavors, there is still a lot of time lost in dissatisfaction and revenue drops. Relying only on lagging indicators in your OKRs process can weigh the entire product discovery process down, preventing your teams from the agility and success you’re seeking.
Leading indicators, though they can be hard to nail down, can help you predict successful product discovery actions so that you save time by avoiding things that won’t work. Leading indicators can help you better determine the key results you’re looking for in the OKRs process model. The challenge with discovering leading indicators is that it requires predicting end-user behavior or situations that may not have come up yet, which doesn’t always come with hard data to back it up. Glen Smith points out that working backward logically from the lagging indicator can help you predict what actions could have been taken to impact the end result, which might point you in the direction of a leading indicator you can use now in your key results phase. Discovering leading indicators is something you do while you’re in the product creation process by taking measures that have a strong possibility of leading to a successful outcome.
One example of a leading indicator that you might see in everyday life is the length of the car line-up to get into a national park on a holiday weekend. By seeing how long the line is, you can reasonably predict what the lagging indicator might be (how much time, in minutes or hours) it will take to get into the park. The exact time can only be measured afterward, but the length of the line and how slowly it’s progressing through the bottleneck can give you some in-process, predictive data that can help you change course. In other words, you might get on your phone and pre-pay so you can switch to a faster line. Even better, you should take your data into account before leaving: it's a holiday weekend. There's a strong chance there will be long lines. We should leave earlier and purchase our tickets before we leave. It's that kind of thinking ahead, using consistent customer feedback, that can help you discover leading indicators in your product discovery process that you can apply to your key results.
As you can see, finding the leading indicator here requires a strong degree of observational data upfront (we see the big line, we know it’s going to take a long time to get into the park, or we know it's a holiday weekend, so crowds are likely to form). This means staying in touch with your target customer before the product is done, adding touchpoints with them into the process so you can get their feedback on iterations of the build. And then you proceed to work forward towards the objective and key results with their feedback in mind—learning, adjusting, and repeating throughout the process. This creates more agility and helps you meet your objectives and see how to determine your key results more clearly. And ideally, moving forward on leading indicators will show up as positive results on the lagging indicators. So it’s important to look at both types of indicators in the OKRs method. Both have value. You can work backwards from lagging indicators to divine leading indicators and go forward proactively in the product discovery process.
Applying OKRs in Product Discovery
So, let’s say that you’ve been watching a competitor’s product leech away some of your customers. Customer churn is something you can measure based on how many customers you had last quarter, based on this quarter. If it’s gone down, something is up. Although this is a bit of a lagging indicator, you can push it forward into leading indicator territory (like the car lineup at the national park) and start making informed predictions about what objectives and key results you need to be working on in your next iteration.
Your product discovery teams should jump into action here with an objective that would look something like this:
Objective: “We will create a product that outshines competitor X’s product, bringing back former customers and gaining new ones.”
At this point, you’ll have to do some detective work to develop your key results so they rely more on leading indicators than lagging indicators. Teams should be able to directly link these results to actions the team takes and they should focus on what is likely to bring success. But because they’re part of the OKRs process (emphasis on process), they need to be agile and flexible, because they might change quite a bit over the goal cycle. Getting to the key results based on leading indicators might look something like this:
- Research into the competitor’s product
- Seeking feedback from former customers
- Creating a focus group of current customers to get feedback during discovery and development
- Polling customers on the top features they want to see in a new version
From that groundwork, the team can base all actions and behaviors that are most likely to create a product that achieves the key results that measure a change in customer behaviors like retention and acquisition by the desired percentages.
While the objective focuses on the impact your product discovery and development teams want to achieve at the organization level, the key results can be broken into two streams that feed back into that objective. Key actions or “output” that reliably predicts key outcomes (results).
Benefits of OKRs for Product Development Teams
One of the most important benefits of bringing OKRs into your product discovery and development mix is that it helps everyone pull together around more solid priorities than just grasping at the foggy notion of figuring out “what’s next". A clear objective about how you want to create something that delights customers creates a cohesive vision that brings the various teams together and breaks down communication barriers between stakeholders in order to reduce lag and increase agility. Drawing the focus of all teams around the objective clarifies the path forward and can reduce the tendency of teams to isolate themselves in the weeds of details that may not lead to the primary objective.
When companies rally around OKRs, which is supposed to involve every team and all levels of an organization, it should, ideally, improve alignment between teams horizontally and vertically.
In addition, teams are freer to find what really works in order to meet the objective. Instead of getting bogged down in this feature or that, the key results help them focus on the “impact” of the product created, which means that the features that do rise to the top will have what it takes to delight customers. It invites teams to hone in on real pain points that need addressing instead of “cool new stuff”. All of this is to say that the OKRs process should ultimately be customer-centric.
Challenges of Using OKRs
Just like any good product discovery strategy, however, OKRs does not make a company immune to systemic issues that can gum up its success. One common problem is creating an OKR that focuses too heavily on specified features and not the impact those features should create. This can happen when there is poor vertical alignment and unclear messaging from the top down. This leaves little room for product teams to adapt as they focus on customer behavior, or behavior changes that they should make within the organization. The goal should be to respond in a more agile way to customer needs based on leading indicators that appear during the continuous discovery process, and if everyone is too focused on granular features, that can get lost. OKRs work best when overall objectives are clear, focused and enable teams to adapt when leading indicators change.
It can also be helpful for teams to understand how OKRs mesh with a dual track scrum product discovery and development framework, breaking down even more barriers to getting excellent, pain-point-relieving products to customers faster without additional meetings and processes that can bog down the process.
OKRs Can Facilitate Continuous Product Discovery
It’s important to view OKRs in the framework of continuous product discovery. Objectives must be clear, overarching, and focus on creating a positive impact for users. As a takeaway - remember that leading indicators are a means and not an end. Listing as many as possible indicators won't automatically create product success, but can instead become a distraction and turn into vanity metrics. OKRs are a useful tool that allows teams to measure the direct impact of product actions instead of waiting for end-of-cycle feedback. Finally, don't feel you need to do this alone. Come together as a group to walk through this process, the discussions will likely be valuable in creating alignment in addition to output. If you have questions or want to evaluate your product processes, chat with us online, or send us a message.
Here at Spark Equation, every day is product discovery day, and we have a lot of positive client feedback implementing the OKRs method to clarify the process of discovering and delivering products. We assist our clients in successfully executing their product goals through engineering and product strategy consulting. Are you using OKRs in product development? Let us know!
Check out the next part of our series on Delivery: The Continuous Impact Loop