The holidays are nearly upon us and many of you may soon be asked by your loved ones “What would you like for Christmas/Hanukkah/Kwanzaa?”
I’ve pulled together a few items that are some of my favorite things. Feel free to share this gift guide with those who might be struggling to pick the perfect present for their beloved product manager!
Although I do most of my work on my laptop, there are times when I prefer to take notes the old-fashioned way. In fact, research has shown that writing notes (rather than typing) helps you retain information better. I prefer this journal that has dotted pages so that you can easily create boxes and diagrams.
I’m lucky that I work from home now, but I remember the days of being in an office surrounded by sick co-workers who would say things like “really, it’s just allergies”. Stay healthy by keeping a desktop air purifier at your desk. This one is nice and small for a personal space.
Many product managers work in open workspace environments, so a good pair of noise-canceling headphones is a must-have to help block out the noise and focus on making better product decisions. I love these Bose wireless headphones! They have a good microphone which is a bonus if you’re doing a lot of calls from your desk. Plus, they’re Alexa-enabled!
For the Product Manager that works from home occasionally, this is a home-office must-have. The Sonos One lets you play your favorite music and control it with your voice or choice of apps. Whether you’re most productive when listening to your favorite Spotify playlist or want to catch up on product management podcasts, the Sonos One is ideal. And because Amazon Alexa and Google Assistant compatibility is built in, you can use it as a virtual assistant to book meetings, check the weather and save you time.
This is a life-changer! If you have a standing desk, this will help you focus on maintaining good posture while — bonus! — strengthening your core and improving your balance. I found that it also prevented back strain associated with standing too long. I also love that this product is made in my home town of Santa Barbara.
Product Managers are typically avid learners. In addition to a few of my favorite books which I’ve listed below, the Amazon Kindle Paperwhite is the best way to devour a stack of books without weighing down your bag. You can load it up with the books below!
In the first part of this three-part series on metrics, we discussed vanity metrics. Vanity metrics are typically raw numbers taken without context, such as page views and app downloads. They don’t correlate directly to customer value. And they don’t provide guidance for what future product changes you should make.
Outcome-oriented metrics, on the other hand, do provide guidance on how to evolve and improve your products over time. Outcome-focused metrics typically do 3 things:
They link actions to results.
They focus on delivering customer value.
They provide insight into the health of your product or business.
Remember the differences between vanity and outcome-based metrics. Vanity metrics might make us feel good, but they don’t help us improve or optimize our business. Outcome-based metrics provide insight into the health of your product.
Where should you start when you’re ready to embrace an outcome-focused approach to measuring success?
Start with your company’s strategy.
When your company has a strong, clear vision, it should serve as your north star and guide you to defining the most important metric. Based on that vision, what is the one metric that matters most to the success of your company and that you can rally your team around?
Let’s look at a great example of leading from strategy with Zappos. Their CEO literally wrote the book on modern customer service. Zappos disrupted online retail sales by focusing on its support department as an opportunity to market and generate revenue rather than as a cost center. Their entire strategy revolves around creating loyalty among its customers using effective KPIs that lead to what they call ‘wow’ moments. And they did this because they found that repeat customers spend more than first-time customers and drive referrals.
Here’s another example from Sonos, where I worked for nearly twelve years as a product management leader. The Sonos mission is to fill every home with music. We used that as our north star when defining key outcomes. For example, ‘fastest time to music’ was a key outcome by which we would measure the success of a particular feature. The rationale was the faster the music starts playing, the better Sonos is doing its job of filling the home with music.
Ask yourself, what is the key outcome you can focus on and set actionable KPIs against based on your companies strategy?
With your company strategy in hand, I recommend selecting a useful framework for measuring key outcomes and the one I like is Pirate Metrics.
Pirate Metrics Framework
Pirate Metrics were coined by Dave McClure back in 2007. Here’s a link to his presentation where he first describes this framework. He breaks it down into 5 components.
Acquisition – how well are you getting customers to your site or app?
Activation – are your customers having a great ‘first run’ experience?
Retention – how often are your customers coming back?
Referral – are they telling others about your product?
Revenue – are they paying for your service? Are you able to monetize your customers?
Let’s dig deeper into this and come up with some examples to help you better understand pirate metrics.
To better understand how to apply pirate metrics, pretend we’re product managers for lynda.com, an online learning platform that lets you watch videos on how to do everything from learning to knit to learning to code and everything in between. What are some outcomes that we might care about that could be measured using the pirate metric framework?
For starters, let’s look at acquisition. We don’t necessarily care how many people come to the lynda.com homepage. That’s a vanity metric. What we probably do care about, though, is how many people are signing up for a free trial of the service within a specific time period, say each month.
From there, how can we measure activation and whether or not our customers are having a great first run experience? A good metric, building on the previous one, would be what percentage of the people signing up for a free trial are watching a video from start to finish? If they’re only watching a few seconds of a video and then leaving, you could hypothesize that they’re not having a great first run experience.
To measure retention, we could measure what percentage of those people are coming back to watch another video within the month. This is probably one of the most important metrics product managers care about. Knowing your MAU, or monthly average users might even be your company’s top line metric, which is the case for a company like Facebook.
For measuring referrals, we could measure what % of people are sharing links of videos to their network. Some companies like to measure NPS, also known as the Net Promoter Score. It’s based on asking customers if they would recommend your product to friends or colleagues. I’ll talk more about this in a few minutes.
The final, and probably most important metric, especially if you work for a SAAS product is measuring how much revenue your customers generate. ARPU, or average revenue per user, is often the top-line metric for SAAS business. How might we measure this? One possibility is to measure the % of people who go from the trial to paying for a monthly subscription, which most companies refer to as their key conversion rate.
Putting outcome-based metrics into practice
Now that you have a framework to help you identify all of the possible types of outcomes you can measure, let’s talk about how to harness this information and put it into practice.
We can boil it down to four key steps:
What is the key outcome based on your company’s vision and strategy? What is the most important thing you can improve upon? Perhaps it’s an increase in your conversation rate.
Form a hypothesis. The important thing to remember is to start small and look for the most meaningful lever that you can pull and focus on that. Don’t change five different things on your landing page and then start measuring your conversation rate. You won’t be able to correlate which of those changes affected your conversion rate. Change only one thing. Your hypothesis could be something like: “If we reduce the price of our product by 10%, we’ll see an increase in our conversation rate of at least 11%.”
Build your experiment. This is where you need to dissect your hypothesis into key components so that you can collect the correct data to validate if your hypothesis is correct or not. Before you build anything, make sure you know what your current state, or baseline metric, is. In this case, make sure you can state what your current conversion rate is. Once you have your experiment ready, set up your analytics to measure the KPI against the current baseline and the goal that you’ve set.
Measure and analyze. Once you’ve got the new data coming in from your experiment, you should be able to quickly analyze if it was a success or not.
If it wasn’t, that means your hypothesis was incorrect. Remember that you should not view these moments as failures or a waste of your precious developer resources. This is a learning moment. Failing fast and learning early is key to allowing you to eventually zero in on what works.
If the experiment was mildly successful, I encourage you to tweak your hypothesis based on the new data you have.
If the experiment was mildly successful, I encourage you to tweak your hypothesis based on the new data you have.
If it was wildly successful, celebrate! And then look for the next lever you can pull to help your business be even more successful.
Ready or not, let’s dig into ugly data and, by that I mean flawed charts.
The Ugly – Flawed Charts
As often as product managers talk about making data-informed decisions, it’s amazing how often data is presented in ways that are completely misleading or downright wrong.
Many of you may remember the Venn diagram incident from Hillary Clinton’s presidential campaign a few years ago.
The general point her campaign team was trying to make was true. The majority of American’s support universal background checks including a majority of gun owners. If you think about it, though, the yellow circle should be completely inside of the blue circle because the gun owners referenced here are all Americans. But that’s now how Venn diagrams work! This information should’ve been presented using a different type of chart.
In this example on the right, it appears the designer messed up the doughnut chart. As you can see, the green line goes around more than half of the circle, even though the number referenced is 41%.
The chart to the left is bad for several reasons. If you follow the use of colors, you’ll see Iceland, Finland, Portugal and Spain all nicely represented with different colors. And then it shows the UK, Denmark, Australia, Venezuela, and Kenya as being part of the same country. This would’ve worked much better as a simple bar chart!
The lesson here is to make sure you use the right type of visualization for displaying your data and that your chart is correct.
Where to get help with selecting and creating charts
If you’re struggling with deciding which type of chart to use, I highly recommend From Data to Viz.
Whether you’re new to the world of creating graphs or you’ve been doing this for years, this website is a treasure trove of information about how to present data.
Based on the type of data you have, such as chronological or numerical, it will guide you through the relevant types of visualizations that are best suited for your particular case.
Depending on the type of chart you’ve selected, it will also inform you about the pitfalls to avoid.
And, if you ever get the chance to attend one of his seminars, it’s well worth it!
Now you now how to avoid some of the biggest mistakes when creating data visualizations. Plus, you’ve got some great resources to get inspiration on the best way to present your data effectively. Being able to communicate data persuasively is a key still all great product managers should have. It will help you manage your stakeholders, influence product decisions and build support for your work. Wield this power wisely, grasshopper!
The next and final post of this series will culminute with how to use data for outcome-based product teams. Don’t miss it. Sign up to stay informed when it comes out.