Objectiv’s Impact Scores

Finding the value of each feature to your product goals.

My colleague Daniel van Dijk recently described the psychological phenomenon of loss aversion and how it can affect PMs to focus not on further improving aspects users already like, but on functionality they perceive as missing.

In this article we highlight one of the foundations of Objectiv that helps PMs deal with this: Impact Scores. As opposed to averting loss, Impact Scores instead enable Product teams to focus on their product’s strengths & what is actually driving their goals, going beyond basic usage metrics.

Focus on product goals

Tools like Google Analytics tell us exactly which features are heavily used. So those are the ones we should focus on, right? Glad you asked :). Objectiv moves beyond pageviews. Why?

Because something that is used a lot does not mean a great deal in terms of driving your product goals.

It can mean that both high- and low-value users like it, or it might just be because you gave it a lot of visibility. Instead, Objectiv uncovers what truly makes your users successful on your product goals. We call this metric Impact.

For the examples in this article, we will take a typical streaming media product — let’s call it MyMedia — with features to listen to and view the content in various ways. A goal for such a product could be to increase Engagement.

Proven Impact

Using your existing analytics solution, Objectiv’s AI assigns an Impact Score to each feature in your product. This is based on the behaviour of each individual product user.

An Impact Score shows how a user’s interaction with a feature drives your product goal.

For example, a Score of 4 shows that, if a user uses that specific feature in their customer journey, they are 4 times more likely to reach your goal — in the case of our example MyMedia product: being Engaged.

Here’s an example in the Objectiv Workspace:

The Objectiv Workspace showing Impact Scores for an example ‘MyMedia’ product

In this specific view, all homepage features are listed with their respective Impact Scores. The ‘Recommended Music’ feature clearly has the highest Impact, scoring 5.42 on the product goal. This means that users who interact with ‘Recommended Content’ features on the homepage are 5.42 times more likely to become Engaged.

Also shown is an indicator for usage, ranging from Low to High. Features with a high Impact Score and low usage, for example, are low-hanging fruit to give more visibility in your product. The example Note in the panel on the right highlights two of these.

Negative Impact

What if you could see which of your features actually discourage users and hurt your goals?

Objectiv’s AI also finds these features, quantifying their negative Impact on your product goals. This is to say, relatively more low-value users are using those features than high-value users. Shown for our example MyMedia product in the Objectiv Workspace:

These features are generally great candidates for making space for high-Impact features, or re-evaluation altogether. As my colleague Daniel wrote:

View weak metrics as opportunities for simplification instead of improvement. Chances are, those features are taking up space that your high-impact features deserve.

How it’s used

Impact Scores easily translate into your next roadmap moves. Some examples of how teams use them:

Making data actionable

Data can be messy. Objectiv data models continuously iron out any kinks and make it actionable. It’s like having a room full of Data Scientists aiding your product decisions.

Causality is built in
Do our Impact Scores only show features used by the most successful users? No. In defining Impact Scores, Objectiv only takes into account the behaviour prior to a user becoming successful on your product goal.

Not influenced by external factors & traffic peaks
Did a big marketing campaign just run, did a national soccer team just win, or is there some form of seasonality at work? Did your new feature really work, or was it just a lucky month? No worries: Impact Scores are designed to work around this. If the Impact Score is high, your feature works, regardless of your traffic peak.

Sensible minimum levels of data
In a perfect world, all your data points are statistically significant. But the reality is that most features are slow to reach significance. That’s why Objectiv has closely worked with customers over 1.5 years to understand what usage level of data is sufficient before it’s used, based on actual customers seeing meaningful results when acting on it. Those minimum levels are built into the Impact Scores.

Make an Impact

We believe Product teams deserve the tools to spend more time building great products that truly make a difference.

Objectiv’s Impact Scores are key in that: they uncover and quantify what features are really driving product goals, going far beyond simple pageview metrics.

Intro to Objectiv

In 30 minutes we'll discuss
  • Your team & analytics workflow.
  • What problems you are looking to solve.
  • How Objectiv can help.