The Allure and Trap of % Goals

Written by Alexis* and Adrienne

Early in my career, I* noticed the allure of % goals. If I had to increase restaurant reviews, then % of users who write restaurant reviews sounded like a great goal metric for my team; if I had to increase comments, then % of posts with comments. I used to think that % goals are a sophisticated proxy for tracking a product team’s progress, since they’re grounded in context (the denominator). It wasn’t until later that I learned that % goals come with a hefty price tag, and are often more work than they are worth. 

Let’s see why with a hypothetical scenario.

Downstream consequences of a % goal

Let's say you are a PM for the local business review site, Yelp. You have the product opinion that in order for reviews to be valuable to users and businesses, a business should have at least 300 reviews. You want your team to prioritize features that would increase the number of businesses with many reviews. You consider two options for your goal metric:

• Option 1, the # goal: # of businesses with more than 300 reviews

• Option 2, the % goal: % of businesses with more than 300 reviews

  i.e. Numerator = # of businesses with more than 300 reviews

  Denominator = # of all businesses on Yelp

Let’s say that you chose Option 2, the % goal. By doing so, you have signed up to the following significant downstream consequences. 

1. Bake in unintentional product opinions into your goal

By introducing a denominator into the goal, you’ve baked in additional product opinions unintentionally. They may lead your team to build features you didn’t intend. For example,

• Opinion 1: Dampening the increase of new business sign-ups on Yelp (denominator) is a valid strategy for increasing the % goal.

  • Shrinking the denominator while keeping the numerator constant will increase the % goal.

  • This % goal could adversely incentivize the team to hamper the growth of the denominator (# of all businesses on Yelp) instead of increasing the numerator (# of businesses with more than 300 reviews). For example, your team could de-prioritize fixing bugs in the new business sign-up flow. 

• Opinion 2: Every business that joins Yelp, whether a cafe or a moving company, should hit 300 reviews as soon as possible.

  • The % goal will decrease every time a new business joins (increasing the denominator while numerator is constant), unless the new business racks up 300 reviews quickly.

  • However, accumulating 300 reviews could take longer for some services than others. Aggressively pushing a moving company (that serves only a few customers a day) to achieve 300 reviews at the same pace as a restaurant, would cause the moving company to think its goals are not aligned with that of Yelp’s.

  • This unintentional product opinion could have been avoided with a # based goal (# of businesses with more than 300 reviews), which is agnostic to which services the “businesses with 300 reviews” come from. It’s only an issue with a % goal, because it has a denominator that you need to define separately (e.g. # of businesses on Yelp except home services), and your team may or may not have done enough research to do that thoughtfully yet.

2. Spend more time communicating your team’s progress to the rest of the company

In order for a % goal to be interpreted correctly, it’s best to send all three numbers: the % goal, the numerator, and the denominator, separately. However, the last two usually get lost in communication since people are used to seeing a single topline metric. This often leads to frequent miscommunication resulting in executives requesting for follow ups. Responding to them will be a pull on your team’s time and resources.

3. A goal that’s susceptible to erratic movements and harder to diagnose

• It’s better to choose a goal metric that’s less jumpy and prone to false alarms.

• % goals can be excessively sensitive or excessively insensitive, but rarely in between. 

  • Insensitive: Usually % goals don’t get reported beyond 1 decimal point.

  • Sensitive: The moment the denominator and numerator moves in the opposite direction, or the amount changes are different, % goal amplifies the impact.

• % goals take longer to diagnose, because

  • It requires a deep dive on two separate metrics (the numerator and the denominator).

  • The dynamics between the numerator and the denominator are could hard to pull apart, especially when one is an indirect lever or function of another.

So do % goals ever make sense?

% goals make sense in an entirely understood system, when

• Most levers impacting the % goals are known

• Shape of the distribution for the numerator and the denominator are well understood

• The metric has some historic data that could enables deeper analysis

Examples include error rate % from a specific code section, call failure rate % in video chatting, or click through rate % for a specific ads flow.

However, most products do not live in an entirely understood system, so choosing a % metric as your goal will come with a lot of overhead.

In summary, % goals require…

• Additional due diligence to make sure you’re not building in unintended product opinions

• In-depth understanding of every component of the goal

• Capacity to frequently diagnose erratic movements and debug misunderstandings in communication

% goals may be very tempting, but beware of the unintentional downstream consequences.