Useless vs. Useful Competitive Analysis

Part 1 of 3

Co-written by Adrienne* and Alexis

When I* first joined Tesla as a product manager, I remember making a competitive analysis chart to understand the competitive landscape. This is a common exercise for product managers. They (1) make a list of competitors, (2) build a set of criteria to evaluate the competition (such as pricing and features), and (3) evaluate all the variables against each other.

Except this was a completely useless exercise and nothing ever came from it. High level industry competitor analysis is useful when you want to understand the lay of the land, but that is not to be confused with affecting your team’s immediate roadmap.

I later realized the exercise wasn’t useful because Tesla customers don’t buy cars by comparing features. Tesla’s customers are a luxury car buying group, and when they buy a Tesla, they buy a feeling — wanting to look good to their friends or feel like they’re commuting to work luxuriously.  But that doesn’t mean the feature comparison chart is always a useless exercise -- because there are people who make their decisions that way: typically, businesses buying enterprise SaaS products make their decisions very logically.

In this case, I was simply falling into the footsteps of copying what people normally did, without thinking about the reasons behind it (this is the difference between being dead vs. live). I had compared Tesla against competitors along arbitrary criteria that did not matter much to Tesla’s customers, and had no clear idea how to act on the analysis. What I had to ask myself instead was, “What part of my roadmap am I trying to change as a result of this exercise?” And design my competitive analysis to serve that purpose.

Ask: How will the results of this analysis change my strategy? 

The competitive analysis for a strategy consultant and a product manager look different. While the analysis for a strategy consultant is more high level, the analysis for a product manager must be precise in order to be useful.

Here are three ways you could use a competitive analysis to improve your roadmap:

1. Set the bar for engineering polish

Competitive benchmarking can help you set expectations for the reliability and performance your team should be targeting. When my team launched Tesla Voice Assistant, I looked at Apple CarPlay, Amazon Auto, and Echo Auto and compared functionality against various variables:

• Average response time

• Transcription accuracy

• Offline functionality

• Number of supported languages

While we considered targeting offline functionality for the initial release, we realized that none of the other services offered this, and so we deprioritized it. This exercise allowed me to understand where other services have set the bar for reliability, so that we could ensure our product was better than comparable services from a reliability perspective.

2. Identify a missed user need

You can also look at different competitors to reverse engineer critical user needs and insights that you might have not identified in your own research.

For example, Alexis leveraged this when she was launching moderators for Facebook Groups. Previously, Facebook Groups only had a single type of role called “Admins,” and Alexis was working on adding a second “Moderator” that had less privileges than Admins but could still help in managing large groups. She had done user interviews with current Admins to identify core features that should be available for the new Moderator role, but wanted to check if there are any missing user needs she failed to identify; after all, she could only interview a few Admins though Groups is a product used by thousands of different use cases.

As a result, Alexis analyzed competitor products (e.g. Reddit, Meetup, Band) for their Moderator equivalent. She found that competitor products often had “auto-moderation” capabilities, a feature which had not come up in prior research. This piece had been missing in user interviews because the need for bulk automated moderation tools only make sense for extremely big Groups, and admins of those groups are rare and hard to get for user interviews. As a result of competitive analysis, Alexis identified “bulk action” as a potential user need, tested it, and added it to a list of Moderator features.

3. Validate a killer feature

If you have a hypothesis for a killer feature that will really make your app, a competitive analysis can help you confirm whether it will make your app unique. Often, the “innovative killer feature” you’re banking already exists in a similar form in a competitor product, or it exists in the exact same form across many competitors at which point you need to re-evaluate your product’s value proposition.

When Alexis was launching Instagram Video Chat, for example, she developed a hypothesis that having video chat on the mini screen while browsing Instagram would be a killer feature. This is because, in user research, there was a commonly observed behavior where two people would leave Facetime on for a long time, and chat casually while browsing Instagram, making conversation based on content they see in the feed. This suggested that a miniscreen that shows both people’s face that floats on top of Instagram feed would be superior to the current experience, where two people can’t see each other’s face on Facetime when browsing Instagram. The team could move confidently knowing that this would be a killer feature, because they did competitive analysis on a huge range of video chatting competitors, and concluded that this feature would truly be unique.

When you’re about to embark on a competitive analysis, instead of jumping in right away, first ask yourself what kind of change you’re looking to affect on your team’s roadmap, so that you can actually drive real impact.

We’ll cover other parts of competitive analysis (finance-driven for part 2 and vision-driven for part 3)