Tag Archives: business models

What Is Good Retention: An Exhaustive Benchmark Study with Lenny Rachitsky

At the end of 2019, I presented Eventbrite’s product plans to the board for 2020. These plans included a lot of the goals you likely have in your company: improvements in acquisition, activation, and retention. One of our board members asked: “I understand these goals for the year. But long term, how high could we push this retention number? What would great retention be for Eventbrite?”

I actually didn’t have a great answer. Soon after, I was chatting with Lenny Rachitsky, and we decided to embark on a holistic study across the industry to ask “what is great retention?” across business models, customer types, etc. Lenny surveyed a lot of the top practitioners in the industry across a variety of companies, and we’re happy to share the results here. You can see the raw data below, but I recommend reading Lenny’s analysis here. Done? Good.

Why is retention so damn important?
Why are Lenny and I spending so much time researching retention? Because it is the single most important factor in product success. Retention is not only the primary measure of product value and product/market fit for most businesses; it is also the biggest driver of monetization and acquisition as well.

We typically think of monetization as the lifetime value formula, which is how long a user is active along with revenue per active user. Retention has the most impact on how many users are active and lengthens the amount of time they are active. For acquisition, retention is the enabler of the best acquisition strategies. For virality or word of mouth, for example, one of the key factors in any virality formula is how many people can talk about or share your product. The more retained users, the more potential sharers. For content, the more retained users, the more content, the more that content be shared or discovered to attract more users. For paid acquisition or sales, the more retained users, the higher lifetime value, the more you can spend on paid acquisition or sales and still have a comfortable payback period. Retention really is growth’s triple word score.

What are effective ways to increase retention?
Okay, so you understand retention is important and want to improve it. What do you do? Well, at a high level, there are three types of efforts you can pursue to increase retention:

  1. Make the product more valuable: Every product is a bundle of features, and your product may be missing features that get more marginal users to retain better. This is a journey for feature/product fit.
  2. Connect users better to the value of the product that already exists: This is the purpose of a growth team leveraging tactics like onboarding, emails and notifications, and reducing friction in the product where it’s too complex and adding friction when it’s required to connect people to the value.
  3. Create a new product: Struggling to retain users at all? You likely don’t have product/market fit and may need to pivot to a new product.

We discuss these strategies in a lot more depth in the upcoming Product Strategy program coming soon from Reforge, and if you really want a deep dive on retention, we build the Retention & Engagement deep dive.

Why does retention differ so much across categories?
One question you might be asking yourself is why does retention differ so much by different categories? This was the impetus for the initial research, and why I couldn’t give a great answer to our board. Every company has a bunch of different factors that impact retention:

  • Customer type: For example, small businesses fail at a much higher rate than enterprise businesses, so businesses that target small businesses will almost always have lower retention.* This does not make them inferior businesses! They also have many more customers they can acquire.
  • Customer variability: Products that have many different types of customers will typically have lower retention than products that hone in on one type of customer very well.
  • Revenue model: How much money you ask from customers and how can play a big role in retention. For example, a customer may be more likely to retain for a product they marginally like if it costs $30 vs. $300,000. A product that expands revenue per user over time can have lower retention than ones that have a fixed price.
  • Natural frequency: Many products have different natural frequencies. For example, you may only look for a place to live once every few years (like my time at Apartments.com), but you look for something to eat multiple times of day (like my time at Grubhub).
  • Acquisition strategy: The way a company acquires users affects its retention. A wide spread approach to new users may retain worse than carefully targeting users to bring to your product.
  • Network effects: Network effects may drive retention rates up more over time vs. businesses that do not have these effects. For example, all of your friends on Facebook or all of your co-workers on Slack makes it hard to churn from either product whereas churning from Calm or Grammarly is entirely up to you.

* In those businesses, the business failing and churning as a result is called “involuntary churn”, though that can also mean a payment method not working for someone who wants to retain in other models.

BONUS: Why are Casey’s benchmarks for consumer transactional businesses lower than others?

For the demand side of transactional businesses, where the retention graph flattens is more important to me than the six month retention rate. And unlike other models, these businesses can take longer than six months to have their graphs flatten. Also, for marketplaces, one of the two common models along with ecommerce in this category, a healthy demand side retention rate is very dependent on what supply side retention looks like and acquisition costs. For example, since Uber and Lyft have to spend so much time and money acquiring drivers due to a low retention rate, in order for their model to work, demand side retention either has to be high or demand side acquisition has to be low cost. For a business where supply side retention is high and acquisition costs are low, demand side retention can be lower, and the company can still be very successful. Etsy and Wag I imagine fit more into this model.

Currently listening to We All Have An Impact by Boreal Massif.

Google’s New Strategy and How It Affects Aggregators

As someone who has had a lot of success using SEO as a tactic to grow companies (for Apartments.com, Grubhub, and Pinterest it was the dominant channel for new users), I get asked a lot of questions about SEO as a strategy today. Andrew Chen’s Law of Shitty Clickthroughs states that all acquisition channels have a shelf life and decay over time. SEO has had by far the longest shelf life of any major internet channel. It has been a stable platform (unlike Facebook), it consistently grew itself, and it was supported by a very strong business model that could drive revenue growth for Google for well over a decade, so Google didn’t need to monetize all of the free traffic they distributed to other companies. Perhaps a more elegant way of explaining this is that since organic search exists to serve user needs, not advertiser needs, it was a more sustainable acquisition channel, precisely because it was not built to be a channel. People never wanted ads, more email, etc. like other acquisition channels. On Google organic search, people do want answers.

It’s this last statement remaining true amidst a platform shift to mobile that will mark the inevitable decline of SEO as a channel for user acquisition. Ben Thompson declared Peak Google a few years ago as a company. Why he was wrong then is why I am right today by declaring we are now past Peak Google as an acquisition channel. To understand this, you have to understand Google’s strategy. Google’s search engine is driven by optimizations that help its users. Ben Thompson does a good job explaining how this plays out with publishers. If you, like Google, have been analyzing its users for the last few years, you may have learned a few things. The first is that the majority of them are on mobile, where their time is more limited, their connections are (still) slower, and there is the threat of an app replacing frequent queries.

What Google is seeing is that their users no longer want to click ten blue links. They don’t have the time or the bandwidth, and there are now a plethora of competitors in the form of apps for many of those queries. The form factor is dictating the optimal user experience, and forcing Google to evolve. Users want an answer, and they want it immediately. So, that is what Google is doing. If you type a question into Google with a clear answer, there’s a good chance Google will just answer the question instead of recommending a site for it. We’ve all seen that. What’s more interesting is what Google is doing when there isn’t an answer, and the solution is to provide options, or what they would likely call a discovery experience. Recipes is a great example. Where you used to be treated to a bunch of “21 best recipes for X” pages, you now just see the recipes as results.

One would think these queries play perfectly into Google’s existing strategy of ten blue links. But Google knows consumers don’t want to click back and forth onto multiple sites to see each site’s recommendations. They want Google to surface up the options directly. And that is what Google is now doing. Take any top query category, and you will see Google replacing links with either answers or options showing up directly in search.

Whereas the top strategy historically for these “options” queries was to build an aggregator and rank at the top by having the most options, Google is now stating that it should be the only aggregator. In the same way Ben Thompson described the squeeze between Google’s demand for fast loading pages and banning of obtrusive ads on Chrome, Google’s search policies are doing the same for aggregators who do well on organic search.

How is Google doing this algorithmically? Google has started to seriously enforce two new policies over the last couple of years in their algorithm: internal search and doorway pages. For internal search, Google says:

Don’t let your internal search result pages be crawled by Google. Users dislike clicking a search engine result only to land on another search result page on your site.
Source

For doorway pages, Google says:
We have a long-standing view that doorway pages that are created solely for search engines can harm the quality of the user’s search experience.
Source

On the surface, these rooms seem sensible. If you continue to read to the end of the doorway pages update, you may start to see a problem:

  • Is the purpose to optimize for search engines and funnel visitors into the actual usable or relevant portion of your site, or are they an integral part of your site’s user experience?
  • Do the pages duplicate useful aggregations of items (locations, products, etc.) that already exist on the site for the purpose of capturing more search traffic?
  • Do these pages exist as an “island?” Are they difficult or impossible to navigate to from other parts of your site? Are links to such pages from other pages within the site or network of sites created just for search engines?

Reading these two together, what Google is saying is that creating pages indexing your search result pages is a bad experience, and creating new pages that replicate your search experience in a different way for search engine visitors is a bad experience. These two rules effectively penalize any presentation of your inventory of content. How do they tell if these pages are created solely for search engines? It’s similar to how they are detecting bad ads: they use Chrome browser data. So, the guideline for an aggregator who would like to show their content to Google is simple: give us your content, and we’ll aggregate it, or play in the tiny space that is a non-internal search page and a non-doorway page. That effectively means creating a page with unique inventory that does not look like search, yet receives traffic to the page from other sources besides Google. There is a window there, but it is a small one.

While updating the algorithms for these rule changes, Google is figuring out how to be the aggregator in many categories now, and over the next ten years will go down the list of every top searched category and figure out exactly how to do that. To do this, Google will either build, buy, or partner with existing players. Let’s take a look at some of these examples.

The Build Case: Google Local
Google tried to acquire Yelp to build local listings and relationships with local businesses. When Yelp refused, Google built out Google Local on top of Google Maps and Google Search, and now has direct relationships with thousands of businesses managing their information directly with Google. This was not very difficult when you have the dominant search product and the dominant maps product to build on top of. When you search a local query, you see no ads, just Google Local above all other search results.

The Buy Case: Google Flights
In July of 2010, Google acquired ITA Software, forever depressing the market caps of many travel-related internet businesses. ITA powered flight search and pricing information for many top online travel agencies. As you might have guessed, that data now appears on Google directly for free. Google is monetizing that space, and looks to moving from a pay per click model to a more transactional model over time.

The Partner Case: Google Events
Late last year, Google launched a dedicated section when people search for events that aggregates events from third party sources, including Eventbrite. Third parties give their inventory to Google, and Google ranks the events on its own. You cannot rely on your business to be in this category. Google will likely partner if:

  • There is no dominant player they can buy
  • Supply is fragmented and data unstructured
  • There are multiple companies willing to implement specific markup to appear in these discovery experiences
  • The area is not one of the leading categories for monetization for Google today

What do you do if you’re affected?
If you are an aggregator, and Google is moving into your space, it changes your SEO strategy entirely. Whereas you used to create and optimize pages that aggregate inventory for popular queries e.g. “san francisco food delivery” for Grubhub, those pages will now be de-valued as Google replaces those listings with its own aggregator. The best solution to this problem is to shift your strategy to distribution of your individual listings, so that you can outrank competition inside Google’s new discovery experiences. These pages usually need to updated to AMP formats, as that it what is powering all of these new discovery experiences inside Google search.

Many companies will attempt to opt out of this strategy, thinking it will help them preserve their current rankings if they delay or hurt Google’s ability to build a compelling, competitive aggregator to their own. This is unlikely to work. If there are any competitors to your aggregator, game theory will incentivize one of them to partner with Google to steal share. If you are a monopoly and opt out, it incentivizes Google to build a competitor that will threaten your monopoly, you will receive a lot less traffic that also threatens your monopoly in the interim, and Google can dedicate a lot of resources to a competitive product over many years. This appears to be working with Google Local vs. Yelp. Certain companies have been able to thrive despite these types of platform changes in the past by building loyal audiences with high switching costs, like Amazon with Amazon Prime when Google launched Google Shopping.


Google’s strategy has changed, though it will take years to propagate throughout every popular category of search queries. You can’t fault them for this change, as it is the right response to cater to their users. Right now is the right time to understand their strategy and to best position your company for its inevitable rollout. Gone are the days where you can rely on Google for a steady stream of free customers without putting in that much effort. You need to think strategically about where the company is going, if there are still opportunities where your content can attract Google visitors, and how you maximize the now declining opportunity.

Thanks to Randy Befumo for helping me through an early draft of this.

Currently listening to Challenge Me Foolish by μ-ziq.

Addressing Common Misconceptions about Food Delivery Marketplaces

I spent five and a half years working at Grubhub, from series A to right before IPO. This allowed me to learn about many of the intricacies of the restaurant market and food delivery in general. More people have started to take notice of the market because of a slew of market entrants and Grubhub ($9.1B), Just Eat ($4.8B), and Delivery Hero ($7.2B) being successful on the public markets. With that, more articles in the press. Articles… that are wrong. I am reminded of the Murray Gell-Mann Amnesia Effect, invented by Michael Crichton, when I read these articles. It says:

Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them.

In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.

As someone who does know, I want to explain some of these misconceptions, so people don’t think wet streets cause rain (even though rain does cause delivery orders).

Misconception #1: Restaurant Margins
One major gripe journalists cite about food delivery marketplaces (more so UberEats than Grubhub due to its higher fees) is that restaurants operate on slim margins. Therefore, if food delivery marketplaces are charging 15-30% for delivery orders, restaurants are not making any money. The issue is understanding the difference between restaurant margins and delivery margins, which are very different.

Most successful marketplaces are built on top of an under-utilized fixed asset. For food delivery marketplaces, this under-utilized fixed asset is not the restaurant, but the kitchen. Restaurants have a fixed capacity they can seat at a restaurant. The kitchen, however, is usually capable of producing much more food on a daily basis than is needed by the patrons that dine in the restaurants. Restaurants are paying for that kitchen capacity regardless of how much they use because one of the highest costs for most restaurants in cities is the cost of rent. That’s why I thought it was so silly when all of these delivery service startups started making their own food. You’re spending a lot of money to build what the incumbent gets for free: excess kitchen capacity to make food. This is why restaurants love catering orders so much. They get big orders that can better leverage their kitchen capacity. After catering, their next favorite is delivery.

Why delivery? It allows them to serve many more customers at a time with their fixed asset, spreading their fixed costs across many more customers. Catering and delivery are pretty much pure margin for restaurants because their only extra cost is a delivery driver (or not, in many cases) who is subsidized by the people ordering the food via tips.

Misconception #2: Paying for Repeat Orders
The second major gripe I hear about food delivery marketplaces is that they charge the same amount for a customer’s first order to a restaurant and repeat orders. Now, a lot of this is drummed up by a competitor who does not drive demand, so it is biased, but I’ll endear it. The misconception here is that all a restaurant has to do is pay an advertising fee to induce trial, and if the food is good, the customer will order again based on that experience. That is now how food delivery works. Food delivery is very fragmented, and while there is differentiation by way of restaurant quality, there are usually quite a few worthy substitutes. Also, the way delivery orderers usually make decisions is method first, restaurant last. The way the average person decides to use Grubhub operates something like this (flowchart):

Restaurant loyalty is one of the least important and last steps in the process of the person ordering food. This means people ordering food do not have loyalty, and you need to compete for every order as if it’s the first. Google Adwords does not charge Apartments.com less if someone who clicked their ad a year ago when searching “apartments” does so again the following year. The reason they don’t is because if Apartments.com wasn’t showing up there, that user would have gladly clicked the ad for ApartmentGuide. Restaurants should absolutely be working to build loyalty, and some do. But expecting that acquisition was the hard work, and restaurants should only pay a SAAS-level fee for retention does not align to the value these marketplaces actually create for restaurants.

Changes That Do Make Sense
Now, there are some elements I would change in regard to repeat orders if I worked at these marketplaces. While Grubhub already charges a much cheaper rate if the order originates from a restaurant’s own website, there are other forms of orders that the restaurant seemingly drives itself without the marketplace’s marketing engine or aggregation. One is if the person ordering food directly types the restaurant’s name into the marketplace. Charging a lower rate for that makes sense as the restaurant is clearly driving the business. I know from the data that this is a small percentage of orders, but this would show good faith to restaurants that marketplaces want to align their revenue model to the value they create.

Another scenario is if the person lands directly on the restaurant’s page on the marketplace from somewhere else e.g. Google. If this happens organically (because the marketplace ranks high or because the restaurant does not have its own website), the marketplace should charge a lower rate. The reality is Yelp and Google Local take 90% of this traffic, but again, it would send the right message.

The other example of this is a little harder to parse. These marketplaces also bid on restaurant names on Google. If that drives an online order, what should be the charge? The restaurant drove the demand, but the marketplace spent the advertising dollars to close the order. In this case, I think these marketplaces should evolve to asking if restaurants want this type of marketing, and if so, charge for the advertising as a service. This was not possible for Grubhub when I worked there due to game theory issues with all the competitors, but with a shrinking field of credible players, it may be possible.

Align Revenue to the Value You Create

“We want to create more value than we capture.”*

Tim Kendall, the former President of Pinterest, repeated those words at an all hands to describe our strategy for monetization a few years ago. My role as an advisor to Greylock’s portfolio companies allows me to work with many different types of businesses: consumer social, marketplaces, SaaS, etc. I’ve come to realize this saying describes an optimal strategy for a lot more than just an ad-supported revenue model. It should actually be the guiding light for most subscription software businesses.

Align Revenue To The Value You Create
One of the most common questions I receive from subscription businesses is when to ask for a signup and when to start charging customers. In freemium businesses, the slightly different question is how aggressively you upsell the paid product, and how good you make the free product. If you talk to entrepreneurs, you will get definitive answers from them, but they are frequently the opposite of each other. “You should never give away your product for free!” “You’ll never succeed without a free trial!” “Ask for credit card upfront! People won’t take the product seriously.” “Never ask for a credit card upfront! You’ll shoo too many people away.” The default answer I gave to entrepreneurs after hearing all of this feedback is that it depends on the business and needs to be tested.

As I researched more into the problem, these questions actually seemed to be the wrong questions to be asking. Harkening back to Tim Kendall’s advice, I started asking entrepreneurs, “What is the path to actually creating value from your service for your customers? How long does it take, and what actions need to be accomplished?” In other words, very similar advice to what is a successful onboarding? Once you learn that, you can determine how to capture some of the value you create.

Capture Value For The Business After Value Has Been Created For The Customer
When your product is subscription based, the prime time to ask for a subscription is after a successful onboarding occurs. It frequently is based on usage, not time. Dropbox is a famous example. The product is free up to a certain amount of storage. Once a user hits that amount of storage, they cannot add more files to Dropbox without paying. This storage amount also happens to be around the point where Dropbox becomes a habit, and represents real switching costs to find another way to share files across devices. So their conversion rates to paid are very high without any sort of time-based trial period. They don’t have a free product and a paid product; they have a free introduction to their paid product, and it becomes paid as soon as value has been created for the customer.

Your company may not have a long time to demonstrate value though, which may force your product to change to display (and capture) value more quickly. For startups based on search engine traffic, people reach your page with intent at that moment, and you frequently learn that this initial session is your only chance to convert them. So you push for a signup during that session after showing a preview of the value you can provide.

That is what we implemented at Pinterest, and it worked well, but it definitely created backlash from users for whom we had not yet created enough value. Once Pinterest was relevant on search engines for multiple topics, we saw people come back multiple times, and pulled back the signup walls on first visit. At that point, Pinterest was confident users would come back and thus focused on demonstrating more value before asking for signup.

Don’t Try To Capture Value In A Way That Reduces Value Created
It’s interesting to map the revenue growth of Dropbox to Evernote over the same time period. Evernote allows you to store an unlimited number of files and only makes you pay for advanced features like offline storage, storing large files, and (later) sharing on more than two devices. These features would have actually increased value created and switching costs if they were free, because Evernote’s value prop is about being able to access notes everywhere. If Evernote had instead mined their data and seen that people stick around after, say 50 notes, that would probably have had more effective monetization.

You only want to hide features from free users if they do not create habits or virality. Hiding sharing functionality before payment never makes sense because it introduces more people to the product for free. Hiding functionality that helps create retention also doesn’t make sense because you can always upsell retained users, but you can never upsell users who did not see the value and therefore don’t come back.

Decreasing Churn Is Long Term More Important Than Maximizing Conversion
Many people will decry that this strategy actually reduces revenue. In the short term, this sentiment is likely to be true. Decreasing churn might have a lower conversion rate upfront, but it aligns to long term successful retention. Churn rate is usually one of the biggest barriers to long term growth, so it’s worth thinking about this type of strategy even if it has a short-term decrease in revenue. It can be much harder to re-acquire someone after they have canceled, than charge someone for the first time who has been receiving regular value because you charged them for value you didn’t create.

What usually happens when a company captures more value than they create is they will have high revenue growth for a period of time (with a lot of investor enthusiasm), followed by a flattening of growth and then a steep revenue decline. This happens because revenue growth is a lagging indicator. Usage growth is the leading indicator. When usage lags revenue, this predicts churn. As you churn more and more users, it becomes harder and harder (and eventually impossible) to replace those churned users with new users to keep revenue metrics flat. Look at Blue Apron’s valuation to see this playing out currently as subscribers start to decrease for the first time year over year.

You Want Your Revenue Model To Align As Closely As Possible To The Value You Create
Lastly, as you start charging customers to capture value you create, you want your business model to align to the value that is being created. Email marketing tools have mastered this. Email marketing tools’ value is based on reaching customers with messages. Most email marketing tools charge on a CPM (i.e. a price for every thousand emails you send via their platform). As your email volume increases, they continue to drop the CPM. This make these companies more money because customers are sending a lot more email over time. But it actually becomes more valuable to the customer as well, because email is now cheaper on a per unit basis to send.

Compare this to Mixpanel, a product analytics tool. Mixpanel charges per event, and their value is delivering insights based on data from events being logged on your website or mobile app. The more events that are tracked in Mixpanel, the more insights the customer can receive, and the stickier the product. Since Mixpanel is charging per event though, a weird calculus emerges for the customer. The customer has to ask if tracking this event is worth the cost because not all events are created equal. Meaning the customer has to decide which data is important before they use the product. So, Mixpanel’s revenue model actually hurts its product value.

— 

It’s easy for subscription businesses to get attracted to the allure of short term revenue. The goal of your business is first to create value. The creation of that value and the understanding of how it’s created allow for more optimal and sustainable revenue generation opportunities. Don’t pursue short term revenue opportunities that prevent the customer from understanding the value your company creates. When you are generating revenue, you want to align that revenue model to how value is created for your customer. If you’re not sure, err on the side of creating more value than you capture rather than the opposite. This leads to long term retention and the maximization of revenue.

Naomi Ionita, General Partner at Menlo Ventures and former growth leader at Invoice2go and Evernote, and I talk more about this and other subscription growth problems in the Greymatter podcast.

*This quote I believe originally stems from Brian Erwin.

Currently listening to Shape the Future by Nightmares on Wax.