Why Onboarding is the Most Crucial Part of Your Growth Strategy

When people talk about growth, they usually assume the discussion is about getting more people to your product. When we really dig into growth problems, we often see that enough people are actually coming to the products. The real growth problems start when people land… and leave. They don’t stick. This is an onboarding problem, and it’s often the biggest weakness for startups. It can also take the longest to make meaningful improvements when compared to other parts of the growth funnel.

In my role as Growth Advisor-in-Residence at Greylock, I talk to startups in the portfolio about getting new users to stick around. Through many failed experiments and long conversations poring over data and research, I have learned some fundamental truths about onboarding. I hope this can function as a guide for anyone tackling this problem at their company.

What is Successful Onboarding?
Before you can fix your onboarding efforts, you need to define what successful onboarding is to you. What does it mean to have someone habitually using your product? Only then can you measure how successful you are at onboarding them. To do so, you need to answer two questions:

  • What is your frequency target? (How often should we expect the user to receive value?)
  • What is your key action? (The action signifies the user is receiving enough value to remain engaged)

To benchmark frequency, look at offline analogs. At Grubhub, we determined how often people ordered delivery by calling restaurants. The answer was once or twice a month, so we used a “once a month” as a benchmark for normal frequency for Grubhub. At Pinterest, the analog was a little harder to determine, but using Pinterest was most like browsing a magazine, which people read weekly or monthly. So we started with monthly, and now they look at weekly metrics.

Identifying the key action can be easy or hard — it depends on your business. At Grubhub, it was pretty easy to determine. You only received value if you ordered food, so we looked at if you placed a second order. At Pinterest, this was a little harder to determine. People derive value from Pinterest in different ways, from browsing lots of images to saving images to clicking through to the source of content. Eventually, we settled on saving (pinning an image to your board), because, while people can get value from browsing or clicking through on something, we weren’t sure if it was satisfying. You only save things if you like them.

Once you know your key action and your frequency target, you have to track that target over time. You should be able to draw a line of all users who sign up during a specific period, and measure if they do the key action within the frequency target after signup. For products with product/market fit, the line flattens as a percentage of the users complete the key action every period:

If the line flattens rather quickly, your successful activation metric is people who are still doing [key action] at [set interval] at [this period after signup]. So, for Pinterest, that was weekly savers four weeks after signup. If your cohort takes a longer time to flatten, you measure a leading indicator. At Grubhub, the leading indicator was a second order within thirty days of first order.

How should you research onboarding?
You can break down cohort curve above into two sections. The part above where the curve flattens are people who “churn”, — or did not receive enough value to make the product a habit. The people below where the curve flattens have been successfully onboarded.

To research onboarding, talk to both groups of people to get their thoughts. I like to do a mix of surveys, phone calls, and qualitative research using the product. I usually start with phone calls to see what I can learn from churners and activators. Our partner Josh Elman talks about best practices to speaking with churners, or bouncebacks. If I am able to glean themes from those conversations, I can survey the broader group of churners and activators to quantify the reasons for success and failure to see which are most common. (Sidenote: You’ll need to incentivize both groups to share their thoughts with you. For those that didn’t successfully activate, give them something of value for their time, like an Amazon gift card or money. For those that did, you may be able to give them something free in your product.)

But it is not enough to just talk to people who already have activated or churned. You also want to watch the process as it’s happening to understand it deeper. In this case, at Pinterest, we brought in users and watched them sign up for the product and go through the initial experience. When we needed to learn about this internationally, we flew out to Brazil, France, Germany and other countries to watch people try to sign up for the product there. This was the most illuminating part of the research, because you see the struggle or success in real time and can probe it with questions. Seeing the friction of international users first hand allowed us to understand it deeper and focus our product efforts on removing that friction.

The principles of successful onboarding
#1: Get to product value as fast as possible — but not faster
A lot of companies have a “cold start problem” — that is, they start the user in an empty state where the product doesn’t work until the user does something. This frequently leaves users confused as to what to do. If we know a successful onboarding experience leads to the key action adopted at the target frequency, we can focus on best practices to maximize the number of people who reach that point.

The first principle we learned at Pinterest is that we should get people to the core product as fast as possible — but not faster. What that means is that you should only ask the user for the minimum amount of information you need to get them to the valuable experience. Grubhub needs to know your address. Pinterest needs to know what topics you care about so they can show you a full feed of ideas.

You should also reinforce this value outside the product. When we first started sending emails to new users at Pinterest, we sent them education on the features of Pinterest. When Trevor Pels took a deeper look at this area, he changed the emails to deliver on the value we promised in the first experience, instead of telling users what we thought was important about the product. This shift increased activation rates. And once the core value is reinforced, you can actually introduce more friction to deepen the value created. When web signups clicked on this content on their mobile devices, we asked them to get the app, and because they were now confident in the value, they did get the app. Conversely, sending an email asking users to get the app alone led to more unsubscribes than app downloads.
Many people will use this principle as a way to refute any attempts to add extra steps into the signup or onboarding process. This can be a mistake. If you make it clear to the user why you are asking them for a piece of information and why it will be valuable to them, you can actually increase activation rate because it increases confidence in the value to be delivered, and more actual value is delivered later on.

Principle #2: Remove all friction that distracts the user from experiencing product value
Retention is driven by a maniacal focus on the core product experience. That is more likely to mean reducing friction in the product than adding features to it. New users are not like existing users. They are trying to understand the basics of how to use a product and what to do next. You have built features for existing users that already understand the basics and now want more value. New users not only don’t need those yet; including them makes it harder to understand the basics. So, a key element of successful onboarding is removing everything but the basics of the product until those basics are understood. At Pinterest, this meant removing descriptions underneath Pins as well as who Pinned the item, because the core product value had to do with finding images you liked, and removing descriptions and social attribution allowed news users to see more images in the feed.

Principle #3: Don’t be afraid to educate contextually
There’s a quote popular in Silicon Valley that says if your design requires education, it’s a bad design. It sounds smart, but its actually dangerous. Product education frequently helps users understand how to get value out of a product and create long term engagement. While you should always be striving for a design that doesn’t need explanation, you should not be afraid to educate if it helps in this way.

There are right and wrong ways to educate users. The wrong way: show five or six screens when users open the app to explain how to do everything — or even worse, show a video. This is generally not very effective. The right way: contextually explain to the user what they could do next on the current screen. At Pinterest, when people landed on the home feed for the first time, we told them they could scroll to see more content. When they stopped, we told them they could click on content for a closer look. When they clicked on a piece of content, we told them they could save it or click through to the source of the content. All of it was only surfaced when it was contextually relevant.

Onboarding is both the most difficult and ultimately most rewarding part of the funnel to improve to increase a company’s growth. And it’s where most companies fall short. By focusing on your onboarding, you can delight users more often and be more confident exposing your product to more people. For more advice on onboarding, please read Scott Belsky’s excellent article on the first mile of product.

Currently listening to Easy Pieces by Latedeuster.

Starting and Scaling Marketplaces Podcast

Brian Rothenberg, VP & GM at Eventbrite, and I discuss how to start and scale marketplaces. We discuss certain topics such as the chicken and egg problem, going horizontal vs. vertical at the beginning, and traditional and non-traditional growth tactics to grow marketplaces. You can check it out below or read the summary here.

The iTunes link is here, and here is the Soundcloud link for email readers.

The Right Way to Involve a Qualitative Research Team

Most teams significantly under-invest in qualitative research. Growth teams especially are all about data, but they think that data can only come from experiments. This can make teams overly reliant on what they can learn from experiments and the quality of the data they have, and under-invest from what they can learn from talking to users. This problem is usually exacerbated by the fact that existing researchers at startups aren’t usually assigned directly to teams or work independently. I’ll talk about some of the problems I’ve seen, and the right way to invest in qualitative research for your growth team.

Learning and Applying from Research
Using the right type and method for your question is key. Of course, qualitative research is one component of the research stack along with quantitative research and market research. There is also different types of qualitative research depending on what you are trying to learn.

I remember when I was at Apartments.com and went to my first focus group, a common type of qualitative research. It was a mess for multiple reasons. The first reason was structure. Finding an apartment is not a large social behavior, so why were we talking with a group of ten strangers at once? As what I later learned usually happened, one or two participants volunteered the majority of the feedback, so while we paid for ten people’s opinions, we really only received two people’s opinions. So, I now only do research with multiple people in the room if it’s a social product, and it’s a group that would use it togethers e.g. friends or co-workers.

The second issue was delivering the feedback to people who weren’t there. I wrote up a long perspective on what the issues were with Apartments.com vs. our competitors. It primarily included product feedback on why we were getting crushed by Craigslist in major cities. I sent it to my VP and received a one sentence reply, “Don’t get ahead of yourself.” What a waste of time, I thought. We do all this research, generate real insights, and no one’s interested.

I’ve now learned that research teams inside companies feel this every day. At Pinterest, we had an amazing research team, but they were originally a functional team, which meant they had to determine their own roadmap of what to research. Depending on the stakeholders you listen to, this can be broad strategic projects like “What is the deal with men?” to specific projects like “Help us test this new search flow already built.” Research can add value at both stages, so the team worked on both.

What I think research found when they worked on the broader strategic issues was similar to my response at Apartments.com. “Cool, but not my roadmap!” say the product managers. Research then gets filed away never to be looked at again. Researchers get very frustrated. To be clear, this is a failure of leadership — not the product teams — if these areas aren’t prioritized. But it is common. On the flipside of working on something already built, success was more variable based on how well the product team defined what they wanted to learn. Frequently, what the product team wanted to learn was that they could ship it, so they selectively listened to feedback to things that indicated they were on the right path.

What I have learned suggests that qualitative research cannot be effective unless 1) its people are dedicated a cross-functional product team and 2) research is involved throughout the entire product development process, from initial research on market to determining a strategy to testing concepts to testing nearly finished products. The value of research accrues the more it is a part of each step in the process.

This approach solves for two main problems. One is that product teams will only pay attention to feedback that is directly related to their current product and on their own timeline. Without being part of the cross-functional team that includes product, engineering, and design, it is hard for research to to be on the same timeline. The second problem this solves is it helps research prevent the rest of the team from locking on assumptions that they may be wrong, so they are focused on the right solution to the problem with research, instead of confirmation bias at the end of a project. The Pinterest team has moved to this model, and for my teams, it made both sides much more successful.

When to Research and When to Experiment
For teams that rely too much on experiments and not enough on research, I tell them two things:

  • Experiments are great for understanding what people do and don’t do. Research helps you understand why they do or do not do those things
  • Experiments don’t help you understand the under-represented groups that might be the most important to learn from e.g. non-users or smaller segments of users

A great way to get started with research as a team is to answer why your experiment didn’t work. Sometimes, the answer is there in the experiment data, but frequently it is not. You have to talk to users to understand why they are doing what they are doing. The best way to do that is to ask them the context of them doing or not doing it.

There is also the middle ground of quantitative research that can be helpful (usually surveys). What I usually like to do is use qualitative research to understand the universe of reasons for something, and use quantitative research if I need to quantify the importance/commonality of those reasons.

Research also helps you isolate users you may not be able to isolate with your usage data. For example, at Grubhub, we were trying to understand how many people used Grubhub regularly for delivery, but not for every order. So, we asked. Then, we called those users to understand why they sometimes don’t use Grubhub, then sent another survey with those reasons to quantify which ones were most important to address. I outline that process more here.

But I Don’t Even Have a Research Team
At Grubhub, we didn’t have a research team for the first couple of years (or even a product team for that matter). So, when we needed to learn things, me, someone on my team, or our sole designer (hello Jack!) would do one of three things: 1) throw flows up on usertesting.com, 2) survey users on our email list, or 3) call users on the phone, and provide them with free food for their time.

You don’t need to be a professional researcher to do this, though they are better at it. You just need to determine what you’re trying to learn and who from. You want to watch people go through that situation if you can. If you can’t, ask them about the last time it happened and what they did and why. You will get better at it the more times you do it. Startups are starting to hire researchers earlier in their development because of the importance of understanding users beyond the data. So, you may be able to justify a full time role here earlier than you thought.

Thanks to Gabe Trionfi for reading early drafts of this and providing his feedback. HeHAH!

Currently listening to Beyond Serious by Bibio.

You Are Not Your Customer

Startups are successful in the early days usually for one of two reasons. One is having a unique insight or pain point in the world that you want to solve (usually for yourself as the founder first), and assuming it is pain experienced by others. The other is to listen to customers, deliver value to those customers, and make sure they understand and appreciate the value you’re providing. The second way requires founders to hone specific skills in the early days of a startup — which can actually make it harder to scale out of the early stage, but pays off with sustainable growth in the long term.

The people that try your product early on see the potential of your product and are willing to forgive flaws — at least for a while. They have done an incredible amount of work to make the product work for them. By most definitions, they become “power users.” These power users are heavily engaged with your product, but they also deliver a ton of feedback on how the product could be better for themselves.

The early employees of your company tend to be very similar to your early customers. They (hopefully) use the product quite a bit, and joined the company because they understood the long term vision. These employees then start recommending and building products for themselves also, especially in consumer businesses. Everyone is excited to build these features because employees want them and existing customers want them, so the company builds them. The features get built, and there is no impact on growth of the business. Our partner Sarah Tavel talks about this in her lessons from scaling Pinterest.

Why is that a problem? Well, in an old Quora question someone asked, “What are some of the most important things you’ve learned in marketing?”, and my reply was “You are not your customer.” As a company employee, even if you look exactly like the early customer, and you built the product for people exactly like you, you have way too much domain knowledge to truly represent the long term customer. Your early users are also no longer the customer. Both employees and early users have have built up too much domain knowledge.

Your customer focus should always be on new or potential users, not early users. Early users will bias experiments, prompt you to build more and more niche features, and stunt growth. Power users can’t be much more engaged, so building more things for them doesn’t usually help the business. It does, however, make the product harder to understand for new customers. Sure, you have to do enough to keep these power users happy enough to stay, but the much more daunting and important task is to find new people to delight, or to figure out how to delight people who weren’t initially delighted by your product.

This post originally appeared on the Greylock blog.

Currently listening to Ambivert Tools Volume One by Lone.

Solving for Snapchat’s Declining User Growth: A New Podcast

Julie Zhou, former director of growth at Yik Yak, and I spent some time discussing Snapchat’s declining user growth now that it is public, what its causes might be, and what we’d tried to do if we were in charging of improving it. You can check it out below or read the summary here.

The iTunes link is here, and here is the Soundcloud link for email readers.

Currently listening to Chnoiseries Pt. 3 by Onra.

The Death Spiral of Startup CMOs

I’ve met with a lot of CMO/VP Marketing types at startups. Generally speaking, they do not have a long shelf life. I was curious why so many startup CMOs leave the company after a short period of time, so I dug in a bit to figure out why. I found that it’s partially due to bad hiring practices for startup executives (which I have previously blogged about), but even more common was that big brand traditional marketers have a tough time “scaling down” into the scrappy, digital-first world of startups tactically.

The traditional marketing executive is more likely to be a brand marketer from a large company than an online marketing or growth person from a startup. These marketers have built careers deploying big budgets and leveraging a few channels that really matter to tell the brand story. The main one of course is still TV. TV is great for brand building, but startups cannot wait for multiple years of investment in brand advertising to start to pay off. They typically will run out of cash before they start seeing a return on the investment in TV. (This problem is particularly acute if you’re a startup that is live only in a few cities and gradually expanding across the country — i.e. most marketplaces.)

How does a marketing executive get a quicker read on effectiveness, especially if their startup doesn’t cover the entire country? One way I’ve seen marketers try to solve this problem quickly is to buy local TV brand advertising in a test market and compare to a control test market where they have not bought TV advertising. They measure if brand metrics improve in the paid market during a three month test. If brand metrics improve, it should be leading indicators for sales to improve.

On paper, this makes sense, but in practice, it is the first trigger of the CMO death spiral. The problem is twofold. First, the cost of local TV is 10x more expensive than national advertising. That premium is hard to make up. Second, TV brand advertising is generally not very effective and even harder to drive quick results. With a typical brand-based buy, ads will run at many different times during many different, popular shows. So even though many people will be watching and thus exposed to the brand, the viewers’ focus is on the show and not on a call to action.

In short, big brand marketing executives have used large budgets for a test and gained small bits of exposure for the company. That exposure usually doesn’t transit into brand lift or sales, and the founders quickly lose confidence in the CMO. From there, I’ve seen it is a matter of months before they either get fired or lose their budget, which will make them leave.

But its not all fire and brimstone! There are savvy ways for marketers with big brand backgrounds to be successful at startups. For one thing, start by testing national, direct response TV. This type of ad buy targets people who are watching TV because they are bored — not because they are super engaged with the show. In this scenario, ads run at off-peak times or on less popular networks and only cover a small percentage of the country. You get the data on the exact times and areas where the ads run. Then, you can watch for an immediate response in website/app traffic right after the ad, and track those users to see what they end up doing. Agencies that sell direct response typically offer data science services to measure the impact, and you can augment their reports with tools like Convertro or C3 Metrics that do it.

Grubhub and Seamless are both good on-demand startup case studies before they merged: Seamless bought national, direct response TV ads while the service only covered 20% of the U.S. population. Grubhub did it with only 40% coverage. Seamless then used all the website traffic data outside their coverage areas to determine where to launch next. At Grubhub, we saw a response immediately during some of these ads airing, and could actually calculate not just brand lift surveys, but CPAs due to transactions.

A broader lesson here is that traditional target marketing can be expensive as the people selling advertising have learned that traditional marketers will pay a premium for it. And while that math still works for a CMO of a CPG brand, it won’t typically work for a startup.

What is the the best way for a CMO to be successful and stick with your company for the long term? First, they need to find ways to reach the target market they want to reach in a cost effective way. (I go deeper into this topic on my post on remnant inventory.) Seamless reached an insane amount of people in New York with DRTV ads, so it didn’t matter that some people in Boise saw them, too. Secondly, CMOs need to work on other ways to add value to the business besides telling big brand stories. Depending on the business, that can include understanding customers better, building communities, performance marketing, or product driven growth.

Currently listening to Children of Alice by Children of Alice.

The Real Value of PR for Startups

Many startups get obsessed with press. Not just CEOs, but also employees, friends of employees, and investors. Press, like paid acquisition, can be a drug though. I’ll talk about some of the ways the startup ecosystem perceives press is bad, what it is actually good for, and some outliers.

Press Abuse
First, let’s talk about some of the abuse of press by startups in the past. Business insider has a great recap of some of the issues Evernote faced in the couple years before their CEO stepped down. I’ll highlight this quote in particular.

“There was a feeling that we were working on the wrong priorities,” a former employee said. “It was clear the motive was to just continually drum up press. They had no idea how to optimize and improve growth.”

From The inside story of how $1 billion Evernote went from Silicon Valley darling to deep trouble

Startups frequently correlate press with growth, and think that they need to create new stories to drive growth. Then they start changing their entire product roadmap to drive press instead of value to customers. Startups should not look for PR to drive growth. PR doesn’t usually drive growth. And when it does, it’s a bump, not an engine. In startups, you want to build growth engines.

This is arguably a better problem than the second way PR drives bad acting in the ecosystem: founders getting addicted to the attention of press. It feels great when there’s a picture of you in the Wall Street Journal and your mom sees it and your dad’s friends. And founders frequently want to create that feeling. It’s as if appearing successful is more important than being successful. This can be a dangerous pattern.

Another problem with press is that it helps competitors understand exactly what you’re doing. I’ve talked in the past of the advantage of being a silent killer and why seeking press attracts unneeded competitor attention. I won’t repeat that here.

What Press Is Good For
A common refrain I tell startups is that press is good for three things: investor interest, employee interest, and links for SEO. I’ll break those down a bit, and talk about why they matter. First, let’s talk about investors. Very few businesses make sense for venture capital, but when they do, they increasingly need larger amounts of it. The venture business has extreme cases of FOMO. One deal can make all the difference. So investors rely on a lot of signals, and the tech press is one important one they look at not only to find new business to research, but to also get comfort in their interest in certain businesses. When Grubhub was raising its Series C from Benchmark, we got written up about our iPhone app on Techcrunch. For Grubhub’s growth, it didn’t matter at all. But it looked great for our fundraising.

Likewise with investors, potential employees rely on press to hear about potential new places of employment and to feel comfortable they are making a good move. Recruiting outreach will have a lower conversion rate if the person has never heard of your company, or if the person’s network hasn’t heard of you when they ask around. This can be overcome, especially if you’re well connected or can tout amazing metrics privately (like WhatsApp), but my guess is WhatsApp always had a harder time recruiting than Snapchat.

Lastly, press is very helpful for SEO. SEO is about relevance and authority, and links from press are a great way to increase authority in the eyes of search engines. Not only can press drive a quantity of links, but the quality of links are probably the highest most startups will receive. So many people link to news publications that they have among the highest domain authority on the internet. Now, getting one of these links won’t make you rank #1, but it’s part of a healthy SEO strategy.

When Can Press Be an Engine of Growth?
Now that I’ve set an argument that press doesn’t usually drive growth directly, I want to talk about some outliers of when it does. Press can drive growth if a certain story about your startup is self-perpetuating. Then, the volume of articles written about you happens in such a regular cadence that readers eventually act on it. I have seen this happen in two major startups over the last ten years in Twitter and Uber. With Twitter, it received such a tremendous uptake from journalists that they kept writing stories about it. This drove their readers to try it, and it became a sustained vehicle for organic acquisition (unfortunately with a low activation rate, but that isn’t press’s fault, but the product). Uber’s growth via press was a little more sculpted. They were able to create a story about the business of Uber fighting against corrupt incumbents and governments that were preventing a better product from succeeding. It was almost a movement. For years this created a headline about Uber almost every week as they faced this resistance in almost every city in which they launched.

I want to highlight that these are anomalies, and it’s unlikely you will be able to create sustained growth directly from press in your company. But that doesn’t mean press is worthless. Just remember what it’s good for, and make it work for you for those interests.

Starting and Scaling a Growth Team

I get a lot of questions about how to start and scale growth teams. Growth teams need to spend as much time thinking about how to scale themselves as they do on scaling the product. In this post, I’ll outline how I’ve seen growth teams start and evolve, define the ideal end state, and describe some of what you work on at each stage along the way.

The first question to ask is why you need to start a growth team. Why not just have traditional product and marketing teams? It’s a good question, and not well understood. Traditionally, product teams make the product, and marketing is in charge of getting people to try and continue using the product. Marketing can include traditional efforts like events, advertising, and PR, and perhaps some branding efforts and some online marketing. Of note, online marketers and traditional marketers have historically been very different, so depending on the manager, all of the budget and team allocation typically went to either online or traditional — not evenly split.

However, the best startups grow super fast not because of traditional marketing or online marketing, but because they tune the product to drive growth. This is why growth teams matter. Marketing teams typically don’t have access to the product roadmap to make changes to things that will impact SEO, conversion, optimization, or virality, nor the expertise to work with engineers and designers on making these changes. And product managers often prioritize new product features over engineering that will drive more people to the existing value. Growth teams are the connective tissue between product, marketing and engineering.

Starting a Growth Team: Focus On One (Easier) Problem
Goal: Improve one area important to the growth of the business
Team: PM, Designer, Engineer(s), Analyst/Data Scientist (sometime PM or engineer assumes this role as well early on)

It can be overwhelming to consider all the problems you need to solve. When you start a growth team, do not to try to solve all those problems at once. A growth team should start by going deep on one carefully chosen problem:

  • Pick a problem the company is not currently working on (to prevent turf wars)
  • Pick one of the easier — not harder — problems to improve (to build up credibility and drive toward early wins)

Since growth teams are typically treated with a healthy dose of skepticism by other teams when they start, carefully choosing your first area of focus will help maximize the growth team’s chances for success. Let’s say a growth team starts with a hard problem: Trying to increase activation of new users. It takes a month to measure increases in activation rates, and it has one of the smaller sample sizes in the traditional growth area. You run an experiment. It doesn’t work. You run a second, and another month goes by, and it still may or may not work. Now two months have gone by without a growth team win. People on the growth team are questioning if this is the right team to be on, and other teams in the company begin to question your purpose. (Hat tip to Andy Johns from whom I am blatantly stealing this example).

There are common growth problems where you can likely show quick progress. One is conversion optimization. Another is returning users via email and notifications. Still another is improving referrals or virality. These areas allow you to run multiple experiments quickly because the metrics will move immediately (they don’t take time to measure like activation), and because they have high sample sizes (all new users or all existing users opted in to communication).

Last note: When starting a growth team, it’s important that they sit together and not with their functional team members. The collaboration between the different skill sets and short feedback loops is how the magic happens.

Growing a Growth Team: Own the Growth Funnel (And Some Real Estate)
Goal: Improve metrics across the growth funnel
Team: PMs, Designers, Engineers, Data Scentists/Analysts

As a growth team grows, the team can start looking at multiple problems. These problems are typically across the AARRR framework by Dave McClure, (however revenue is frequently a separate team in an ad supported business.) The growth team separates into sub-teams who have their own meetings, and entire growth team meetings become less frequent. PMs, designers, and analysts may work on multiple teams. At this point, the growth team assumes ownership of certain areas of the product, including (but not limited to):

  • The logged out experience, which can involve both SEO and conversion
  • All emails and notifications (sometimes coordinating with marketing)
  • The onboarding flow, sharing flows, et al.

Owning these areas creates easy swim lanes between teams and prevents turf wars with the core product team. Growth leaders have to determine how to allocate people to work on the most impactful problems across the growth stack, and if a sub-team does not have enough support to make an impact, the growth leader should consider whether to support that problem at all. Growth teams also have to own goal setting, which means understanding historical performance and setting absolute goals.

At this point, teams start to work on both improvements in infrastructure and experiments. For example, at Pinterest, I was managing both the acquisition and retention teams. The retention team was struggling to grow because of an unwieldy email infrastructure. The infrastructure was built to support triggered, social notifications, but our strategy had evolved to more personalized, content-based recommendations. These jobs were taking days to run and would send automatically when complete, even though someone might have received a social notification five minutes before. So we spent nine months rebuilding it to allow us to scale more effectively. The retention team saw a step change in its performance and actually hit its Q1 goal in the middle of January after it was complete.

Evolving a Growth Team: A Special Forces Team Operating Across Borders
Goal: Reduce friction across the product that prevents people from connecting to product’s core value
Team: PMs, Designers, Engineers, Data Scentists/Analysts, Researchers, Marketers, Operations

As the growth team continues to grow, it involves more stakeholders and expands its scope. Instead of having clear areas of focus separate from the core product, the team shifts to analyzing the entire product and trying to figure out the biggest obstacles that are preventing the company from growing faster. At this point, the growth team has built up enough credibility that it doesn’t fight turf wars — it is laser focused on finding and solving the biggest problems that prevent people from connecting to the current value of the product.

Usually, the problems at this stage are deep inside product features and go beyond improving SEO, signup, viral and on-boarding flows, and sending the right emails and notifications (though growth teams still work on those too). The growth team is now identifying the friction that prevents people connecting to the core value elsewhere in the product. A lot of this work is simplifying product flows or building country-specific optimizations for slower growth countries. Growth teams can also become service organizations to marketing initiatives around this time, and I’ll talk about that in another post.

At Pinterest, we migrated to this phase rather recently. As we started to look at friction in the entire product, we saw in qualitative research that new users were getting bombarded with so many concepts it confused them: what a Pin is, where they come from, how to add your own Pins, what a board is, group boards, profiles, etc. The growth team made a list of the core things new people needed to know to get excited and start to build an understanding of the service. Then, we started experiments to remove everything else from the new user experience and only introduce it back once the core concepts were clearly learned. This helped improve activation rates.

What I described above is an amalgam of what I’ve seen in the market that works best. Of course, you should always tweak your approach based on the talent of your team and your company’s culture. I would love to hear what you think of this approach, and any additional insights you think would be useful to share with other growth leaders.

This posted originally appeared on the Greylock blog.

Currently listening to 28 by Aoki Takamasa + Tujiko Noriko.

Three Mistakes Integrating Growth and Design Teams and How to Address Them

While I encourage growth teams to start with a dedicated designer, that doesn’t usually happen. Usually, growth teams scale with just engineering and product, and as they scale to a certain size start to eventually “earn” a dedicated designer from the organization. This news makes growth teams extremely happy for a short period of time, but pretty quickly starts to create problems and culture clash. I’ll talk about what happens, and some ways I’ve found to solve these issues. Note: You can replace growth with marketing, and just about the same thing happens in these scenarios.


When you hear you’re getting a dedicated designer, you’re happier than Jim Carrey was back when he had a career.

Problem #1: Team Control AKA The Two Scenarios
One of two scenarios usually emerges when designers join a growth team. In the first scenario, the engineers or PM or marketing person starts with, “I’m so glad you’re here. I need this done and this done and this done for an experiment…tomorrow. No, we don’t need research to understand the problem. Don’t worry. I’ll only ship it if the metrics increase.” The designer realizes the growth team didn’t want a designer. They wanted a pixel monkey to just do what they say, not ever use their brain.


I got 99 problems, but a designer using their brain ain’t one.

The other scenario is just as bad. In this scenario, I call it the designers “moving in”. I liken this to the scenario when a significant other moves into your place and brings way more things than you own to move in. “Don’t worry, I got better furniture and books, and, oh, we’re going to have to get rid of those curtains.”

The design version of this is something like “I’m going to completely rethink all our strategies. I need to spend three months just doing research and then at least double that for design concepts. That home page really needs to change though.” Engineering thinks “this isn’t what we need. These people don’t even know what they’re talking about.” Engineering gets kicked out of the process of figuring out what to work on, and since the design process has no deadlines on it, engineering begins to have nothing to work on.


I tried to find a picture of people dressed in all black moving in, but the best I could do was chambray.

In both scenarios, instead of designers, engineers, product managers, and marketers trying to unify to form one team that leverages all of their strengths, one team tries to dominate the direction. What we did at Pinterest to try to solve this problem was design a process where the teams, specifically design and engineering, are jointly responsible for problem definition and solutions. Here’s what it looks like:

Project Kickoff:
Attendees: PM (leader), engineering lead, design lead, engineer and designer that will be working on the project

Goals:

  • Define the problem we’re trying to solve
  • Any existing ideas on how we might solve it
  • Any previous attempts to solve the problem
  • What metrics we need to inform potential solutions
  • How will we tell if we’re successful

Output:

  • Notes emailed to broader sub-team
  • Slack channel created just for the project, which includes notes and all future communication for the project

Project Brainstorming:
Attendees: engineer and designer on the project, PM (optional so as to prevent their calendar from being a bottleneck)

Goals:

  • Produce multiple potential solutions to the defined problem
  • Prepare those for feedback from attendees of kickoff
  • Cover additional measurement needed for directions that have been chosen e.g. how many people click top right login button

Brainstorm Review:
Attendees: PM, engineering lead, design lead, engineer and designer working on the project (leaders)

Goals:

  • Feedback on concepts from brainstorm
  • Choose one or two directions for experiment

Experiment Launch:
Attendees: engineer and designer working on the project (leaders), PM

Output:

  • Experiment doc that team agrees reflects what we’re testing and why
  • QA and approval over Slack from every team member that experiment is ready to launch to 1%
  • Note: Ramp ups of experiment communicated in Slack channel

Experiment Review:
Attendees: PM (facilitator), engineering lead, design lead, engineer and designer that worked on the project (leaders)

Goals:

  • Determine if you gathered data on key questions you needed to answer for the experiment
  • If yes, what does the data say?
  • If not, how do we get the data?

Output:

  • Ship/kill/iterate decision
  • Emailed notes to the entire team of what happened and why
  • Updated experiment doc on what happened
  • Updated Slack channel
  • Designer brings ship experiment candidates to design manager/lead for any feedback before shipping

      • Problem #2: Design Best Practices
      Designers bring with them a history of best practices from working on other design teams and schooling/training. It can be a culture shock for a designer joining a growth team and seeing little of these being followed. Designers will usually respond to this environment by trying to educate on many things the growth team is managing are designed “wrong” and how they need to be changed. These recommendations based on best practices are usually rejected by engineers on the team as it contradicts what they’ve seen in the data.


      A growth engineer after he hears about a marketing or design best practice.

      Design best practices were created with good purpose, but they quickly become inferior to AB testing in an environment where a direction can be put in front of users and its value quantitatively determined in days or weeks. This doesn’t mean you should AB test every design decision, but it does mean a designer can’t put their foot down by saying something is a best practice. In growth, the users being tested on determine what’s best.

      Another issue with design best practices being applied to growth is that design best practices are usually suited entirely for building user value only. The growth team in many cases needs to trade off user value and business value, or short term user value vs. long term user value. For example, someone can come to my site, and I can offer a great experience. The user then leaves and never comes back. Growth might decide to let someone get a preview of that experience, then ask the user to sign up so they can personalize and deliver more value. The result is that some people will not sign up (creating a worse user experience) and some will sign up, allowing the site to create a better user experience that day and over the long term. This accrues value for the business as well.

      Designers usually feel threatened by this environment initially. They see testing as a threat to their expertise. What you have to do is teach them to use it as a tool to get closer to user feedback at scale and be more efficient with their time. Most design work is wasted because it is spent on a direction that later proves to be flawed. With AB testing, a design can get quick feedback on an idea, validate the direction, then spend the time making it amazing once they know it’s worth that effort.

      That said, while this user feedback at scale is good at showing what users do, it’s not great at explaining why. So, along with spending time getting designers comfortable with testing, growth teams need to start doing more regular qualitative research as well. Designers will usually volunteer to do it themselves if there isn’t the right person to staff for it. Some PM’s are comfortable doing it as well. Engineers and engineering managers can be resistant to spending time watching the sessions, so the first couple you schedule have to be on critical areas you feel there is some understanding the team is missing by only looking at the metrics.

      Once designers get a little more seasoned on growth, you should also work with them to create the new best practices for growth to make the design and engineering process move faster. They will look very different than what designers recommended at the beginning, and be backed by data. Along with this process, I think it’s important to start creating user experience goals for your growth team at this time. These will be components that may not affect the metrics, but ensure a quality and consistent user experience. At Pinterest, we made compliance to design spec a requirement to ship after an experiment is validated, and said our top five flows had to be completely audited for quality user experience. This is a worthy compromise with design to show you actually care about the users and not just the business, a common complaint.

      Problem #3: Growth Onboarding
      Sometimes, you want to make sure you just nail the basics. Most growth designers are joining a new team in a discipline they’ve never worked on before, yet they don’t receive any onboarding as to what growth is and why it’s different. This is also an issue with other new growth team members. It’s like being dropped into a foreign country, not knowing the language, having no one help you, and being expected to contribute to the culture. Like France (okay, I’ve actually been to France, and the people were surprisingly welcoming even though they’re not known for being so). What happens is designers get frustrated and churn from the team.

      It’s critical that every new team member, especially designers, go through a growth team onboarding process. The first thing you do is state the purpose for your growth team, why it’s different from other product teams, and why it’s important. At Pinterest, I would say that while other product teams create value or improve the value of the product, growth teams focus on connecting people to the current value of the product and reducing the friction that prevents that connection. This is important because it’s not enough to build a good product. If no one knows about it, it will die. If it’s too hard to uncover its value, it will also die.

      What we did at Pinterest was create a history of major growth projects that were successful, walked through them, and explained why they were successful. That led into some of our team principles. We also spent a lot of time educating new people on the metrics we use and why. You can’t be expected to design winning experiences if you don’t understand the major criterion for success we’ll be using.

      After onboarding, instead of starting designers on large, complicated projects, it’s important to start them on a well scoped, smaller project, preferably with an experienced PM and engineer, hopefully patient ones. These should still be projects that are worth doing, but not large projects. We had a designer start on growth at Pinterest, and we immediately put her on one of the most strategic, long term investments for the team. While she did great work there, after a few months, she did a smaller side project redesigning our mobile web home page. The conversion rate increased, and we shipped it. She said, “Guys, this is the first time my design increased the metrics!” She was beaming. You want to get new growth designers to that moment as soon as possible.

      Lastly, once onboarded, you want designers contributing growth ideas as soon as possible. I like the idea of forcing people to bring one new idea to the team per week. I believe this is a muscle that needs to be developed via practice. New entrants to the team (design or otherwise) will typically propose bad ideas for a while. That’s okay. The trick is to provide a framework for generating ideas that makes new team members think about elements that typically make for good growth ideas, give feedback on the ideas submitted, and have the ideas be submitted as metrics wins or user experience wins. The Pinterest template focused on :

      • How many people would see this experience if it were built? This is usually by far the most important criterion for a successful growth idea. Any experience has to be seen by a lot of people to have a big possible impact on the business.
      • Has the company tried something in this area before? If so what were the results? This helps us make sure we use any previous learnings to avoid making the same mistakes. Just because we tried something before doesn’t mean it’s a bad idea though as growth environments change frequently.
      • How much effort is required for this idea? On growth, we also try to do the least amount of work to validate an idea is worth it.

      There is no reason designers can’t be key contributors to a growth team, but expecting it to happen automatically is usually a recipe for failure. I hope some of these tips can help you create a thriving cross-functional growth team with all the right disciplines involved. If you’ve have any other issues integrating design into your team, I’d love to hear about them in the comments.

      Currently listening to Sorry I Make You Lush by Wagon Christ.