Tag Archives: team building

Product Visionary vs. Product Leader

Many people want to work in product management. One of the most common questions I receive is how to break into product management. It’s a hard question for me to answer, because 1) there is no default path (the same is true for trying to land a business development role), and 2) most of these people really don’t know what they’re asking for. My most common response is, “Are you sure? Product management can kind of suck.” The reason for the dichotomy of people who haven’t done product management finding it so alluring, and people who have done it cautioning people trying to get in is the difference between what I call a product visionary and the product leader.

Product visionaries are who we all hear about in the press. They are the people who come up with brilliant products that go viral or solve real needs in the market that no one else thought of. They appear to be masters of finding product/market fit. I’ve been lucky enough to work with a few of them in my previous jobs and a few in the portfolio at Greylock. These tend to be founder/CEOs, and they generate brilliant insights that create product opportunities others don’t see. Ev Williams is on his third breakout product in Medium after Blogger and Twitter. Ben Rubin created two products that hit product/market fit in two years with Meerkat and now Houseparty.

Anyone working in technology hears these stories, and they think the shortest path to that sort of glory is becoming a product manager. They are excited to get to a role where they can drive the vision of a product, even if it’s only one part of the company. This excitement is exacerbated by the commonly propagated myth that the product manager is the mini-CEO of their product. The reality is that in 90+% of cases, product management is not about being a visionary. It’s about being a leader.

What does a product leader do at a tech company? It’s actually very little of creating a vision and a strategy from scratch. It’s about helping everyone understand what the vision and strategy is. It’s about communicating to the entire team why the company is doing what it is doing. It’s about building a process that helps a team execute on that vision. It’s about when there are competing visions, aligning and motivating the team to focus on one, and getting people to disagree and commit (including sometimes yourself). It’s about looking at data to measure if product changes are having a positive impact on the customer and the company’s growth. It’s about talking to users to understand why they’re doing what they’re doing, and the problems they still face even though your product exists. It’s about mentoring more junior people on your team, across product as well as engineering, design, and analytics. And they don’t have to listen to you, so you have to use influence rather than authority to be successful.

In the scaling phase of a startup, it’s product leadership that drives performance, not vision. Vision is needed early to find product/market fit and plot a course to scale, and then the less that vision wavers over time the better. This vision is usually done by the founder/CEO. The reason founders hire product managers and VPs of Product is not to set vision, but to help execute the vision. Don’t get me wrong; that will sometimes mean coming up with solutions to problems your customers face. If a company is scaling by having the founders solve all the customers’ problems instead of product teams, it will struggle. But much of the time, it will be wrangling the ideas of the individual engineers, designers, and analysts on your team and matching that to an overall vision set by the founder(s). It’s very rarely your ideas you’re executing on as a product leader, and it shouldn’t be.

https://twitter.com/stevesi/status/943945949352665088

I also don’t want to make it sounds like I am devaluing visionaries. They are, of course, critical in finding the initial idea(s) that create a growing company and maintaining a vision for that growing company. Having visionaries also becomes more important as you saturate your core market and need to tackle new value propositions to drive new growth opportunities. That is the ideal time for non-founder visionaries to enter a growing company. These are not going to be typical product managers or VP’s though. They are usually ex-founders. The best outcomes for these people entering an organization that is scaling is giving them a team and space to experiment with ideas until they find their own product/market fit, where a business unit is built out around that vision with product leaders to help them scale.

As you’re thinking about your business, think about whether you need a product visionary or a product leader. Most founder/CEOs are already visionaries, so they need a product leader to help execute. Some businesses minded founders need the opposite to be successful. If you’re thinking about product management, think about whether what you really want to be a is product visionary instead of a product leader. In that case, it might be a better idea to start a company than take a role expecting to execute on your vision and instead managing other people’s visions.

Currently listening to Ambivert Tools Vol. 3 by Lone.

How to Set Up and Hire an Analytics Team


Analytics has become a critical role at tech companies. A common question I receive is how to hire analysts and where they fit into an organizational structure. Below I share some tribal knowledge around common team structures, options I think work best and hiring tips that leverage this talent.

Functional vs. Embedded Teams
One of the first questions organizations face when hiring analysts is how they should structure the team. There are two common choices organizations pursue: a functional and an embedded model. The functional model is an analytics team reporting to a Head of Analytics. In the embedded model, every department (sales, marketing, product, customer service et al.) is in charge of solving for its analytics needs, hiring analysts for their teams when needed, and determining to whom they report.

Benefits of the functional model are having a senior seat at the table in major discussions for the company. The advantages this presents is getting analytics its own budget for tools and infrastructure and solving other analyst specific needs that may not be a top priority on any other specific department. The downside of a functional team is how analysts’ time are allocated. In a functional team, an analysts’ time is usually allocated on a project by project level, meaning they usually enter a project once that project is already well defined (whereas analytics could have helped by define the project, had it been involved earlier). And the analyst has not developed any specific expertise for that area. In my experience, analysts get frustrated in this model because they can’t go deep into any one area, and the other departments get frustrated because analysts provide a more cursory benefit than expected.

The embedded model solves a lot of these pain points, but introduces its own. With an embedded model, analysts are hired into one specific team, and therefore can develop expertise for that team very quickly. Teams are happy because they always have a teammate ready to help who understands their problems. While analysts seem to be happier in this model, the downsides are the reverse of the functional model. When there are cross-departmental analytics needs, they usually fall by the wayside. Investment in infrastructure and tooling is usually massively under-invested in, and it’s unclear where budget comes from to solve these needs.

At Apartments.com and at Grubhub, we implemented the embedded model. Marketing took control of analytics infrastructure initially, but we had trouble applying it cross-functionally. The analysts across all teams started meeting regularly to share learnings, but also limitations. When we added the Seamless team into Grubhub, they were used to the functional model. Analyzing the two together created awareness for me of a new approach.

The Hybrid Model
At Grubhub, the value of a dedicated analytics team for infrastructure and tools became clear, but also the value of the embedded analyst. Once I started at Pinterest and dealt with the functional model, we began to work toward a hybrid approach. This is a dedicated analytics team with a Head of Analytics, but with the analysts dedicated to specific areas full-time. So the person reports to a Head of Analytics, but sits with the department they support (in my case, the growth team). As the growth team grew, we created a Growth Analytics Lead who reported to the Head of Analytics and managed other growth analysts dedicated to specific areas, like conversion optimization or on-boarding. This allowed Pinterest to have the analytics seat at the table for budgets and resourcing, but the expertise at the team level to make the most impact. It’s now what I recommend to all teams that are scaling.

Hiring Analysts
If you are scaling a company and need more analytics help, it can be hard to understand who to hire that will actually help your teams. Hiring from analysts at other companies, especially larger ones, proved not to be a great strategy for me. I found during the interview process that most analysts were actually what I call “reporters”, in that they ran well defined reports for people who needed them but didn’t actually analyze anything. If you read analyst job descriptions, they inadvertently screen for these types of people by saying the candidate needs experience with all of these special tools. I can’t tell you how many job requirements that list Omniture (or whatever it’s called this week), Google Analytics, SPSS, Tableau, etc.

Experience with tools is not actually what you care about (though SQL and Excel are a big help). The more tools someone has worked with, the less likely they are to analyze the output of those tools. What you actually want are people who are analytically curious. Our first successful analyst at Grubhub was a new graduate whose cover letter talked about how he tracked his sleep patterns and his diet to find ways to improve his health. He crushed dozens of analysts with multiple years’ experience in our interviews because he was using his brain to analyze results instead of just report. So I now screen for roles where analysis, not reporting, is the unit of value. Many analyst teams at other companies are structured that way, but the majority are not.

You also have to test analytical ability in these interviews. At Grubhub, I gave people a laptop with a bunch of data in Excel and some vague questions to answer from it. The question was based on a real question we gave an analyst intern, who returned it to me saying there was no trend in the data. I ran the analysis myself and found one of the most important correlations for our business (the impact of restaurants per search on the likelihood to order). So I said, you have to be better than our intern to get an offer. It turned out to be an incredible screener. Most people never got far with the data, or their answers were spectacularly wrong. The few good analysts cut right through to a direct way to solve the problem and could explain it easily.

I like this approach because it actually shows the analyst what the job is (and if they’ll like it), and I can walk the candidate through how I would solve the problem so if they did get it wrong, they could learn something from it.

 — 
Analytics team are one of the hardest teams to scale. One of the keys is building a model of a team that will scale with the needs of a company, and the hybrid model is the best model I have found to maximize the important levers of effectiveness (and happiness!). Structure is not all that is important though. Hiring the right candidate is critical, and the market is doing a poor job of preparing people for what emerging companies actually need in an analytics organization. If you can hire correctly and structure correctly though, you will have a competitive advantage over those who do not.

Currently listening to Rezzett by Rezzett.

The Right Way to Involve a Qualitative Research Team

Most teams significantly under-invest in qualitative research. Growth teams especially are all about data, but they think that data can only come from experiments. This can make teams overly reliant on what they can learn from experiments and the quality of the data they have, and under-invest from what they can learn from talking to users. This problem is usually exacerbated by the fact that existing researchers at startups aren’t usually assigned directly to teams or work independently. I’ll talk about some of the problems I’ve seen, and the right way to invest in qualitative research for your growth team.

Learning and Applying from Research
Using the right type and method for your question is key. Of course, qualitative research is one component of the research stack along with quantitative research and market research. There is also different types of qualitative research depending on what you are trying to learn.

I remember when I was at Apartments.com and went to my first focus group, a common type of qualitative research. It was a mess for multiple reasons. The first reason was structure. Finding an apartment is not a large social behavior, so why were we talking with a group of ten strangers at once? As what I later learned usually happened, one or two participants volunteered the majority of the feedback, so while we paid for ten people’s opinions, we really only received two people’s opinions. So, I now only do research with multiple people in the room if it’s a social product, and it’s a group that would use it togethers e.g. friends or co-workers.

The second issue was delivering the feedback to people who weren’t there. I wrote up a long perspective on what the issues were with Apartments.com vs. our competitors. It primarily included product feedback on why we were getting crushed by Craigslist in major cities. I sent it to my VP and received a one sentence reply, “Don’t get ahead of yourself.” What a waste of time, I thought. We do all this research, generate real insights, and no one’s interested.

I’ve now learned that research teams inside companies feel this every day. At Pinterest, we had an amazing research team, but they were originally a functional team, which meant they had to determine their own roadmap of what to research. Depending on the stakeholders you listen to, this can be broad strategic projects like “What is the deal with men?” to specific projects like “Help us test this new search flow already built.” Research can add value at both stages, so the team worked on both.

What I think research found when they worked on the broader strategic issues was similar to my response at Apartments.com. “Cool, but not my roadmap!” say the product managers. Research then gets filed away never to be looked at again. Researchers get very frustrated. To be clear, this is a failure of leadership — not the product teams — if these areas aren’t prioritized. But it is common. On the flipside of working on something already built, success was more variable based on how well the product team defined what they wanted to learn. Frequently, what the product team wanted to learn was that they could ship it, so they selectively listened to feedback to things that indicated they were on the right path.

What I have learned suggests that qualitative research cannot be effective unless 1) its people are dedicated a cross-functional product team and 2) research is involved throughout the entire product development process, from initial research on market to determining a strategy to testing concepts to testing nearly finished products. The value of research accrues the more it is a part of each step in the process.

This approach solves for two main problems. One is that product teams will only pay attention to feedback that is directly related to their current product and on their own timeline. Without being part of the cross-functional team that includes product, engineering, and design, it is hard for research to to be on the same timeline. The second problem this solves is it helps research prevent the rest of the team from locking on assumptions that they may be wrong, so they are focused on the right solution to the problem with research, instead of confirmation bias at the end of a project. The Pinterest team has moved to this model, and for my teams, it made both sides much more successful.

When to Research and When to Experiment
For teams that rely too much on experiments and not enough on research, I tell them two things:

  • Experiments are great for understanding what people do and don’t do. Research helps you understand why they do or do not do those things
  • Experiments don’t help you understand the under-represented groups that might be the most important to learn from e.g. non-users or smaller segments of users

A great way to get started with research as a team is to answer why your experiment didn’t work. Sometimes, the answer is there in the experiment data, but frequently it is not. You have to talk to users to understand why they are doing what they are doing. The best way to do that is to ask them the context of them doing or not doing it.

There is also the middle ground of quantitative research that can be helpful (usually surveys). What I usually like to do is use qualitative research to understand the universe of reasons for something, and use quantitative research if I need to quantify the importance/commonality of those reasons.

Research also helps you isolate users you may not be able to isolate with your usage data. For example, at Grubhub, we were trying to understand how many people used Grubhub regularly for delivery, but not for every order. So, we asked. Then, we called those users to understand why they sometimes don’t use Grubhub, then sent another survey with those reasons to quantify which ones were most important to address. I outline that process more here.

But I Don’t Even Have a Research Team
At Grubhub, we didn’t have a research team for the first couple of years (or even a product team for that matter). So, when we needed to learn things, me, someone on my team, or our sole designer (hello Jack!) would do one of three things: 1) throw flows up on usertesting.com, 2) survey users on our email list, or 3) call users on the phone, and provide them with free food for their time.

You don’t need to be a professional researcher to do this, though they are better at it. You just need to determine what you’re trying to learn and who from. You want to watch people go through that situation if you can. If you can’t, ask them about the last time it happened and what they did and why. You will get better at it the more times you do it. Startups are starting to hire researchers earlier in their development because of the importance of understanding users beyond the data. So, you may be able to justify a full time role here earlier than you thought.

Thanks to Gabe Trionfi for reading early drafts of this and providing his feedback. HeHAH!

Currently listening to Beyond Serious by Bibio.

Starting and Scaling a Growth Team

I get a lot of questions about how to start and scale growth teams. Growth teams need to spend as much time thinking about how to scale themselves as they do on scaling the product. In this post, I’ll outline how I’ve seen growth teams start and evolve, define the ideal end state, and describe some of what you work on at each stage along the way.

The first question to ask is why you need to start a growth team. Why not just have traditional product and marketing teams? It’s a good question, and not well understood. Traditionally, product teams make the product, and marketing is in charge of getting people to try and continue using the product. Marketing can include traditional efforts like events, advertising, and PR, and perhaps some branding efforts and some online marketing. Of note, online marketers and traditional marketers have historically been very different, so depending on the manager, all of the budget and team allocation typically went to either online or traditional — not evenly split.

However, the best startups grow super fast not because of traditional marketing or online marketing, but because they tune the product to drive growth. This is why growth teams matter. Marketing teams typically don’t have access to the product roadmap to make changes to things that will impact SEO, conversion, optimization, or virality, nor the expertise to work with engineers and designers on making these changes. And product managers often prioritize new product features over engineering that will drive more people to the existing value. Growth teams are the connective tissue between product, marketing and engineering.

Starting a Growth Team: Focus On One (Easier) Problem
Goal: Improve one area important to the growth of the business
Team: PM, Designer, Engineer(s), Analyst/Data Scientist (sometime PM or engineer assumes this role as well early on)

It can be overwhelming to consider all the problems you need to solve. When you start a growth team, do not to try to solve all those problems at once. A growth team should start by going deep on one carefully chosen problem:

  • Pick a problem the company is not currently working on (to prevent turf wars)
  • Pick one of the easier — not harder — problems to improve (to build up credibility and drive toward early wins)

Since growth teams are typically treated with a healthy dose of skepticism by other teams when they start, carefully choosing your first area of focus will help maximize the growth team’s chances for success. Let’s say a growth team starts with a hard problem: Trying to increase activation of new users. It takes a month to measure increases in activation rates, and it has one of the smaller sample sizes in the traditional growth area. You run an experiment. It doesn’t work. You run a second, and another month goes by, and it still may or may not work. Now two months have gone by without a growth team win. People on the growth team are questioning if this is the right team to be on, and other teams in the company begin to question your purpose. (Hat tip to Andy Johns from whom I am blatantly stealing this example).

There are common growth problems where you can likely show quick progress. One is conversion optimization. Another is returning users via email and notifications. Still another is improving referrals or virality. These areas allow you to run multiple experiments quickly because the metrics will move immediately (they don’t take time to measure like activation), and because they have high sample sizes (all new users or all existing users opted in to communication).

Last note: When starting a growth team, it’s important that they sit together and not with their functional team members. The collaboration between the different skill sets and short feedback loops is how the magic happens.

Growing a Growth Team: Own the Growth Funnel (And Some Real Estate)
Goal: Improve metrics across the growth funnel
Team: PMs, Designers, Engineers, Data Scentists/Analysts

As a growth team grows, the team can start looking at multiple problems. These problems are typically across the AARRR framework by Dave McClure, (however revenue is frequently a separate team in an ad supported business.) The growth team separates into sub-teams who have their own meetings, and entire growth team meetings become less frequent. PMs, designers, and analysts may work on multiple teams. At this point, the growth team assumes ownership of certain areas of the product, including (but not limited to):

  • The logged out experience, which can involve both SEO and conversion
  • All emails and notifications (sometimes coordinating with marketing)
  • The onboarding flow, sharing flows, et al.

Owning these areas creates easy swim lanes between teams and prevents turf wars with the core product team. Growth leaders have to determine how to allocate people to work on the most impactful problems across the growth stack, and if a sub-team does not have enough support to make an impact, the growth leader should consider whether to support that problem at all. Growth teams also have to own goal setting, which means understanding historical performance and setting absolute goals.

At this point, teams start to work on both improvements in infrastructure and experiments. For example, at Pinterest, I was managing both the acquisition and retention teams. The retention team was struggling to grow because of an unwieldy email infrastructure. The infrastructure was built to support triggered, social notifications, but our strategy had evolved to more personalized, content-based recommendations. These jobs were taking days to run and would send automatically when complete, even though someone might have received a social notification five minutes before. So we spent nine months rebuilding it to allow us to scale more effectively. The retention team saw a step change in its performance and actually hit its Q1 goal in the middle of January after it was complete.

Evolving a Growth Team: A Special Forces Team Operating Across Borders
Goal: Reduce friction across the product that prevents people from connecting to product’s core value
Team: PMs, Designers, Engineers, Data Scentists/Analysts, Researchers, Marketers, Operations

As the growth team continues to grow, it involves more stakeholders and expands its scope. Instead of having clear areas of focus separate from the core product, the team shifts to analyzing the entire product and trying to figure out the biggest obstacles that are preventing the company from growing faster. At this point, the growth team has built up enough credibility that it doesn’t fight turf wars — it is laser focused on finding and solving the biggest problems that prevent people from connecting to the current value of the product.

Usually, the problems at this stage are deep inside product features and go beyond improving SEO, signup, viral and on-boarding flows, and sending the right emails and notifications (though growth teams still work on those too). The growth team is now identifying the friction that prevents people connecting to the core value elsewhere in the product. A lot of this work is simplifying product flows or building country-specific optimizations for slower growth countries. Growth teams can also become service organizations to marketing initiatives around this time, and I’ll talk about that in another post.

At Pinterest, we migrated to this phase rather recently. As we started to look at friction in the entire product, we saw in qualitative research that new users were getting bombarded with so many concepts it confused them: what a Pin is, where they come from, how to add your own Pins, what a board is, group boards, profiles, etc. The growth team made a list of the core things new people needed to know to get excited and start to build an understanding of the service. Then, we started experiments to remove everything else from the new user experience and only introduce it back once the core concepts were clearly learned. This helped improve activation rates.

What I described above is an amalgam of what I’ve seen in the market that works best. Of course, you should always tweak your approach based on the talent of your team and your company’s culture. I would love to hear what you think of this approach, and any additional insights you think would be useful to share with other growth leaders.

This posted originally appeared on the Greylock blog.

Currently listening to 28 by Aoki Takamasa + Tujiko Noriko.

Three Mistakes Integrating Growth and Design Teams and How to Address Them

While I encourage growth teams to start with a dedicated designer, that doesn’t usually happen. Usually, growth teams scale with just engineering and product, and as they scale to a certain size start to eventually “earn” a dedicated designer from the organization. This news makes growth teams extremely happy for a short period of time, but pretty quickly starts to create problems and culture clash. I’ll talk about what happens, and some ways I’ve found to solve these issues. Note: You can replace growth with marketing, and just about the same thing happens in these scenarios.


When you hear you’re getting a dedicated designer, you’re happier than Jim Carrey was back when he had a career.

Problem #1: Team Control AKA The Two Scenarios
One of two scenarios usually emerges when designers join a growth team. In the first scenario, the engineers or PM or marketing person starts with, “I’m so glad you’re here. I need this done and this done and this done for an experiment…tomorrow. No, we don’t need research to understand the problem. Don’t worry. I’ll only ship it if the metrics increase.” The designer realizes the growth team didn’t want a designer. They wanted a pixel monkey to just do what they say, not ever use their brain.


I got 99 problems, but a designer using their brain ain’t one.

The other scenario is just as bad. In this scenario, I call it the designers “moving in”. I liken this to the scenario when a significant other moves into your place and brings way more things than you own to move in. “Don’t worry, I got better furniture and books, and, oh, we’re going to have to get rid of those curtains.”

The design version of this is something like “I’m going to completely rethink all our strategies. I need to spend three months just doing research and then at least double that for design concepts. That home page really needs to change though.” Engineering thinks “this isn’t what we need. These people don’t even know what they’re talking about.” Engineering gets kicked out of the process of figuring out what to work on, and since the design process has no deadlines on it, engineering begins to have nothing to work on.


I tried to find a picture of people dressed in all black moving in, but the best I could do was chambray.

In both scenarios, instead of designers, engineers, product managers, and marketers trying to unify to form one team that leverages all of their strengths, one team tries to dominate the direction. What we did at Pinterest to try to solve this problem was design a process where the teams, specifically design and engineering, are jointly responsible for problem definition and solutions. Here’s what it looks like:

Project Kickoff:
Attendees: PM (leader), engineering lead, design lead, engineer and designer that will be working on the project

Goals:

  • Define the problem we’re trying to solve
  • Any existing ideas on how we might solve it
  • Any previous attempts to solve the problem
  • What metrics we need to inform potential solutions
  • How will we tell if we’re successful

Output:

  • Notes emailed to broader sub-team
  • Slack channel created just for the project, which includes notes and all future communication for the project

Project Brainstorming:
Attendees: engineer and designer on the project, PM (optional so as to prevent their calendar from being a bottleneck)

Goals:

  • Produce multiple potential solutions to the defined problem
  • Prepare those for feedback from attendees of kickoff
  • Cover additional measurement needed for directions that have been chosen e.g. how many people click top right login button

Brainstorm Review:
Attendees: PM, engineering lead, design lead, engineer and designer working on the project (leaders)

Goals:

  • Feedback on concepts from brainstorm
  • Choose one or two directions for experiment

Experiment Launch:
Attendees: engineer and designer working on the project (leaders), PM

Output:

  • Experiment doc that team agrees reflects what we’re testing and why
  • QA and approval over Slack from every team member that experiment is ready to launch to 1%
  • Note: Ramp ups of experiment communicated in Slack channel

Experiment Review:
Attendees: PM (facilitator), engineering lead, design lead, engineer and designer that worked on the project (leaders)

Goals:

  • Determine if you gathered data on key questions you needed to answer for the experiment
  • If yes, what does the data say?
  • If not, how do we get the data?

Output:

  • Ship/kill/iterate decision
  • Emailed notes to the entire team of what happened and why
  • Updated experiment doc on what happened
  • Updated Slack channel
  • Designer brings ship experiment candidates to design manager/lead for any feedback before shipping

      • Problem #2: Design Best Practices
      Designers bring with them a history of best practices from working on other design teams and schooling/training. It can be a culture shock for a designer joining a growth team and seeing little of these being followed. Designers will usually respond to this environment by trying to educate on many things the growth team is managing are designed “wrong” and how they need to be changed. These recommendations based on best practices are usually rejected by engineers on the team as it contradicts what they’ve seen in the data.


      A growth engineer after he hears about a marketing or design best practice.

      Design best practices were created with good purpose, but they quickly become inferior to AB testing in an environment where a direction can be put in front of users and its value quantitatively determined in days or weeks. This doesn’t mean you should AB test every design decision, but it does mean a designer can’t put their foot down by saying something is a best practice. In growth, the users being tested on determine what’s best.

      Another issue with design best practices being applied to growth is that design best practices are usually suited entirely for building user value only. The growth team in many cases needs to trade off user value and business value, or short term user value vs. long term user value. For example, someone can come to my site, and I can offer a great experience. The user then leaves and never comes back. Growth might decide to let someone get a preview of that experience, then ask the user to sign up so they can personalize and deliver more value. The result is that some people will not sign up (creating a worse user experience) and some will sign up, allowing the site to create a better user experience that day and over the long term. This accrues value for the business as well.

      Designers usually feel threatened by this environment initially. They see testing as a threat to their expertise. What you have to do is teach them to use it as a tool to get closer to user feedback at scale and be more efficient with their time. Most design work is wasted because it is spent on a direction that later proves to be flawed. With AB testing, a design can get quick feedback on an idea, validate the direction, then spend the time making it amazing once they know it’s worth that effort.

      That said, while this user feedback at scale is good at showing what users do, it’s not great at explaining why. So, along with spending time getting designers comfortable with testing, growth teams need to start doing more regular qualitative research as well. Designers will usually volunteer to do it themselves if there isn’t the right person to staff for it. Some PM’s are comfortable doing it as well. Engineers and engineering managers can be resistant to spending time watching the sessions, so the first couple you schedule have to be on critical areas you feel there is some understanding the team is missing by only looking at the metrics.

      Once designers get a little more seasoned on growth, you should also work with them to create the new best practices for growth to make the design and engineering process move faster. They will look very different than what designers recommended at the beginning, and be backed by data. Along with this process, I think it’s important to start creating user experience goals for your growth team at this time. These will be components that may not affect the metrics, but ensure a quality and consistent user experience. At Pinterest, we made compliance to design spec a requirement to ship after an experiment is validated, and said our top five flows had to be completely audited for quality user experience. This is a worthy compromise with design to show you actually care about the users and not just the business, a common complaint.

      Problem #3: Growth Onboarding
      Sometimes, you want to make sure you just nail the basics. Most growth designers are joining a new team in a discipline they’ve never worked on before, yet they don’t receive any onboarding as to what growth is and why it’s different. This is also an issue with other new growth team members. It’s like being dropped into a foreign country, not knowing the language, having no one help you, and being expected to contribute to the culture. Like France (okay, I’ve actually been to France, and the people were surprisingly welcoming even though they’re not known for being so). What happens is designers get frustrated and churn from the team.

      It’s critical that every new team member, especially designers, go through a growth team onboarding process. The first thing you do is state the purpose for your growth team, why it’s different from other product teams, and why it’s important. At Pinterest, I would say that while other product teams create value or improve the value of the product, growth teams focus on connecting people to the current value of the product and reducing the friction that prevents that connection. This is important because it’s not enough to build a good product. If no one knows about it, it will die. If it’s too hard to uncover its value, it will also die.

      What we did at Pinterest was create a history of major growth projects that were successful, walked through them, and explained why they were successful. That led into some of our team principles. We also spent a lot of time educating new people on the metrics we use and why. You can’t be expected to design winning experiences if you don’t understand the major criterion for success we’ll be using.

      After onboarding, instead of starting designers on large, complicated projects, it’s important to start them on a well scoped, smaller project, preferably with an experienced PM and engineer, hopefully patient ones. These should still be projects that are worth doing, but not large projects. We had a designer start on growth at Pinterest, and we immediately put her on one of the most strategic, long term investments for the team. While she did great work there, after a few months, she did a smaller side project redesigning our mobile web home page. The conversion rate increased, and we shipped it. She said, “Guys, this is the first time my design increased the metrics!” She was beaming. You want to get new growth designers to that moment as soon as possible.

      Lastly, once onboarded, you want designers contributing growth ideas as soon as possible. I like the idea of forcing people to bring one new idea to the team per week. I believe this is a muscle that needs to be developed via practice. New entrants to the team (design or otherwise) will typically propose bad ideas for a while. That’s okay. The trick is to provide a framework for generating ideas that makes new team members think about elements that typically make for good growth ideas, give feedback on the ideas submitted, and have the ideas be submitted as metrics wins or user experience wins. The Pinterest template focused on :

      • How many people would see this experience if it were built? This is usually by far the most important criterion for a successful growth idea. Any experience has to be seen by a lot of people to have a big possible impact on the business.
      • Has the company tried something in this area before? If so what were the results? This helps us make sure we use any previous learnings to avoid making the same mistakes. Just because we tried something before doesn’t mean it’s a bad idea though as growth environments change frequently.
      • How much effort is required for this idea? On growth, we also try to do the least amount of work to validate an idea is worth it.

      There is no reason designers can’t be key contributors to a growth team, but expecting it to happen automatically is usually a recipe for failure. I hope some of these tips can help you create a thriving cross-functional growth team with all the right disciplines involved. If you’ve have any other issues integrating design into your team, I’d love to hear about them in the comments.

      Currently listening to Sorry I Make You Lush by Wagon Christ.

The Present and Future of Growth

Quite a few people ask me about the future of growth. The idea of having a team dedicated the growth in usage of a product is still a fairly new construct to organizations. More junior folks or people less involved with growth always ask about the split between marketing and growth. More senior folks always ask about the split between growth and core product. Growth butts heads with both sides.

Why do more senior folks tend to turn to the difference between core product and growth than marketing? For this I’ll take a step beck. Now, I’m a marketer by trade. I have an undergraduate degree in marketing and an MBA with a concentration in marketing. So I consider everything marketing: product, growth, research, and I’ve written about that. I used to see what was happening in tech as marketing’s death by a thousand cuts. I now more so see it as marketing’s definition has gotten so broad and each individual component so complicated that it can by no means be managed by one group in a company.

So if marketing is being split into different, more focused functions, growth teams aren’t really butting heads with the remaining functions that are still called marketing over responsibilities like branding. They are butting heads with the core product team over the allocation of resources and real estate for the product.

So how do growth team and core product teams split those work streams today, and what does the future look like? The best definition I can give to that split for most companies today is that growth teams focus on getting the maximum amount of users to experience the current value of the product or removing the friction that prevents people from experiencing current value, and core product teams focus on increasing the value of the product. So, when products are just forming, there is no growth team, because the product is just beginning to try to create value for users. During the growth phase, introducing more people to the current value of the product becomes more important and plays in parallel with improving the value of that core product. For late stage companies, core product teams need to introduce totally new value into the product so that growth isn’t saturated.

My hope is that in the future, this tradeoff between connecting people to current value, improving current value, and creating totally new value is all managed deftly by one product team. That team can either have product people naturally managing the tradeoffs between these three pillars, or three separate teams that ebb and flow in size depending on the strategic priorities of the organization. All three of these initiatives – connecting people to current value, improving current value, and creating new value – are important to creating a successful company, but at different stages of a company, one or two tend to be more important than another.

We should evolve into product organizations that can detect which of these three functions adds the most value at a particular point in time naturally, fund them appropriately, and socialize the reasons for that into the organization so these different functions don’t butt heads in the future. I believe that is the product team of the future. I now believe this is more likely than marketers evolving to manage branding, research, performance marketing, and product effectively under one organization.

Currently listening to Good Luck And Do Your Best by Gold Panda.

Hiring Startup Executives

I was meeting with a startup founder last week, and he started chatting about some advice he got after his latest round of investment about bringing in a senior management team. He then said he spent the last year doing that. I stopped him right there and asked “Are you batting .500?”. Only about half of those executives were still at the company, and the company promoted from within generally to fill those roles after the executives left. The reason I was able to ask about that batting average is that I have see this happen at many startups before. The new investor asks them to beef up their management team, so the founders recruit talent from bigger companies, and the company experiences, as this founder put it, “organ rejection” way too often.

This advice from investors to scaling companies is very common, but I wish those investors would provide more advice on who actually is a good fit for startup executive roles. Startups are very special animals, and they have different stages. Many founders look for executives at companies they want to emulate someday, but don’t test for if that executive can scale down to their smaller environment. There are many executives that are great for public companies, but terrible for startups, and many executives that are great at one stage of a startup, but terrible for others. What founders need to screen for, I might argue about all else, is adaptability and pragmatism.

Why is adaptability important? Because it will be something that is tested every day starting the first day. The startup will have less process, less infrastructure, and a different way of accomplishing things than the executive is used to. Executives that are poor fits for startup will try to copy and paste the approach from their (usually much bigger) former company without adapting it to stage, talent, or business model. It’s easy for founders to be fooled by this early on because they think “this is why I hired this person – to bring in best practices”. That is wrong. Great startup executives spend all their time starting out learning about how an organization works so they can create new processes and ways of accomplishing things that will enhance what the startup is already doing. When we brought on a VP of Marketing at GrubHub, she spent all her time soaking up what was going on and not making any personnel changes. It turns out she didn’t need to make many to be successful. We were growing faster, had a new brand and better coverage of our marketing initiatives by adding only two people and one consultant in the first year.

Why is pragmatism important? As many startups forgot over the last couple of years, startups are on a timer. The timer is the amount of runway you have, and what the startups needs to do is find a sustainable model before that timer gets to zero. Poor startup executives have their way of doing things, and that is usually correlated with needing to create a very big team. They will want to do this as soon as possible, with accelerates burn, shortening the runway before doing anything that will speed up the ability to find a sustainable model. I remember meeting with a new startup exec, and had her run me through her plan for building a team. She was in maybe her second week, and at the end of our conversation I counted at least 15 hires she needed to make. I thought, “this isn’t going work.” She lasted about six months. A good startup executive learns before hiring, and tries things before committing to them fully. Once they know something works, they try to build scale and infrastructure around it. A good startup executive thinks in terms of costs: opportunity costs, capital costs, and payroll. Good executives will trade on opportunity costs and capital costs before payroll because salaries are generally the most expensive and the hardest to change without serious morale implications (layoffs, salary reductions, et al.).

Startup founders shouldn’t feel like batting .500 is good in executive hiring. Let’s all strive to improve that average by searching for the right people from the start by testing for adaptability and pragmatism. You’ll hire a better team, cause less churn on your team, and be more productive.

Giving and Receiving Email Feedback at a Startup

If your startup is anything like Pinterest, you receive a lot of email. Sometimes, that email is feedback on the things you’ve worked on. Since email only communicates 7% of what face to face communication does (with 55% of language being body language and 38% being tone of voice), email feedback can sometimes be misread. Email feedback can be given especially directly in a way that can be hurtful to the team it’s given to, making them defensive instead of receptive, because they fill in a tone and body language that isn’t there. I liken some kinds of email feedback I’ve received to someone walking in your house uninvited and starting the conversation like this:

“Man, what’s up with your door? You need to get that fixed. Oh man, those curtains are awful. Why on earth did you pick those? Is that your wife? You could have done better.”

Startups are making tradeoffs all the time. Everything is harsh prioritization with very limited resources. Employees at startups know this because they live and breathe it. But quite often, when startup employees give feedback to other startup employees, they forget that those people have to make the same kind of hard tradeoffs they do, and that might lead to some of the issues they’re emailing feedback on in the first place.

If you’ve gotten in the habit of giving this type of email feedback, a better way to give email feedback is to ask questions:

“Hey, I came across this experience today. Is it on your roadmap to take a look at this? If now, how did you come to that decision? Is there a experiment/document that explains this because I’m happy trouble understanding why this experience is this way? Here were some things I didn’t understand about it.”

If you’re on the receiving end of harsh email feedback, there are generally two things to think about. Firstly, if the email is to you personally, what I tell myself is to divorce the content from the tone, because the tone is in my imagination. A thought out response to the details of the email and why things are the way they are may seem to be annoying, but it’s worth it. What would be even better is if you point the person to a place they can learn about these things in the future.

If the email is sent to other members of your team, long term, you want to train your team on divorcing the tone as well. If you haven’t, you might need to use the email response to defend the team. Otherwise, they think you are not sticking up for them. What I do in this case is send an email defending the decisions as well as explaining them. Then, I will follow up with the email sender in person and tell them “Sorry for the harsh email. You really put my team on the defensive with the perceived tone of the post, and I felt I had to defend them. Next time, can you word your email a bit differently so we can focus on the issues instead of the team feeling like we have to defend ourselves?”

Currently listening to Sold Out by DJ Paypal.

Building Up Respect For a Product Team

An under-appreciated challenge in a tech company is creating a new product team and building it up from scratch into a valuable, high functioning, and well respected team. Having seen it done well and done poorly, much of what will make a team successful in doing this is pretty counter-intuitive. There is a well established sequence to doing this successfully in a high percentage way. There are two key components to optimize for:

  • team health
  • organizational understanding of the purpose of the team and its progress

Team Health
Team health is about trust between the individuals of the team and confidence of the team. It’s amazing how much of this is solved by having the team collaborate on a few successful projects out of the gate. It is tempting for a team to go after a huge opportunity right out of the gate, but this is typically a mistake as the team isn’t used to working with each other and won’t do its best work on its first project.

The right approach is to find small projects that have a high probability of success to start. This gets the team comfortable with each other, and they build up confidence in each other as well as the mission of the team as they see things ship that impact key metrics. How I like to prioritize projects is to forecast impact, effort, and probability of success. These can be guesses, but ideally a new team has quite a few high probability of success projects with low effort it can start with.

If you’re a team leader or product manager building a roadmap, you should be upfront that you’re prioritizing low effort, high probability of success projects to start for team building purposes. Otherwise, the team will be itching to start on high impact projects they might not be ready for. What happens when you start with one of those types of projects is that is by definition they are less likely to succeed, and with a new team working on it, that increases the project’s probability of not being successful. If the project isn’t successful, the team starts to doubt the mission of the team in general as that was supposed to be one of the highest impact projects for the team.

Organizational Understanding
Once a team is working well together and has some victories under its belt, it is time for the team leader to evangelize the team and its mission. I have seen high performance teams not do this second step as well, and it leads to things like organizational distrust and inability for the team to increase its headcount, which then impact overall team health.

So, how do you optimize for organizational understanding of a team? This depends a lot on the culture of an organization. What’s important to remember is that you need to optimize this understanding both above you and across from you. So, this means you need to increase understanding not just at the senior leadership level, but also to other peer teams of yours. This is not easy. I advise you start with senior leadership and optimize communication for whatever the way that team works. Do they like long strategy documents? Then write one. Do they have status updates? Leverage those.

Once senior leadership has a good understanding of why you exist, you need to address peer teams. For this, you need to understand how information diffuses at your organization. If product managers or engineering managers are hubs, start there. Email them directly with your strategy saying you wanted to give them a heads up as to what is going on with your team. Send them documents. Occasionally ask for feedback even if you don’t need it. Have a notes list? Over-communicate via that. Don’t be afraid to send emails about significant wins the team has had either. You also need to remember new employees and optimize for how they learn about things at the company.

There can be a tendency to just want to move fast with your team if you’re gelling and not invite feedback from other parts of the organization. This is a mistake. Lack of clarity for your team’s role outside your team can kill your progress if you’re not careful. You need to have the entire company on board with what your team is doing, or their lack of awareness could lead to distrust or roadblocks in the future. Addressing both team health and organizational understanding is the only way to have long term progress with a team in a growing organization.

Currently listening to Bizarster by Luke Vibert.

Scaling Up, The Three Stages of a Startup and Common Scaling Mistakes

One of the biggest mistakes I’ve seen management make at startups is mis-managing how their startups scale. There are distinct stages of a startup. Early on, you prioritize speed over precision. Later on, you will trade off speed for understanding exactly what makes the numbers move. Early on, you prioritize self-managers and weed out employees that need a lot of support to be successful. Later on, you have to expand into developing people and training as most employees do not thrive being thrown into the deep end right away. I’ll talk about these stages, the right way to think about how you manage processes, teams, and rigor during these times, and some mistakes to avoid.

The Early Stage
“What are lemons? Okay, I’ll go find some.”

When you are early in a startup, you need to have a bias toward getting stuff done. Strategic thinking doesn’t matter a whole lot. You need to try things and see what works. You do not have a lot of data, so when you try things, you are looking for huge, noticeable gains in aggregate data. You’re not doing any AB testing. Your analytics investment is small, and you pay attention to a small number of metrics. Every investor question requires you to get back to them because you’ve never done that analysis before, or you don’t have enough data yet. You’re also looking to hire entrepreneurial, self-starting jack of all trades. These are people that spot opportunities and just immediately go work on them even if they don’t have much experience. They don’t ask for permission; they just go try to figure it out. Whether it’s manning the phones, pulling data, optimizing an Adwords account, they’ll do it. You use the cheapest tools you can find to achieve your needs. In the early stage, you also want to do as few things as possible, especially from a product perspective. The CEO is deciding many things on a day to day basis and manages almost everyone. The early stage is defined by a few rules:

  • Speed > Precision
  • Jack of all trades > Specialist
  • Done > Perfect
  • Focus > Breadth
  • Execution > Strategy
  • Hungry > Seasoned
  • Cheap > Robust
  • Teamwork > Process
  • Doers > Managers

The Middle Stage
“Let’s make some lemonade.”

In the middle stage, the things that were easy for jacks of all trades to cover with a few thousand visits or a few customers become impossible to maintain with more customers and more visits. So, you start to hire more specialized people, but still relatively hungry and more junior, and people are still doing multiple jobs. You bring in a few or promote some people to managers so the CEO doesn’t have tons of direct reports. The CEO isn’t aware of all the decisions being made, but keeps the company still very focused. The manager’s job is mainly to clear roadblocks for the executors, and they generally still do individual contributor work themselves. The managers handle some of the what is now cross-functional communication gaps, and put in some lightweight process to organize what’s going on. Focus is still extremely important, but you start to think about some expansion opportunities. You typically have over a year’s worth of data at this stage, so you expend some effort understanding seasonality, maybe making some projections. You have more metrics and formalize things like LTV, CPA, runway, and can generally answer investor questions when they are asked. You still mostly rely on SQL and Excel for detailed analysis instead of full dashboards or analytics suites. You do start to invest in some better tools since you have scaled beyond some of your initial choices. The new rules:

  • Speed with some precision
  • Specialist = Jack of all trades
  • Doers with some doer-managers
  • Focus > Breadth
  • Execution > Strategy
  • Hungry > Seasoned
  • Cheap and robust are more closely traded off
  • Some process
  • Done > Perfect

The Late Stage
“Screw your lemons. We ain’t going anywhere until I get 5 apples, ten oranges, and some kiwi”.

In the late stage, you have a large team, and you need full-time managers and a senior leadership team that can filter communication up and down. The CEO is approving large, strategic decisions made below him rather than driving every decision. Every individual contributor has a specialized role, is much more seasoned than before, and you’re appropriately staffed for every job you’ve prioritized to get done. You create a good amount of process to streamline work between teams. Shipping changes in product and marketing are hard to measure for effectiveness, and could have significant negative effects, so you rely on experiments to measure impact of your work and prevent catastrophe. You invest in analytics tools to easily understand the high level and detailed metrics without having to do custom work. You start to work on expansion opportunities as you max out your initial product and market’s value. You invest in sophisticated forecasts so you can understand if you’re off track and what causes are for fluctuations. You buy or build enterprise level tools to help specialists do their jobs better, whether that’s advanced analytics packages, marketing software, sales CRM systems, etc. You also start to codify specific strategies before executing instead of just trying different things and seeing what works. This is valuable to make sure you’re going in the right direction, and to build organizational confidence in different teams who don’t work so closely together anymore. The new, new rules:

  • Precision > Speed
  • Specialist > Jack of All Trades
  • Managers + Specialists
  • Breadth traded off with focus
  • Strategy just as important as execution
  • Seasoned > Hungry
  • Robust > Cheap
  • Process first
  • Perfect vs. done more clearly traded off

The Common Mistakes: Not Scaling and Scaling Too Early
The biggest mistake I see startups make is staying in the early stage longer than they should, or adapting the policies of the late stage too early. The former is typically led by the CEO. In this case, the CEO loves being hands-on and can’t let go to help the company scale. What this actually does is keep the company in the early stage and prevent it from growing. I’ve seen it happen. The best way to help the CEO realize he is entering a new stage is to have a board shepherd that process or former CEOs as mentors who have gone through this transition. It isn’t easy.

Conversely, many CEOs see what the best companies do and assume their company should do that, not recognizing the difference in stage between them. These companies invest in robust analytics and testing solutions before they have enough data to use them, hire full-time people managers too early who need to justify their existence by hiring big teams, invest in too much process that inhibits growth, and have a team of too many strategists and too few doers. Their burn rates are high, and their growth rates are low, and they typically need to raise huge rounds to continue operating. The best way to prevent companies from doing this is to have them recognize the stage they’re in and hire people appropriate for that stage. A too late stage of hire in an early company will cause massive distraction, culture shock, and an increased burn rate. These CEOs need to let the problems of their business guide them to scale up in stage, not emulate other companies at later stages. It isn’t about where you want to be, but where you are today that should judge how you run the company.

Currently listening to Hallucinogen by Kelela.