Tag Archives: product

Product Positioning and the Product Marketing Partnership

This was adapted from an internal blog post in collaboration with Tamara Mendelsohn, Eventbrite’s CMO. Slightly edited to make more sense for people outside of Eventbrite.

When you actually launch new products to customers (which many teams don’t do), product managers and designers that have focused on scaling issues and optimization have to flex a new type of cross-functional muscle: how to work with product marketing. The default mode of this partnership across the industry is beyond terrible. Product works in isolation on product, then at the last minute hands things off to product marketing to name it and “launch it”. Product marketing then does their best to come up with a brand forward approach to make the product or feature sexy to their customers, but more so for their portfolios or their friends. Marketing doesn’t match the product, dev teams are pissed that marketing doesn’t understand the feature, marketing is pissed they got brought in last minute, customers don’t know what the hell the thing the company released is and don’t use it. Rinse and repeat.

If you don’t want to do that at your company, you actually have to put in work to make sure the default mentioned above doesn’t happen, and for a company that isn’t in a rhythm of shipping new products and features over time, muscles can not be particularly well toned for that. Consider this post a workout training video that helps you understand how to tone those muscles and learn how to position products well so that customers adopt and understand.

The Brand Marketing Pyramid

If you’ve ever had the pleasure/misfortune of getting an MBA, you’ve likely suffered through a few mostly worthless marketing classes. But they’re not all bad. Perhaps the most important concept you can learn from these classes is how positioning works. If you’re positioning any product or feature or service, there are three different layers of branding that need to occur, and they get harder to execute the closer to the top you try to get:

  • functional benefits: what the features deliver
  • logical benefits: what the product enables you to do
  • emotional benefits: how it makes you feel

Most brands work their way up the pyramid over time. At first, they try to clearly describe what their product or service does, taking a list of features and translating them into benefits. As the brand learns more about what customers actually care about, they usually determine that focus isn’t as optimal as describing what the features actually enable — sometimes called “the job to be done.” Describing a phone as having “10 megapixels” (feature) means very little to most people, and even saying “higher resolution” (functional benefit) is often not as impactful as saying “it takes great pictures” (logical benefit), but it depends on the audience. If a brand gets really good at describing the functional and logical benefits, over time it tries to reach the customer with more emotional resonance: “Feel like a professional photographer” (which was basically the “Shot on iPhone” campaign). But Apple is able to do this because it’s built up high awareness of its features and functional benefits over time. 

Let’s take a look at how this evolved at Grubhub while I was there. When I joined the company, our tagline was “discover who delivers”, clearly what the company did. When we hired our Head of Marketing, we overhauled the brand and changed the tagline to “eating made easy”, more of the logical benefit. Eventually, we moved up the pyramid again to “happy eating”, an emotional benefit.

Once it all came together, Grubhub had a playful, visual style that resonated with young professionals, advertising that focused around the irrationality of hunger, and a nice mixture of emotional resonance, features, logical and functional benefits in its advertising.

While Grubhub got better at this type of work over time (and has gotten so much worse since that Head of marketing left), Nike and Apple are incredibly fantastic at this balance. While they receive accolades for their work at the top of the pyramid, they still do all the forms of positioning in different contexts.

Position

 Apple

Nike
Features iPhone SE
Small 4.7′ design, Touch ID, powerful A13 bionic chip, portait lighting effects, 4K video, water resistant, long battery life, privacy built in.		 The Lebron 17
Heel max air unit, air max zoom pods, high tenacity yarns, heat molded.
Functional Benefit iPhone SE
Powerful, but not pricy		 Nike Joyride
Make running feel easy.
Logical Benefit iPhone privacy
iPhone lets me control who sees my data.		 Built for Speed
I will go faster in these shoes.
Emotional Benefit Make Movies Like The Movies
You can be as awesome as this when you use the iPhone camera.		 You Can’t Stop Us
Nike athletes defeat all odds. So can we during the pandemic.

Positioning Features Is Harder Than Positioning Products

I have a real appreciation for great brand marketing, because I personally suck at this shit. Ultimately, what great positioning does is provide potential customers a promise of the product experience. If branding obfuscates that, it’s not doing its job. This is why when many brands go for the emotional benefit without pairing it with the other elements of the pyramid, it doesn’t work. But the reason brands try to do this in the first place is there is a lot of value to be gained by pushing what the product can be considered for over time, and creating loyalty through emotional connection in the process. That shit is hard though! 

Positioning part of a product is even harder than normal positioning because the promise of the feature has to sit inside the broader context of the positioning for the overall product. Comprehension of how products work and what they can do is already incredibly difficult, so adding new elements to products compounds problems most products are already not good at. Product people tend to suck at positioning themselves. We’re often too wrapped in our jargon to speak in ways the customer understands. So we’re happy to have marketing take the lead. But the default mode of positioning features from marketing teams is to try to enfuse the brand into them without understanding how it can create even more comprehension issues. Scott Belsky, the Chief Product Office at Adobe, talks about how his startup initially got this wrong, and how it hurt comprehension (the whole post is great, but the example is in the “be more accommodating” section).

We faced this same issue when I was at Pinterest. When we launched the product internationally, retention was a lot lower than in the U.S. As we watched international users try the product, we realized that many were struggling with the key call to action saying “Pin it!”, which didn’t mean anything in their language. The product team wanted to change it to the local word for “Save”, but the brand marketing team was absolutely against it. We tried it anyway, and saw a 15% increase in activation rate, even in the U.S.! Our own brand was confusing users, the exact opposite of what it is supposed to do. We faced a similar issue from an SEO perspective. 

Our images were called “Pins”. By changing our title tags to say “images” instead of “Pins”, we received 5% more traffic from Google. There is plenty of time to get activated users to build an emotional connection with our brand, but if our brand makes the product not enticing to try or too confusing, it needs to back away until a later time when it can help, not hurt, company goals. Many times product and marketing teams will try to “go their own way” to solve the problem. Product creates a page for SEO. Marketing creates a page for the launch that is more branded. Not a good idea. The buzz the marketing team can create with that landing page will help with SEO goals because it can drive links that help Google trust the page more to rank it higher. Instead, product and marketing must work together to carefully balance positioning to maximize what helps the page rank, what gets potential customers interested, and what drives loyalty over time. This is what the best brands like Apple and Nike do.

Product Positioning Is A Delicate Dance

Many times when we talk about product marketing, we talk about the launch. But positioning is where the real long-term value is for features and new products. Dev and marketing teams have a lot of different variables to balance in launching new products and features, and positioning is at the heart of many of them and the hardest to get right. In almost all cases, we will need to iterate to optimize comprehension, adoption, retention, satisfaction, and that emotional connection that leads to word of mouth and improved adoption of things teams will launch in the future.

Thanks to Tamara Mendelsohn for contributing to this essay.

Currently listening to my Vocal Tones playlist on Spotify.

Fire Every Bullet

In crisis situations, a new style or management and prioritization has to occur. Andy Grove famously called this “wartime”, and he and others like Ben Horowitz have described what it’s like to be a “wartime” CEO. I haven’t ever seen anything written about being a wartime CPO though. Being a wartime CPO means a lot has to change in how you approach building product. Most product leaders think about their team as the product, and they continue to iterate on their team through hiring, training, coaching, delegation, etc. In wartime, you’re not hiring, you don’t have time to train or coach, and you make more top down decisions. But there’s one big thing that needs to change for wartime CPOs I want to cover today, and that is prioritization and evaluation.

I hope you never have to be a wartime product leader. But as you might have surmised, Eventbrite has been in a wartime situation for over a year now. Let me give you some background. In February 2020, Eventbrite started to become aware of a big problem. While the company was off to a fantastic start to the year, we could see the tidal wave of a global pandemic coming. Neither the world, nor us, was ready for what our metrics were showing could happen: the global shutdown of the live events industry. We were about to go to war, but not against a competitor, instead against nature.

There were, of course, differing opinions about the severity of the situation. The media was making fun of VCs for banning handshakes and telling us to be more worried about the seasonal flu, but the media failed to take into account the true potential downside. When the downside is unknown and potentially disastrous, as Nassim Taleb would say, the only rational reaction is over-reaction.

We planned for our mission, to bring the world together through live events, to be tested like never before. We acted swiftly to ensure the longevity of our business and mission, and the wellbeing of our employees, creators and investors. We launched resources and tools, we increased communication to keep employees informed, engaged and connected, and we realigned cost structure and secured access to funding. We also re-wrote the strategy of the business to align to this new reality.

More specifically, on the product side, we shifted all of our focus to help our creators prepare for the reality of a global pandemic and what that meant for their businesses. As we got to work, I recalled a conversation I had with Luc Levesque years ago when we brought him on as an advisor to Pinterest. Luc was the VP of SEO at Tripadvisor at the time (now VP Growth at Shopify), which was the only American company I could find that had succeeded at international SEO. Pinterest’s biggest impediment to growth was international growth, and SEO had been our primary lever to get U.S. growth into good shape after our initial strategy of leveraging the Facebook Open Graph stopped working. I was asking Luc whether we should prioritize different tactics to make international SEO work like international link building, localizing our URL structure, or making sure only content in local languages showed up on a page. His response was basically, “Dude, you do ALL of it.” We could figure out which one mattered the most after we’re successful. This is the approach we took at Eventbrite when the pandemic went into effect.

We built every possible thing we could to help creators and the company. We helped creators pivot online through virtual events; we published information on how to apply for loans; we built a bulk refunds tool, introduced the ability for creators to pay-in or wire money to Eventbrite to refund consumers and a credits system, and an easy postponement tool. We fired every bullet we had at the pandemic problem. We did not prioritize. It didn’t matter how much effort each activity was. We were going to do it all, and do it fast. We released things that would not meet our normal quality bar, because the only thing better than releasing these features for creators today would have been releasing them yesterday. We later learned other companies were doing the same.

One thing I would say, maybe feeling somewhat self-conscious right now, we made a company wide decision to lower our minimum acceptable quality bar due to the pandemic. If you observe Shopify right now, we are launching a lot. Almost just because of this one change. So we are pulling a lot of our roadmap forward in time. Everything that can help businesses right now, just because, again, we are in a crazy world right now.
Tobi Lutke, CEO of Shopify, on Invest Like The Best (May 2020)

You never want to be on the other side of a catastrophic event having failed saying “we could have done more,” and that has been top-of-mind for us throughout the pandemic. What is interesting when you take the approach of doing everything is some of the tactics work really well, and some do not. There is a tendency to evaluate those tactics in isolation for their success and failure. This is a mistake. You have to evaluate success or failure via the aggregate of all the bullets you fired. Did we save the company? We did. Did we save a lot of our creators’ businesses? We did. You can’t do an experiment or results review on those tactics individually. You have to look at the portfolio.

When should you fire every bullet? Do you fire all the bullets only in near death situations? How about live events recovery? Is Eventbrite firing every bullet to maximize the success of that? Prospect theory states that humans dislike downside impact more than they like upside impact of the same level. Many people have called this irrational. In fact, it is not, because extreme downside effects are things like death and ruin. Extreme upside effects don’t have the same degree of effect. So, firing every bullet makes a lot more sense for extreme downside scenarios than extreme upside scenarios. Think of the two by two.

The difference between we could have grown a little faster and death vs. safety is not at all the same. That doesn’t mean firing every bullet in upside situations is wrong; it’s just not as clear of an answer as in the extreme downside scenario. Quibi did everything to grow for its launch and burned through $100 million. It might have been better to do a smaller launch and iterate on content to find stronger product/market fit. Uber burned through billions for many years to try to both accelerate the size of their market and their market share within it, and it’s almost a $100 billion company.

The important thing to remember here is in extreme circumstances a lot of the best practices are different. Decisiveness, waste, and measurement will trade off in very different ways. It’s important to register when in these extreme situations that a different rulebook may be required to do the right thing for your business or product when in normal operations. Wartime isn’t over at Eventbrite, but I like to think of the situation right now as we’re finally having peace talks. 

Currently listening to Primitive Arts by Ron Trent.

The Rise of Department Operations Roles, and How to Tackle Inefficiency

At every company, there are well defined roles you would expect to see. In software, they tend to be roles like engineering, product management, design, marketing, legal, finance, etc. But as companies scale, they tend to invest in more specialized roles over time. During that transition, operating in one of these roles can be confusing. One interesting trend emerging at many companies including Eventbrite is the creation of departmental operations roles. Oh, you’ve seen them: Marketing Operations, Sales Operations, Design Operations, Product Operations et al. My first instinct when I hear an “operations” title is “inefficiency.” Not that the operations person is inefficient, but that there is inefficiency at the company. It is okay to be inefficient at things at your company. At various times, it will make sense to be inefficient in tons of areas of the company. The danger is in accepting inefficiency for the long term. We should strive to eliminate inefficiency over time at companies, not embrace it. Thus, the goal of departmental operations teams can be somewhat paradoxical. By explicitly attempting to remove inefficiency, they are actually attempting to remove the need for their roles entirely. 

The Intercom Biz Ops team was the first to just come out and say it. This can create a remarkable conflict of interest though. By doing a good job, will I eliminate the need for my job? As a result of this conflict of interest, we have started to see people in these roles try to define them as long-term necessities in organization, which means employees are optimizing for themselves instead of the company. What is interesting about this trend is that this job security issue shouldn’t be a conflict of interest. The goal of every job inside an organization should be to eliminate the need for it so you can do more important things. This is a great way to get more responsibility and promotions. There would be no greater pride for me than feeling like I am no longer needed in my current role at Eventbrite, for example.

So, if you want to do a great job in a departmental operations role, how do you do that? Well, the real question is how do we solve inefficiencies at companies? It’s like solving any problem within your company. You use various tools. I have a very specific hierarchy I like to follow to solve these problems, and I wish more companies used it instead of defaulting to hiring people and creating roles. If you’re in a department operations role, you can consider this a playbook to helping the company be more successful.

#1 Software
Any problem or inefficiency we can solve with software is superior to other forms because it automates away the inefficiency in the future once it is built. Now, it may be the case that we cannot think of or do not have the resources to develop a software solution to the problem, permanently or temporarily, or that there is no software we can buy that helps. This is okay, even normal. Companies tend to under-invest in leveraging engineering to build solutions that help their companies become more efficient, largely because so many companies are engineering-constrained. But just like engineers have started to work more on business problems like growth or marketing, they will eventually start to work more on internal tooling that enables their and other teams.

#2 Process
My next goal would be to solve the problem or inefficiency with process. Process innovation is a remarkable thing. By understanding the issue and proposing a way of operating to solve the problem, the pain point can frequently be mitigated or disappear. Now, this is not as automatic as software because it requires people doing things to follow the process and usually more training than software. But processes can become hard-to-break habits, especially once employees see the benefits of them inside an organization. This is more of where departmental operations roles tend to thrive today. By understanding the strategy of the business and the day-to-day operations, operations team members can spot inefficiencies, test different process solutions and assume responsibility for scaling out ones that work. But these solutions can be led by anyone. The more this happens without dedicated operations people, the more efficient an organization is.

#3 People
My next best option after process is to solve a problem with a person. Hiring is expensive, so it should be considered carefully. I definitely prefer hiring for concrete roles that have well defined value inside companies. Engineer, designer, analyst, etc. If you need to hire an Operations role because the problem is nebulous, temporary, or today can’t fit into one of those roles, you can do it. But you should have a career plan for those roles to evolve into a more well-defined role. Fortunately, team members that showcase generic problem solving skills are great fits for many other types of roles inside companies, so they rarely have to worry about their next internal career move. Still, it may be more efficient to actually look at one of the other solutions below.

#4 Advisors or Consultants
Sometimes the right person isn’t available, or the problem is well time-boxed. In this situation, I prefer to find consultants or advisors. Subject matter expertise exists out there in the world for almost anything. Working with individuals that have specific experience makes it easy to align incentives even if this subject matter expertise isn’t hireable full-time. If the issue is temporary, you may not want to hire full-time anyway.

#5 Agencies and Firms
Lastly, I look at agencies or firms. With these companies, it is hard to prevent misaligned incentives. But they do have expertise and bodies that you can throw at a problem with less risk of being stuck with them full-time. 

Where Operations Wins
I am not against Operations roles; I just want companies to understand how to use them well and to scale out of them whenever possible. Early stage companies can absolutely crush with Operations. Smart people Swiss Army Knifing problems is many times the fastest way to scale in the early stage. Operations roles can also thrive in larger companies, if they root out inefficiency with the tools above and don’t succumb to empire building. We can’t yet automate transportation needed for the delivery of the business? Call that the Operations team. We can’t automate training for our customers yet? Call that the Operations team. 

The COO role is very common, though its definition is ambiguous because it tends to be a grab bag of executive inefficiency. Usually, the COO manages what the CEO is inefficient at. That can be management in general, or parts of the company the CEO is less capable of adding value to. The COO role can also be a roll up of certain executive functions when the CEO has too many direct reports as well. 

This is all fine and even optimal. But every operations role you add you should also add to a list of inefficiencies you hope your company can solve one day, and always be thinking towards software or processes that make these current roles unnecessary so those people can take on even more important roles over time.

Currently listening to Good Times by Moire.

Reducing Product Risk and Removing the MVP Mindset

I originally wrote this piece internally at Eventbrite and thought it might be valuable to folks who do not work at Eventbrite as well. Slightly edited to remove Eventbrite jargon.

What is the right way to build products? Earlier in my career, I would have told you everything should be AB tested, and that you should build only as little as you need to validate a hypothesis. Every problem should have user research involved at the beginning, aligned on the problem instead of just validating or invalidating solutions. Every key result should be outcome based. Every engineer, designer and product manager on the team should be involved in defining the problem and the solution. These are all good ideas. However, once you add enough of these “best practices” up, it reads kind of like those articles talking about the morning habits of the most successful people in the world. If you actually try to follow all of those habits, it would take you six hours a day and cost a lot of money. I was once reading an article about what a futurist at Google eats for breakfast every morning to live longer. The article added up that his diet of food and pills cost him $1 million per year. My morning ain’t that long, and I ain’t got that much money.

So if the documented “best practices” take too long or are too expensive, how do you approach different product situations? I want to share a few frameworks that I think can help, but there is no substitute for judgment here. Eventbrite has lots of different product problems. One of the things we are trying to get better at is defining the type of product problem we are working on,  and outlining how we build based on who we are building for, whether it’s consumers or the many different types of event creators we support.

RULE #1 OF PRODUCT WORK: IT IS NEVER DONE

We should all know that products and features need to continually improve to stay relevant for our users over time. And it should come as no surprise that no product or feature is perfect on initial release. They need to be refined over time to get better, incorporating feedback from actual users. Many companies just launch new features and forget they exist, rarely improving them. This is not how great products are built. What this means though is that initial releases will never have everything teams want. But if you continue to iterate, this is not a problem. As long as we’re getting value to users as they become available.

If you’re going to continue to iterate on features, supporting them takes effort to maintain and improve over time. The much bigger question then becomes whether product work should be started, and when progress should be shared with users over time. If you start the work, you’re implicitly saying the company should expect to support it for years. That’s a big commitment. So, at Eventbrite we try to do work to de-risk that commitment up front. And if you have a long amount of work mapped out, figuring out what chunks can be released independently so that incrementally more value is being experienced by users is tricky. I’ll dive into how at Eventbrite we are trying to improve our approach to reduce risk and ship releases incrementally.  

A FRAMEWORK FOR REDUCING RISK IN WHAT WE BUILD

Product development projects always have very high opportunity cost. So, no matter who we are building for, we try to de-risk investments into projects by learning more about the problems before we invest time in solutions. How do you de-risk projects? An obvious answer may be to talk to customers, right? But it’s not that simple. Users of products are generally unreliable narrators of their own behaviors and preferences. How unreliable depends on who the customer is. For our consumer products, asking our consumers what they want is usually a lost cause. Sure, we can and should regularly talk to them about their problems, but we have to infer solutions ourselves. We can’t expect our consumers to be excellent product thinkers. If we did build what they asked for, they probably wouldn’t end up using it. As Henry Ford allegedly said, “If I’d asked customers what they wanted, they would have said a faster horse.” 

For event creators, it depends on how sophisticated they are. Eventbrite is a broad, horizontal platform used for many different types of events, from small meetups to large festivals and conferences and everything in between. For our most sophisticated customers (and most enterprise businesses), it’s not a bad approach to ask customers what they want, ask them to prioritize how bad they want it, and build and charge for things in that order. For less sophisticated customers, it’s still likely a bad idea. This ends up creating two spectrums to help you understand what to do by level of sophistication of the user vs. how much data users generate. Each of these quadrants tends to have a different approach to de-risking projects.

So what this implies is that our approach to de-risking projects should be very different depending on the type of customer we’re building for. Eventbrite isn’t Instagram, but we do have a lot of consumers compared to most companies. So, attendee experience projects are going to bias toward a data-first approach with research to back up the why when needed. They will also be the most likely to AB test. The self-service creator side of our business is more towards the middle of these axes. So we’ll blend some data and some research needed in addition to direct or proxy customer feedback. We also have segments of frequent creators that are smaller in number, but more sophisticated than the average self-service creator, so direct feedback plays more of a role in what we build, but it also has to be supported by data and user research. When we expanded our sales team into enterprise segments in the past, we were much more likely to build what they asked for.

A FRAMEWORK FOR DIFFERENT PRODUCT APPROACHES WITH DIFFERENT LEVELS OF AMBIGUITY

Cody Evol, a Director of Product Design at Eventbrite, helped develop one of our more useful frameworks to help us understand how much time we devote to a project. When I joined Eventbrite almost two years ago, every team was describing their project as an MVP, which is quite strange, as the Minimum Viable Product framework was built for launching new products, which is not what most teams were doing.

Product teams using inappropriate frameworks off the internet to justify their approach? Inconceivable!

The key thesis of Cody’s design quality framework is that the level of investment before something reaches a customer is tied to how confident we are that we understand both the problem and viability of the solution.

Your development approach should change based on how ambiguous the problem and ideal solution are.

When there is a lot of ambiguity about the problem or the solution, we should generally try to de-risk the project before investing a lot of time into it by using the approaches above. And we have many tools at our disposal to do this. With enough validation, teams should invest in building minimum viable products or features. Minimum viable product is probably the best known product development framework outside of perhaps Agile. It is about delivering value to users by building the smallest product you can to test your hypothesis. You ship to learn, which influences future product development. However, most things we build aren’t minimum viable products. It’s a product development framework for brand new products, and we rarely build brand new products. More likely, we are building features. These can be even more lightweight because you are building something on top of an existing product (you can think of a product as a bundle of features). The “Follow This Creator” feature was a great example of a minimum viable feature on the consumer side of our product. We didn’t know if consumers would adopt it, so we launched it just on Android without even building the push notification functionality. As we saw consumers adopt it, we built it for iOS and web and with email and push functionality.

The goal is usually to reduce the ambiguity around the product problem and solution so that you are not building an MVP/MVF, but instead doing phased delivery. This is a type of development where you have a strong vision and the problem and solution are validated, so the goal is to build toward that desired vision, and release incrementally as valuable pieces of that vision are complete.

MVP’s and MVF’s are to prove that our ideas actually solve a problem. If we prove that, we typically need to invest a lot more to reach the potential of the product or feature idea. Rarely do MVP’s or MVF’s crush expectations out the gate.

OUR APPROACH MOVING FORWARD

When projects are de-risked, we start building the desired product directly in phases. No shortcuts on design quality. No accumulation of technical debt to “get something out”. These are features we intend to be very valuable and support for a long time, so they should be built the right way. But how do you break up the product vision into phases when you have such a clear vision of the whole you want to build? Shouldn’t you just build it all upfront? There are a few reasons this is a bad idea. One is that if you wait to deliver some value to customers until all of the value you envisioned is completed, customers are dealing with sub-par experiences that do not improve for a long time. It is better to deliver some improvement over time than no improvement for a long time, and then a big reveal. Otherwise, you are holding back things that could add value to them today unnecessarily. Secondly, even with strong visions backed by data, usage of products can surprise you. It’s better to get the learning from usage incrementally over time to make micro-adjustments to the vision. Getting no feedback from customers on what you’re building for a long time is dangerous as customer preferences evolve. Lastly, releasing regularly allows you to de-risk your vision technically as well. You get to see how what you’ve built scales or breaks incrementally instead of all at once.

What if there is no way to break up the value delivered to the customer in phases, meaning it’s literally not valuable until all of the work is complete? It is generally still a good idea to release the code incrementally to de-risk the technical aspects of the project even if the customer doesn’t see it. One of my favorite recent examples of this is Apple. Hardware is usually very expensive to iterate on compared to software. Apple was working on a multi-year effort to launch Apple Pay using a customer’s fingerprint as the verification. Instead of launching a new way to authenticate and the credit card payment at the same time, it launched TouchID on the iPhone 5S to see if fingerprint authentication would be a successful interaction. That de-risked the launch of Apple Pay on the iPhone 6 using fingerprint for payment.

To bring it all home, product development is hard. You can’t read one article on the internet and all of sudden build great products, just like you can’t talk to one customer and know exactly what to build. We create frameworks not so that we can shut our brains off and follow an exact guide to building our product, but so that we can focus our brain power on the thousands of other very confusing and ambiguous questions we will undoubtedly face in trying to build this product over time. I hope these frameworks can help you understand how Eventbrite approaches situations it comes across, and gives you some things to think about as you build products.

Thanks to Cody Evol for his help on these frameworks.

Currently listening to Pool by Skee Mask.

Why Product Leaders Fail

I’ve yet to meet a fellow Chief Product Officer or Head of Product say, “yeah, I’m crushing it right now.” In my conversations with fellow product leaders, there’s even a meme that’s started to form around product leadership roles. Effectively:

“Yeah, you just try to put some points on the board before you inevitably get fired.”

So, if a typical CPO feels their role is about trying to survive a couple of years i.e. long enough to help the business a little bit, what is causing that? Why is it so hard to endure as a product leader?

I would say there are three common failure modes depending on how far along the company is. The earlier stage of a company it is, the more likely the answer is going to be misalignment with the founder/CEO. What no one tells product leaders when they accept product leadership roles is that nine times out ten the founder and CEO still wants to drive the product vision. They want you to help execute that vision. And as founders scale, their founder intuition ebbs in effectiveness in comparison to product expertise in-house, but it takes a long time for founders to accept that. That transition period can be very rocky.

For later stage companies, the more likely answer is that the CPO is really only good at one type of product work, and the type of product work needed for the business changes over time. This can manifest in two ways: The product leader has skills that don’t match the type of job needed today, or as they execute, the skills needed change, and the leader cannot adapt. Not only is the product leader not good at that new job, they are also less likely to be interested in it.

Let’s talk about each of these failure modes and what both product leaders and CEOs can do to make them less likely to happen.

Failure Mode #1: Who Owns Product Vision?

Founders tend to have insanely good intuition about their customers and products because, let’s face it, no one has spent near as much time thinking about them and their problems. When startups tend to hire product leadership, it’s their first time hiring this type of leader. Interview processes can be clumsy or unoptimized, with the founder still figuring out how to articulate the real need. Commonly, what happens during this process is both the founder and prospective product leader end up jamming together on the future product vision. Both sides love this engagement, but for the founder, it’s not an effective test on how much the prospective product leader will help the founder, and for the product leader, it can give a false impression that they will have a ton of say in the product vision.

If at all possible, founders should leverage outside expertise to structure the recruiting and interview process for this type of role. One of the key questions to get explicit on before the process begins is what role will this leader play in setting the product vision? For non-product/eng/design founders, they may be asked to define it. For founders with product, eng, or design backgrounds, that is typically not the case until the company becomes much larger. Product executives usually play a consulting or execution role in a vision that is founder led. Founders need to tell candidates which one it is, and candidates need to ask. Neither of these happen as often as they should. 

Once founders understand their answer to this question, they need to vet for the appropriate skills. If what you really need is help executing on the vision, don’t spend much of the interview process jamming on the vision. Sure, candidates need to understand the vision, but what you really need to learn is are they comfortable receiving a vision from you and bringing it to life in the myriad ways that are difficult. 

Failure Mode #2: Does the Expertise Match the Type of Product Work Needed?

Different companies require very different approaches to their product strategy to be successful over time. Most of us understand there are different types of product work. There is tech and process scaling, there is building new products to find new product/market fit, there’s building new features and iterating on the user experience to strengthen current product/market fit, and there is growth work to get the maximum number of users to experience the product/market fit that exists. Traditionally, product leaders lean toward being experts in one or another. For example, I am definitely most known for my expertise in growth.

Founders often lack the understanding of what type of product challenge they are actually facing when they attempt to hire a product leader. Network effects businesses tend to focus more on growth because more users make the product stronger in a much more meaningful way than new features. DTC ecommerce companies / brands are always launching new products. SaaS businesses tend to need to launch lots of new features over time. Hiring a product leader that wants to build new features all the time into a network effects business likely isn’t going to work that well.

Failure Mode #3: Can the Product Leader Adapt as Needs Change?
Even if founders hire the leader with the right skills at the right time, as companies scale, how much weight they put on these will need to change over time. Today, the product leader’s job is to be what the business needs them to be. So while the old school product leader is a specialist, the new school product leader needs to be a chameleon, who can balance a portfolio across scaling, new product work, feature work, and growth weighted toward the needs of the business, and re-weight it considerably over time as business needs change rather than leave once business needs change. That’s hard, but inevitably how product leaders have to evolve to be successful over the long term within a company.

As a growth oriented leader, I am actually spending more of my time at Eventbrite on scaling, features, and new product expansion work, because that is what the business needs right now.

—

Product leadership is incredibly hard. Both founders and product leaders can eliminate some of what makes it so difficult by aligning on expectations before hiring roles, and on aligning which problems the organization is focused on right now. It is then up to product leaders to be able to evolve as the product needs change over time. They are both the best equipped to understand when needs change and help the organization change with them.

Currently listening to Rare, Forever by Leon Vynehall.

The Types of Product Team Organizational Structures

There is no one perfect way to structure a product organization within a company. Like most things in a company, organization structures only make sense for given moments in time, and as changes happen with the business, so does the organizational structure to match. Product teams are no different, but they tend to vacillate between four different types of structures over time, which I’ll explain along with their pros and cons. These structures tend to work best when aligned with how engineering, design, and other partner functions are structured, so there are 1:1 relationships between leadership roles in those functions.

Alignment of Structure

By far the most important rules of product organizational structures is that they should closely mirror the partner functions. So engineering, product management, and design should tend to have structures that match each other whenever possible. This can’t always be 1:1 as engineering teams tend to be much larger, but if engineering is organized in a very different way from, say, design or product management, it can cause many problems. It can create misaligned incentives as well extra coordination between managers as one director may have to interface with four different peers to align their work instead of one. Assuming the company tries to align all of the development functions, let’s talk about a few ways they can be aligned.

Organization by Type of Product Work

In the Reforge Product Strategy program, we talk about the different types of product work. A common organization structure is to separate teams based on the type of work they are doing. One pillar is the innovation pillar working on new products. One pillar is the core pillar trying to strengthen the core product with additional features and maintaining the core. One pillar is the growth pillar trying to grow overall usage of the current product. One pillar is the platform pillar working on internal features that support other product and engineering teams to allow them to scale. 

The organizational structures can be quite stable, but the level of investment in them can change quite dramatically over time. Product leaders need to consistently be thinking about the right allocation between these different types of work and changing resourcing among them based on the needs of the business. This reshuffling can create change fatigue of individuals that have to move around teams.

Organization by Customer

If a product has different types of customers, a common structure is to separate product teams by the customers they build for. This is very common in marketplaces that have supply and demand teams, or in subscription companies that have self-serve and enterprise, or free customers and paid customers. This allows product people to become experts in understanding their customer well to deliver them value. 

These organizational structures tend to be very stable in marketplaces, but less so in subscription companies if the contracts between those teams are not written correctly. For example, I’ve seen companies that have split into free and paid where the free team is penalized when a person upgrades, which should be the very goal of a free plan. But, since the free plan was goaled on usage, whenever someone upgraded, their usage was removed from the free pillar’s metrics. Even in marketplaces, there can be issues with this model as great products are things that benefit both supply and demand, and it’s hard to build a great product if you only know one side well.

Organization by Value Proposition

If a product has different value propositions, a common structure is to separate product teams by those different value props. For example, at Pinterest, we thought about our value props as Discover, Save, and Do, and had different product pillars for each of those value props. This makes it easy to align OKRs or goals to the value you’re trying to provide your users.

This organizational structure can last for quite a while as value propositions do not change dramatically over time in most companies, but some necessary product functions don’t easily fit into this structure. Great product goals tend to be focused around value created for the customer that, in the long run, create value for the business. Value prop structures are more likely to ignore or de-prioritize that second part of long run value for the business. These structures can also overly focus on the product engineering part of the engineering stack as their goal is to deliver more value. If not handled correctly, this structure can lead to an under-investment in important maintenance, scaling, or growth work as the desire is always for new features that aid in delivering this value prop.

Organization by Initiative

Companies tend to have strategic initiatives that emerge from annual planning, so a common organizational structure is to align product and engineering around those initiatives. This process can work well when initiatives are new, and the organization isn’t sure what types of product work will be most important to succeed in an initiative yet. The initiative structure allows PM’s and engineers to flex between innovation, growth, scaling, etc. as needed to support the ultimate outcome for the business. This requires product folks to be more agile in their planning and in the skill sets they are leveraging. We organized this way at Eventbrite when I started as Chief Product Officer.

This organizational structure tends to be the least stable because initiatives change frequently or at least on an annual basis. Once the type of work that is required to be successful for an initiative becomes clear, many times teams like to specialize by switching into one of the other structures above.

Mixing and Matching 

What a lot of companies end up doing is mixing and matching a couple of these approaches to fit their needs. For example, while Pinterest used the value prop structure for the core product team, it also had growth and monetization teams to balance the business needs of the company. That combination proved fairly stable for a while. No organizational structure is perfect, and emerging trends in product, like having product managers work deeper into the technical stack on platform, infrastructure, and DevOps, will continue to shape changes in how product leaders structure their teams to drive effectiveness. Nevertheless, it’s important to think through the tradeoffs of different organizational structures to make sure they match best to the needs of the company at the specific point in time. What is most important is aligning this structure between engineering, product, and design as much as possible to the teams work in unison.

Currently listening to Hand Cranked by Bibio.

Casey’s Guide to Finding Product/Market Fit

As a product leader with a background in growth, it’s surprising how much what I actually end up working on is product/market fit. Product people should only be focused on growth i.e. connecting people to the value of a product once they’ve confirmed the product is delivering value. So it’s important to have a strong understanding of what product/market fit looks like before investing in growth. Founders and product leaders struggle with answering this question however, and advice and blog posts on the internet frequently espouse that you’ll know product/market fit when you see it, and that all of a sudden everything will start working. It’s not very actionable advice. I don’t claim to be the world’s foremost expert, but this is what I learned through scaling multiple startups, launching new products, and advising and investing in dozens of companies.

The Quantitative Approach to Product/Market Fit

I define product/market fit as satisfaction that allows for sustained growth. Satisfaction is tricky to understand because, well, customers are rarely truly satisfied. Jeff Bezos frequently drops pearls of wisdom in his shareholder letters, and my favorite of them is the following:

One thing I love about customers is that they are divinely discontent. Their expectations are never static – they go up. It’s human nature. We didn’t ascend from our hunter-gatherer days by being satisfied. People have a voracious appetite for a better way, and yesterday’s ‘wow’ quickly becomes today’s ‘ordinary’.

To put that in product/market fit parlance, product/market fit has a positive slope. The expectations of the customer continue to increase over time, and in fact, total satisfaction is likely an asymptote impossible to achieve. So what is product/market fit then? Product/market fit is not when customers stop complaining and are fully satisfied. They’ll never stop complaining. They’ll never be fully satisfied. Product/market fit is when they stop leaving. Represented visually, customer expectations are an asymptote a product experience can rarely hope to achieve, but product/market fit is a line a product can jump over and try to maintain a higher slope than over time.

All products start below a theoretical product/market fit line and some cross that line and work to stay above it over time.

So, for most businesses, instead of measuring satisfaction, measuring retention is the best signal of product/market fit. Measuring retention is pretty easy. Perform a cohort analysis, graph the curve over time and see if there is a flattening of the retention curve. As I’ve discussed before, what you measure for your retention curve matters quite a bit though. For your product, there is usually a key action the customer takes in a product that best represents product value. For Pinterest, it was saving a piece of content. For Grubhub, it was ordering food online. For your product, there is also a natural frequency to product use. For Pinterest, we eventually determined that people would browse sites for topics of interest on a weekly basis. For Grubhub, people usually ordered food once or twice a month. Once you have a key action and a designated frequency, the cohort graph should have the key action as the y axis and the designated frequency as the x axis. This does not mean companies should ignore other measurements of satisfaction, but to understand definitively what product/market fit looks like, this is the best start.

This graph measures a group of users and how many of them perform the key action of your product over time. There will usually be a stark drop at the beginning, but for products with product/market fit, a percentage continue to find value consistently.

If you’ve been around startups long enough, you’ve undoubtedly seen startups with retention of customers that struggle to grow. If you’re not growing, you definitely do not have product/market fit. So product/market fit cannot be measured by retention alone. That retention has to create sustainable growth, which means the rate of retention matters. Why does the rate of retention matter? Well, most acquisitions of new customers come directly from retained customers through a few key acquisition loops. Either retained customers: 

  • talk about the product to others to attract them to the product
  • invite people directly to the product to attract them to the product
  • create content that they or the company can share to others to attract them to the product
  • make the company enough money that the company can invest that money into paid acquisition or sales loops with a healthy payback period to attract them to the product

I’ve seen many founders misunderstand this, looking for growth hacks to drive growth or a PR bump. These types of tactics are only useful if they help you sequence to a sustainable growth strategy, but they rarely are sustainable growth strategies directly. This is why other measurements of satisfaction can still be important. If it doesn’t exist, the first two of these loops are impossible to drive growth from.

Sustainable growth is measured by one or more of these loops growing the size of your monthly acquisition cohorts month over month with a flattened retention curve. A flattened retention curve of your key action at the designated frequency plus month over month growth in new customers is the best way I have found to measure true product/market fit. If the rate of retention can’t support acquisition loops that continue to scale new users, retention needs to be improved to find product/market fit. Some companies can scale with 10% retained users, and some may need 40%, all depending on the strength of the acquisition loop. The graph below represents one month’s cohort retention over time vs. monthly growth in new users.

Consistent flattening of a month’s retention curve over time plus growth in new users every month is true product/market fit.

What if satisfaction of your product cannot be measured by retention? This happens. Some products are one time use or extremely infrequent in nature. In that case, I like to use custom surveys to measure the level of satisfaction depending on the nature of the product. Rahul Vohra describes the process they used at Superhuman for this, and it is excellent. Don’t forget to measure new user cohorts month over month though. Acquisition becomes even more important in these scenarios.

The Paths to Product/Market Fit

It takes most products a couple of years to find product/market fit. If you are not there yet by the above measurement framework, don’t be alarmed. The first step is to understand how far along you are and the approach to improve. There are two stages of the push toward product/market fit:

  • Building enough of the product vision so that the product is valuable and ready for customers
  • Getting customers to understand and receive the value you’re targeting for them

If you are in the first phase, you generally aren’t allowing customers to use the product anyway to measure product/market fit. There are two main schools of thought for how long you should stay in the first phase, which I will oversimplify into calling the Eric Ries model and the Keith Rabois model (unfair to both of them), and they are diametrically opposed on many axes including how long you build before you allow customer access. Thinking about these axes will help you understand what to do next and what successful companies did in this phase. I will outline some of the main differences below.

This is certainly oversimplified, but should help give an idea of the spectrum of things to consider with a new product. Let’s break these down in a little more detail.

Time To Market

The Ries model emphasizes talking to customers early and often to understand what to build, and whether what you have built is actually solving problems for them. The Rabois model is so driven by the vision of the founders of the company that customer feedback is less important than building what the founders have envisioned before any customers interact with it.

Success can be achieved by both modes. In the food delivery space, Tony Xu and the founders of Doordash and Arram Sabeti, the founder of Zerocater, famously used spreadsheets and founder manual labor to prove out their early models before investing in a lot of technology to scale. Shishir Mehrotra, the founder and CEO of Coda, spent four years building the product before any public launch, and now the company is growing quickly. In reality, companies that take longer to reach the market usually have alpha or beta customers that are still driving a feedback loop into the product. Coda used employees at Uber, for example. 

Focus

The Ries model tends to target a customer segment and attempt to find out their pain points to build something valuable. The Rabois model starts with a strong vision of both a problem and a target solution and works to build that from the start. The Ries model is common in enterprise business where customers just want to deeply understand data scientists or general contractors or some other segment, identify a problem they deal with a product solution could solve, then build and test with that segment. Here, you’re more likely to see some early pivots as companies try out different solutions to the same problems or target different problems of the same type of customer. We incubated a few of these types of companies while I was at Greylock with success, but it was rarely the first iteration that was successful.

One issue with the Ries model here is successful product/market fit rarely avoids all speed bumps. Without a strong vision and motivation by the founders and being “in love with the problem”, those speed bumps can instead appear to be roadblocks causing companies to change direction too early, causing a lot of “failing fast”, which is just a more acceptable way to say failing. The Thumbtack founders are a good example of grinding through these moments and not giving up on eventual product/market fit.

Many founders with vision put out a product experience and eventually find a market interested in that product solution. TikTok and Houseparty didn’t initially set out to build a product for children. Clubhouse started blowing up in Atlanta, and its founders are both in Silicon Valley. Square and Faire, however, had pretty strong ideas of their initial target markets and did find product/market fit with those markets.

Initial Launch Goals

The only goal of launching a product in the Eric Ries model is to generate feedback from the target customer. These launches are generally limited in size to receive enough signal from the target customers to know if something is working or not. The goal of a launch in the Rabois model is to achieve the initial vision that sparked the creation of the product. Launches are usually broader in this model, because the product may be looking broadly for a market that will be attracted to the product solution being launched. Even if there is a strong thesis for a target market, launches are more likely to go big to try to reach the maximum amount of the market possible, because the product should be ready to provide them value day one. In network driven businesses, a bigger effort out the gate may be required to reach the liquidity necessary for people to experience the product/market fit. In marketplaces, this may be sequenced by targeting supply or demand first, but with direct network effects, it is more driven by volume. Twitter and foursquare both launched at SXSW to get initial liquidity correctly. At Grubhub, whenever launching a new market, we would target to sign up 50 restaurants in the first two weeks to have a good selection to show consumers. 

Changes to Vision

The Eric Ries model is very flexible on vision, and companies on this side of the spectrum frequently pivot around customer needs quite a bit. Instagram, Twitter, Groupon, Slack, Pinterest, GOAT and many other successes started in one direction, and through either failure, seeing only part of their initial idea work, or fresh thinking altered their product and mission dramatically to find product/market fit. In the Rabois model, a singular vision drives the company from the start. Opendoor started with a mission to remake real estate through data science and instant sales, and has not strayed much from that vision. 

I think this is an area where despite all the news we hear about successful pivots that leaning more towards the Rabois model is a dominant strategy. Blindly trying out a bunch of startup ideas is like being in a dark room and feeling around for a door. A successful vision can turn on a light to that room so everyone can see the door and run toward it. Even many of those major pivots were guided by a strong, albeit new, vision from their founders.

Growth Strategy

The Eric Ries model thinks about product change as the main way to unlock growth in the early stages. This requires a tight feedback loop between customers and the team so the team can tweak and adapt and potentially built totally new things to please potential customers. You’ll typically see these types of companies leverage growth hacks that are long-term unscalable, but could get the initial product to more people efficiently. The Rabois model implies product changes will be more difficult. The product vision is usually taxing, so relying on product change to grow is very expensive. Rabois prefers to leverage a strong go to market model and heavy marketing muscle to scale usage of the product. Neither of these models go particularly in depth on scalable growth loops, and of course I believe thinking in terms of the scalable loops you’re unlocking is a dominant strategy.


Risks

It’s weird to talk about failure modes in starting companies as most companies do fail. But there are tendencies of failure types in each model that founders should be familiar with as they are avoidable. The Eric Ries model tends to lead to either iteration to no eventual destination or a lot of “failing fast” that feels like progress and isn’t. We saw a lot of this with startup studios five or so years ago. I can’t recall one of them creating an enduring company. The Rabois model has entirely different risks. This is where you can find solutions looking for a problem, like many crypto pitches you hear these days (how many ICO’s found product/market fit? Is there even one?). Some memorable failures in this area in the past are Color Labs and Clinkle. These companies can spend so much effort selling what they have really hard without ever confirming people want it. Startups in this area can also become so consumed in their vision they fail to actually launch. Magic Leap is a recent example of this in the AR space. The Rabois model can also hide some of the key insights to unlock the market if not careful. Instagram and Pinterest only had a small part of their original visions work, so they pivoted to focus on just that element to build massive businesses.

Applicable Company Types

While these models can be used for any type of business, they tend to lean towards certain models. The Eric Ries model is very common in enterprise businesses because founders are confident certain segments have day-to-day problems that aren’t solved and money to pay to create a reliable business model, so they learn all about that segment to find that problem with the willingness to pay to solve. The quick to launch elements are common in marketplaces where matching can be done manually to validate liquidity before investing in a lot of technology. The Rabois model is more common for hardware because iteration has such long timelines, and consumer models where typically founders have new habits or interactions they need to convince a broad market to try.

So Which Should You Use?

Hopefully, you see from these examples that either extreme is pretty dangerous, and even those that vehemently believe in one of these models over the other seems to have selective memory as to when companies veered away from their approach when appropriate. Founders should build a strong opinion over which parts of which model they need to apply to maximize the chance of finding product/market fit for their business. My personal belief is a strong vision combined with market feedback is a pretty dominant combination of these two approaches whereas many of the other axes depend on the product you are building. I actually find both models fairly weak on the growth strategy side. “Sell tickets” ignores how the product is frequently the most effective way to unlock growth through strong acquisition loops. Assuming new products equals new growth is a “build it and they will come” fantasy unless you get lucky.

—
The reason to understand what product/market fit looks like is of course important for founders of companies. While measuring the product/market fit line directly is impossible, measuring whether a company is above it or not is possible by measuring retention curves and new user growth. If the retention curves every month flatten and new user numbers increase at a healthy payback period, you can feel confident you have product/market fit. If you can’t say this for your product, I hope I’ve been able to give you some axes to understand what you need to work on and how to apply it to your specific product. The next problem is to scale it and keep as customer expectations continue to grow. If you want to learn more about these concepts, we expand on them dramatically in the Reforge Product Strategy program.

Currently listening to my Scrambled Eggs playlist.

Thanks to Kevin Kwok and Emily Yuhas for feedback on drafts of this post.

Feature/Product Fit

Through various methods, Silicon Valley has drilled into the minds of entrepreneurs the concept of product/market fit. Marc Andreessen says it’s the only thing that matters, and Brian Balfour has an amazing series of posts that talk about how to find it. But what happens after you find product/market fit? Do you stop working on product? I think most people would argue definitely not. Post-product/market fit, companies have to balance work creating new product value, improving on the current product value, and growing the number of people who experience the current value of the product. While I have written a lot about how to do that third step, and even wrote a post about thinking about new product value, I haven’t written much about how do that second part.

What is Feature/Product Fit?
Every product team tries to make their core product better over time. But sadly, at most companies, the process for this is launching new features and hoping or assuming they are useful to your existing customers. Companies don’t have a great rubric for understanding if that feature is actually valuable for the existing product. This process should be similar to finding product/market fit, but there are some differences. I call this process feature/product fit, and I’ll explain how to find it.

In product/market fit, there are three major components you are searching for. I have written about my process for product/market fit, and Brian Balfour, Shaun Clowes, and I have built an entire course about the retention component. To give a quick recap from my post though, you need:

  • Retention: A portion of your users building a predictable habit around usage of your product
  • Monetization: The ability at some point in the future to monetize that usage
  • Acquisition: The combination of the product’s retention and monetization should create a scalable and profitable acquisition strategy

Feature/Product Fit has a similar process. We’ll call this the Feature/Product Fit Checklist:

  • The feature has to retain users for that specific feature
  • The feature has to have a scalable way to drive its own adoption

Feature/Product Fit has a third requirement that is a bit different: the feature has to improve retention, engagement, and/or monetization for the core product.

This last part can be a bit confusing for product teams to understand. Not only do the products they are building need to be used regularly and attract their own usage to be successful, they also need to make the whole of the product experience better. This is very difficult, which is why most feature launches inside companies are failures. What happens when a feature has retention and adoption, but does not increase retention, engagement, or monetization for the company? This means it is cannibalizing another part of the product. This might be okay. As long as those three components do not decrease, shipping the feature might be the right decision. The most famous example of this is Netflix introducing streaming so early in that technology’s lifecycle, which cannibalized the DVD by mail business, but was more strategic for them long term.

What is a Feature Team’s Job?
You would be surprised how many core product features are shipped when the new feature decreases one of those three areas. How does this happen? It’s very simple. The team working on the feature is motivated by feature usage instead of product usage, so they force everyone to try it. This makes the product experience more complicated and distracts from some of the core product areas that have feature/product fit.

If you own a feature (and I’m not saying it’s the right structure for teams to own features), your job is not to get people to use that feature. Your job is to find out if that feature has feature/product fit. You do this by checking the three components listed above related to feature retention, feature adoption, and core product retention, engagement and monetization. During this process, you also need to determine for which users the feature has feature/product fit (reminder: it’s almost never new users). Some features should only target a small percentage of users e.g. businesses on Facebook or content creators on Pinterest. Then and only then does your job shift to owning usage of that feature. And in many teams, it’s still not your job. The feature then becomes a tool that can be leveraged by the growth team to increase overall product retention, engagement, and monetization.

Mistakes Feature Teams make searching for feature/product fit
Feature teams commonly make mistakes that dissatisfy the third component of feature/product fit at the very beginning of their testing.

  • Mistake #1: Email your entire user base about your new feature
    Your users do not care about your features. They care about the value that you provide to them. You have not proven you provide value with the feature when you email them early on. Feature emails in my career always perform worse than core product emails. This inferior performance affects the value of the email channel for the entire product, which can decrease overall product retention.
  • Mistake #2: Put a banner at the top of the product for all users introducing the new feature
    New features usually target specific types of users, and are therefore not relevant to all users. They are especially irrelevant to new users who are trying to learn the basics about the core product. These banners distract from them, decreasing activation rates. It’s like asking your users if they’re interested in the Boston Marathon when they don’t know how to crawl yet.
  • Mistake #3: Launch with a lot of press about the new feature
    PR for your feature feels great, but it won’t help you find feature/product fit. PR can be a great tool to reach users after you have tested feature/product fit though. It should not happen before you have done experiments that prove feature/product fit. And it will not fix a feature that doesn’t have feature/product fit.

Many features won’t find feature/product fit
Many of the features product teams work on will not find feature/product fit. When this happens, the features need to be deleted. Also, some older features will fall out of feature/product fit. If they cannot be redeemed, they also need to be deleted. If you didn’t measure feature/product fit for older features, go back and do so. If they don’t have it, delete them. Some of our most valuable work at Pinterest was deleting features and code. A couple examples:

    • The Like button (RIP 2016): People did not know how to use this vs. the Save button, leading to confusion and clutter in the product

    • Place Pins (RIP 2015): Pinterest tried to create a special Pin type and board for Pins that were real places. As we iterated on this feature, the UI drifted further and further away from core Pinterest Pins and boards, and never delivered Pinner value

  • Pinner/Board Attribution in the Grid (RIP 2016): Attributing Pins to users and their boards made less sense as the product pivoted from a social network to an interest network, and cluttered the UI and prevented us from showing more content on the screen at the same time

How do I help my feature find Feature/Product Fit?
All features should be launched as experiments that can test for feature/product fit. During this experiment, you want to expose the new feature to just enough people to determine if it can start passing your feature/product fit checklist. For smaller companies, this may mean testing with your entire audience. For a company like Pinterest, this might start with only 1% of users. The audience for these experiments is usually your current user base, but can be done through paid acquisition if you are testing features for a different type of user.

I’ll give you a few tactics that have helped the companies I’ve worked on find feature/product fit over the years. Most good product development starts with a combination of data analysis and user research. User research should be involved at the right times to add the most impact, prevent confirmation bias, and determine what components users are struggling to see the value of. For example, when we launched the Grubhub mobile app, we saw in the data when people used the current location feature, their conversation rate was lower than people who typed in their address. This turned out to be an accuracy problem, so we turned that component until off until we were able to improve its accuracy.

In research, we saw people were having trouble figuring out which restaurants to click on in search results. On the web, they might open up multiple tabs to solve this problem, but this was not possible in the app. So, we determined what information would help them decide which restaurants were right for them, and started adding that information into the search results page. That included cuisine, estimated delivery time, minimum, star rating, number of ratings, On the surface, the page now feels cluttered, but this improved conversion rates and retention.

Since Grubhub is a transactional product, we were able to leverage incentives as a strategy to help find feature/product fit. Our early data showed that people who started to use the mobile app had double the lifetime value of web users. So, we offered $10 off everyone’s first mobile order. This transitioned web users to mobile for the first time and acquired many people on mobile first. The strategy was very successful, and the lifetime value improvements remained the same despite the incentive.

Grubhub also uses people to help find feature/product fit. Since mobile apps were new at the time (and we were the first food delivery app), we monitored any issues on social media, and had our customer service team intervene immediately.


Not all complaints are this entertaining.

At Pinterest, we launched Related Pins in 2013. For Pinterest, we did not have a revenue model at the time, so tactics around incentives and people do not make as much sense. One thing we did instead was use notifications to drive feature/product fit. Once this algorithm was developed, after you Pinned something, we could now email you more Pins you might like that are related to that Pin. These emails were very successful.

Pinterest also used the product to drive feature/product fit. We launched the algorithm results underneath the Pin page to start, and interaction when people scrolled was great. But many people didn’t scroll below the Pin. So, we tried moving them to the right of the Pin, which increased engagement, and we started inserting related Pins of items you saved into the core home feed as well, which increased engagement.

The Feature/Product Fit Checklist
When helping a product find feature/product fit, you should run through this checklist to help your feature succeed:

  • What is the data telling me about usage of the feature?
  • What are users telling me about the feature?
  • How can I use the core product to help drive adoption of the feature?
  • How can I use notifications to help drive adoption of the feature?
  • How can I use incentives to help drive adoption of the feature?
  • How can I use people to help drive adoption of the feature?

When confirming feature/product fit, you need to ask:

  • Is the feature showing retention?
  • What type of user(s) is retaining usage of the feature?
  • How do I limit exposure to only those users?
  • What is the scalable adoption strategy for the feature for those users?
  • How is this feature driving retention, engagement, or monetization for the overall product?

Every company wants to improve its product over time. You need to start measuring if the features you’re building actually do that. You also need to measure if existing features are adding value, and if not, start deleting them. Asking these questions when you build new features and measure old features will make sure you are on the path to having features that find feature/product fit and add value to users and your business.

Thanks to Omar Seyal and Brian Balfour for reading early drafts of this.

Currently listening to Breaking by Evy Jane.

The Three Personas: How Marketing, Product, and Analytics Attempt to Define The Customer

In my career, I’ve worked in marketing, product, and analytics. While the American Marketing Association defines all of that as marketing, the reality is those are rarely under the scope of one functional team, and the people in those groups see things very differently much of the time. One of the key ways this manifests that creates confusion for the organization is in the creation of personas. All three groups have their own ways to define personas that don’t tell the same story. And in many cases, these are called marketing personas even though they are very different. I’ll walk through each of them to try to define them separately, and talk about how to use each of them best and avoid common pitfalls.

The Analytics Persona
The analytics persona is the most straightforward of the group. The analytics persona is created by looking at clusters of users based on their usage and defining them based on that usage. At Pinterest, these were defined as core, casual, marginal, and dormant users. Core people came every day, casual people came every week, marginal people came every month, and dormant users had stopped coming to Pinterest altogether. This segmentation can be useful to see if your product is becoming more or less engaging over time. Key projects can include migrating people from, say, marginal to casual, by understanding the statistical differences between the two groups. Then, a product team might take something the casual group does that the marginal group does not and try to make the marginal group try to do that thing as well. Some of these differences are just correlation (or represent the people more so than the action itself being important), but some may be causal, and experiments like incentives or education to marginal users may help them become casual users.

The Product Persona
The product persona, just like the analytics persona, focuses on understanding existing users. In contrast to the the analytics persona, it relies on qualitative research to define who these people are, not just what actions they take. Someone that uses the product monthly can be the same persona as someone who uses it every day. When we built our personas at Grubhub, we had to use a mix of qualitative and quantitative research to define them. After a back of forth of customer calls and surveys, we were able to define four personas that used Grubhub based on two specific criteria: whether they ordered spontaneously or planned ahead of time, and whether they ordered for themselves or with others. When we mapped these personas back to our data, we were able to find that one segment was detrimental to serve because of high customer service costs, and that one segment was high potential, but low value currently because we had not built the right product for them yet. This helped inform our product strategy for the future.

The Marketing Persona
The marketing persona, unlike the first two personas, is usually a forward-looking persona: about who the company is going to try to reach. These are customers you want rather than customers you have. Marketing personas are common for product launches and market expansions since there are no existing users or data to build analytics or product personas.

The marketing persona exists to define a target market to go after. Marketers rarely try to target all people. They attempt to define a niche with specific needs (physical and emotional) and make their product attractive to that group. Marketers have many tools to define this persona. They pore over demographic and psychographic data and map competitive landscapes to spot opportunities for new markets. Since this persona is about targeting people outside the product, one common tool created during this process is a mapping of the target customer’s typical day. What they listen to on the radio on the trip to work, where they get their morning coffee, what they watch on TV during dinner, etc. From that, marketers identify opportunities to advertise to the target during one of those times to introduce them to the product.

Issues with Different Personas
All of these personas have pros and cons, and I recommend using them in tandem rather than a one size fits all approach. The danger with the analytics persona is it looks at people solely based on their activity and not also their motivations, their personalities, etc. There are key insights you will never find out from data that you can learn within ten minutes by talking to a customer. The product and the analytics personas also assume the current customers are the most important customers to understand. This is not always the case. Many times, you need to expand your market. Other times, focusing on existing users makes the product worse for incoming users.

Marketing personas also have their flaws. Since they are based on secondary data, marketers can sometimes invent personas that don’t exist or are too small to be important. This is why so many marketing teams create personas that are just cooler versions of themselves. It’s why whenever a fashion designer is asked who they design for, they say something like “a gallery owner in New York City.” They are maybe 500 of those people in the world. Now, I should be clear in that last scenario there is a blurred line between the marketing persona and the aspirations of the true persona, but you get the idea.

Marketing personas also can be unhelpful to some organic growth strategies or for products that have infinitely large markets since they are based on niches and targeting. When we tried to build marketing personas to target at Pinterest at around 50 million users, I asked, “What exactly are we going to do with this? We don’t spend money on advertising. Our targeting is defined by whoever searches for something on the internet that Pinterest is relevant for.” Similarly, does Google search have a target market? Sure, if you can call every single person with an internet connection a target. I bet you the Google Home team had a very specific target in mind for its launch though.

Personas of all types are also mistakenly used in lieu of personalization for many internet products. Email is a common example. Many companies still spend a lot of time creating dedicated emails for specific personas, segmenting lists, writing custom copy, and adding custom content. This frequently doesn’t makes sense in a world of personalization and one to one messaging. Pinterest has automated, personalized one to one messaging across email and push to over 200 million users. It doesn’t need to know what analytics, product, or marketing persona you are to be effective. This is not proprietary tech either. Many third party companies can help build this same type of system for smaller companies with less data.

—
Personas are very useful tools to help you identify opportunities to grow your business and better serve your customers. You should be using them in almost all phases of a company. Understanding the different types of personas, how you could use them, and how to prevent making mistakes with them is key to making sure they are worth the effort to define them.

Currently listening to Big Loada by Squarepusher.

Product Visionary vs. Product Leader

Many people want to work in product management. One of the most common questions I receive is how to break into product management. It’s a hard question for me to answer, because 1) there is no default path (the same is true for trying to land a business development role), and 2) most of these people really don’t know what they’re asking for. My most common response is, “Are you sure? Product management can kind of suck.” The reason for the dichotomy of people who haven’t done product management finding it so alluring, and people who have done it cautioning people trying to get in is the difference between what I call a product visionary and the product leader.

Product visionaries are who we all hear about in the press. They are the people who come up with brilliant products that go viral or solve real needs in the market that no one else thought of. They appear to be masters of finding product/market fit. I’ve been lucky enough to work with a few of them in my previous jobs and a few in the portfolio at Greylock. These tend to be founder/CEOs, and they generate brilliant insights that create product opportunities others don’t see. Ev Williams is on his third breakout product in Medium after Blogger and Twitter. Ben Rubin created two products that hit product/market fit in two years with Meerkat and now Houseparty.

Anyone working in technology hears these stories, and they think the shortest path to that sort of glory is becoming a product manager. They are excited to get to a role where they can drive the vision of a product, even if it’s only one part of the company. This excitement is exacerbated by the commonly propagated myth that the product manager is the mini-CEO of their product. The reality is that in 90+% of cases, product management is not about being a visionary. It’s about being a leader.

What does a product leader do at a tech company? It’s actually very little of creating a vision and a strategy from scratch. It’s about helping everyone understand what the vision and strategy is. It’s about communicating to the entire team why the company is doing what it is doing. It’s about building a process that helps a team execute on that vision. It’s about when there are competing visions, aligning and motivating the team to focus on one, and getting people to disagree and commit (including sometimes yourself). It’s about looking at data to measure if product changes are having a positive impact on the customer and the company’s growth. It’s about talking to users to understand why they’re doing what they’re doing, and the problems they still face even though your product exists. It’s about mentoring more junior people on your team, across product as well as engineering, design, and analytics. And they don’t have to listen to you, so you have to use influence rather than authority to be successful.

In the scaling phase of a startup, it’s product leadership that drives performance, not vision. Vision is needed early to find product/market fit and plot a course to scale, and then the less that vision wavers over time the better. This vision is usually done by the founder/CEO. The reason founders hire product managers and VPs of Product is not to set vision, but to help execute the vision. Don’t get me wrong; that will sometimes mean coming up with solutions to problems your customers face. If a company is scaling by having the founders solve all the customers’ problems instead of product teams, it will struggle. But much of the time, it will be wrangling the ideas of the individual engineers, designers, and analysts on your team and matching that to an overall vision set by the founder(s). It’s very rarely your ideas you’re executing on as a product leader, and it shouldn’t be.

https://twitter.com/stevesi/status/943945949352665088

I also don’t want to make it sounds like I am devaluing visionaries. They are, of course, critical in finding the initial idea(s) that create a growing company and maintaining a vision for that growing company. Having visionaries also becomes more important as you saturate your core market and need to tackle new value propositions to drive new growth opportunities. That is the ideal time for non-founder visionaries to enter a growing company. These are not going to be typical product managers or VP’s though. They are usually ex-founders. The best outcomes for these people entering an organization that is scaling is giving them a team and space to experiment with ideas until they find their own product/market fit, where a business unit is built out around that vision with product leaders to help them scale.

As you’re thinking about your business, think about whether you need a product visionary or a product leader. Most founder/CEOs are already visionaries, so they need a product leader to help execute. Some businesses minded founders need the opposite to be successful. If you’re thinking about product management, think about whether what you really want to be a is product visionary instead of a product leader. In that case, it might be a better idea to start a company than take a role expecting to execute on your vision and instead managing other people’s visions.

Currently listening to Ambivert Tools Vol. 3 by Lone.