“This is the best thing I’ve ever tasted!”

My wife and I were sitting on the patio in front of the Jolly Holiday Bakery at Disneyland eating raspberry rose macarons.

“I’ll make them for you when we get home! They can’t be that hard to make,” I said.

I was wrong.

I had no idea how finicky macarons are to bake. The slightest thing can ruin a whole batch.

Too much mixing. 

Not enough mixing. 

Too much heat. 

Not enough heat. 

Too much moisture in the air. (Yes, you read that right: In the air!)

I’m on my fourth batch of macarons, I’ve spent over $100 on ingredients and supplies … and I’ve yet to make a batch I’m proud of. I’m trying really hard not to think about how much each attempt is costing me–both in money and time.

One batch got burned on the outside but was uncooked and gooey on the inside.

One batch dissolved into a soupy mess when I tried to add the filling to shells that were still warm out of the oven.

One batch’s batter was too thick, and the cookies were all lumpy and deformed.

As it turns out, I fell victim—over and over again—to the Dunning-Kruger effect, completely overestimating my own competence while underestimating the difficulty of baking this notoriously challenging cookie.

And it didn’t feel great.

So, how does this relate to win-loss analysis?

If you’ve ever debated hiring a third party to run your win-loss analysis program vs. just doing it yourself, you might be on the verge of making the same mistake I did with my macarons.

Because just like baking macarons, running a successful win-loss analysis program is far more complicated than it initially seems. 

I’m going to shine a light on the intricacies of running a “well-baked” (pun intended) win-loss program so you can make an informed decision about whether or not you want to build your own DIY win-loss analysis program or outsource it.

Keep an eye out for links to additional resources scattered throughout this post. They’re meant to help you dive deeper on certain topics.

General pros and cons of a DIY program vs. third party

Before we jump into the tactical details of running a win-loss analysis program—and there are many—I think it’s important to compare some of the overall costs and benefits of running a program in-house vs. hiring a third party.

Learning your business vs. knowing how to interview

When you consider hiring a third party, your first thought might be, “Nobody knows our business, products, internal politics, or customers better than we do. How could a third-party win-loss provider ever learn enough about how we work and what we do to run this program effectively?”

That makes total sense.

You know your business better than anyone

When it comes to understanding the nuances of your products, or which leaders within your organization can drive change, you’re the expert.

You understand the problems your customers face and how to solve them. You know your products and their features, integrations, and applications. You’ve learned the industry jargon and know how to talk to prospects who fit your ideal customer profile.

Your ability to why a buyer doesn’t purchase improves as your knowledge of your buyers improve.

A third-party provider likely won’t have the wealth of experience that your product marketing, sales, or rev ops team does.

That’s one point in favor of running a DIY program.

But being an expert on your products, customers, and industry is only part of what makes a good win-loss practitioner.

Conducting strategic buyer interviews is a skill

Most people underestimate—or don’t even consider—the learning curve of conducting probing, informative, high-value win-loss interviews. It takes a lot of practice to quickly develop rapport with a buyer and keep an interview moving along while also knowing when it’s important to slow down and dig deeper for the most valuable insights.

It takes time to develop the right follow-up questions that provide context around complex and nuanced issues and to get a feel for when you’ve landed on a truly important Decision Driver (a factor in the sales process that impacts the outcome of a deal).

In our experience, companies that decide to run DIY programs spend 3–6 months interviewing and surveying customers only to realize that their interviews were inconsistent, directionless, and lacking enough data to draw any meaningful conclusions about their buyers. At that point, many programs get shut down because leaders can’t justify investing more time and resources into a program that doesn’t produce a clear ROI.

In most cases, developing the skill of performing high-impact interviews takes more time than learning enough about a company’s industry, products, and customers to perform a series of interviews that gather valuable, revenue-boosting data.

"I like the summary of key features/benefits that helped us to win/lose a deal. The way [Clozd] categorizes information and pays attention to normalizing the experience has been good as a way to filter responses. Additionally, the interviewers are adept at 'following the clues' during a conversation -- this is not always a strong competency of 'in-house' interviewers. Clozd has hired and trained their staff really well on this dimension.”

Misha Gautreaux | Vice President of Sales at Auctane

In most cases, this is a point in favor of third-party providers.

Combating bias

Buyer feedback is only valuable when it’s honest and unfiltered. (Nobody wants to build their go-to-market strategy on incorrect or incomplete data.) It’s kind of like experiencing your first romantic breakup—you ask your ex why they don’t want to date you anymore, and they respond: “Oh, it’s not you, it’s me.”

But deep down, you know that’s not the full truth.

Then one of your friends does a little investigating and finds out that your crush hated your bad breath, didn’t like how slow you were to respond to their texts, and was bothered that you said something to offend their best friend.

Just like your ex, buyers tend to pull their punches when talking directly to the seller. They don’t want to burn bridges, hurt feelings, or initiate an awkward conversation where they list off everything a rep did wrong.

Buyers are more likely to speak with an unbiased third party about their experience (higher response rates), and they tend to be more honest and forthcoming with their negative experiences when talking to someone not directly associated with the deal resulting in more accurate data.

We do post-mortems for many of our large, strategic deals whether we've won them or lost. And I think we get decent information from that. However, the Clozd feedback gives us way more insight than we get from our sales reps and individuals engaged in the deal. I've been thoroughly impressed by how deep and relevant the insights are. Every Clozd review I read, I legitimately learn new things that we can improve about our sales process.”

Brandon Hanson | Global Head of Customer Care CX Strategy at Qualtrics

‍That’s one point for third-party providers.

Employee churn

Let’s imagine that you invest a year of time and resources getting your DIY win-loss analysis program up and running.

You’ve avoided most of the common pitfalls (because you did your homework) and things are humming along nicely. You’re getting high-value feedback from your efforts, and your sales, marketing, and product teams are improving based on the insights they’re gathering from your buyers.

Then, one day, the administrator of your win-loss program turns in their two-week notice.

It’s common for win-loss analysis programs to die with the departure of their internal champion, leaving leaders in a panic, wondering how they’re going to keep the program going.

A perk of working with a third party like Clozd is that we have an entire team of trained win-loss experts who understand how to conduct amazing win-loss interviews. They know your industry, and they’ve worked with clients with similar business models and ICPs.

That means your program will continue humming along no matter what happens with your employees internally.

That’s another point in favor of third-party providers.

Opportunity costs

There’s a real opportunity cost to managing your DIY win-loss program.

For every minute your employees spend doing administrative tasks like sending emails, rescheduling missed appointments, and editing transcripts to keep your win-loss program running, they can’t be championing change, discovering ways to make or save money, or developing strategies to dominate your category. 

Working with a third party means you’ll be able to spend less time managing win-loss logistics and more time acting on win-loss insights.

“[Clozd has a] seamless interview scheduling and customer reach out [process]. We do not have to worry about the process, and it gives us the time we need to concentrate on other initiatives.”

Shashank Shekhar | Competitive Intelligence Manager at Cohesity

Clozd has developed industry-leading technology and processes to automate the most time-consuming and painstaking elements of conducting win-loss analysis—from setting interview appointments and transcribing and tagging the meaningful insights in buyer conversations to integrating your CRM, survey, and interview data all into one dashboard that enables detailed and dynamic reporting into what’s actually driving revenue in your business.

This allows your employees to spend most of their time diving into the data to find meaningful insights that can help you win more deals, keep more customers, and generate more revenue. 

Another point in favor of third-party providers.

Don’t take our word for it …

Now, I realize that it might be hard to take this article at face value considering we’re a third-party win-loss provider, so here’s a direct quote from someone who’s been in your shoes:

"If you’re trying to do win-loss analysis in-house, consider getting a third party like Clozd to achieve an objective set of results. We have often found this to be true, because when you take out the feelings/emotions that an in-house analysis will undoubtedly surface (from product, marketing, sales, etc.), you will get actionable feedback from prospects and customers alike. This result is harder to come by doing this in-house when you are potentially 'leading the reviewer' in a particular way to make things look/feel better.”

Misha Gautreaux | Vice President of Sales at Auctane

What actually goes into running a comprehensive win-loss analysis program?

Now that we’ve talked about some of the general pros and cons of a DIY program vs. hiring a third party, let’s get into the tactics. As you read this section, instead of thinking, “Can we do this?” it might be worthwhile to consider, “Are these the most valuable tasks my team should be spending their time on?”

Step 1: Focusing your efforts on a goal

Win-loss analysis programs are most useful when you leverage them to solve specific problems, such as: 

• Why are we losing to X competitor?
• Why are we underperforming in X market?
• What elements of our product are preventing us from winning more deals?
• Why are we losing customers?

Someone has to choose a goal

Before kicking off your program, someone needs to choose a goal.

This will likely require meeting with executives and revenue leaders to decide which problems are a priority, and what they’d specifically like to know about their buyers’ experience.

This will not only give you a target to aim the program toward, but it will also help to get your leaders invested in the outcomes of the program.

Create an interview guide

Now that you have an idea of the problem your win-loss program will be attempting to solve, you need to create an interview guide. An interview guide is a list of open-ended questions that help you uncover key insights about your sales process, product, competitors, and more.

Attempting to improvise your buyer interviews will almost certainly end up being a waste of time. You still might discover an interesting insight or capture a meaningful anecdote, but without any structure, you’ll lack consistency in future interviews—which makes it virtually impossible to prove that your findings are consistent enough to warrant change.

Here's a training on how to create a solid interview guide.

Step 2: Getting feedback from the right people

Once you have clarity around the problem you want to investigate and you’ve developed a framework for your approach via your interview guide, it’s time to start reaching out to qualified buyers.

Getting a list of buyer data

It’s common to run into roadblocks when you start trying to get your hands on buyer contact information.

Sales leaders and reps can be protective of the contacts on their accounts. They might want to look over any list you pull from the CRM, and they may even require you to approve the outreach messaging and any questions you might ask them before they give you access to buyer data.

This process of pulling lists and approving contacts can take days or even weeks, which can be discouraging.

Additionally, a cherry-picked list of contacts often won’t accurately reflect the buyer experience. If you only talk to people who had a good experience, you run the risk of missing out on the perspectives of buyers you can learn the most from. 

Getting buy-in from leaders and reps

For your program to have any positive impact on the business, you have to have buy-in from the leaders who can actually enact change. Providing leaders with transparency into your process from the outset can be a good way to gain their interest and trust.

When they know important closed-lost deals or consistent revenue-related issues are getting your attention, they’ll be curious to see the results.

Step 3: Buyer outreach

Now that you’ve got the contact information of qualified buyers, you need to reach out to them and ask for feedback. This can easily be the most frustrating part of the process because there’s so much that lies outside your control. 

People are reluctant to respond to invitations to be interviewed or surveyed.

Or they do respond but they don’t show up.

Or not enough people respond to your outreach to get a meaningful sample size.

Plus you have to keep track of dozens of email conversations coordinating meeting times.

Getting your buyers to talk to you after a deal has closed is really hard.

It’s not one (email) and done

The first mistake people make when doing this for the first time is they assume that they can send out one email to their list of buyers and get an awesome response.

At Clozd, we’ve found that the majority of buyers actually book interviews after multiple outreach attempts.

It takes time, and you need to develop a cadence of email outreach that doesn’t make you seem overbearing, disrespectful, or annoying.

Offering incentives

One thing that helps improve response rates (especially for more introverted buyers) is offering an incentive in exchange for an interview.

But there’s a science to incentives.

If you’re not strategic, incentives can burn up your budget really fast. But if you’re smart with how you use them, they can help you get highly valuable buyers on the phone who will teach you some incredible things about your business.

Setting appointments

Finding a time to meet can be tedious. Few things are worse than passing emails back and forth trying to find a date and time that works for both of you.

Using a tool like Calendly can be a real timesaver.

Don’t get discouraged if a decent percentage of your interviews get canceled at the last minute or if you have no-shows. Just follow up and try to reschedule.

Polite persistence is the name of the game here.

Here's a full training on how to successfully reach out to buyers.

Step 4: Conducting the buyer interview

Now that you’ve spent dozens of hours building interview guides, pulling contact lists, sending and replying to emails, and chasing down buyers, it’s FINALLY time to perform the interview.

Most interviews should last around 30 minutes. Any shorter and it’s hard to get any meaningful insights out of the conversation. Any longer and it can start to feel like a burden to your buyers.

A good interview is a delicate balance between following your interview guide without sticking to it too rigidly. You’d be surprised what insights you stumble upon if you pull on some of the loose threads your buyers offer you during an interview.

Just remember, the interview has a specific objective. Always loop back to the points that are most important to the outcomes of the program.

Here's a deep dive workshop on how the top win-loss experts conduct buyer interviews. 

Step 5: Transcribing interviews and tagging insights

If done right, your buyer interviews will be your most valuable source of buyer and customer data. They’ll be chock full of experiences, insights, successes, and failures that can shape the future of your company.

But those interviews won’t do anybody any good if they just sit in a Google Drive folder on your computer.

In order to extract the value from your win-loss interviews, you should transcribe them, study them for trends and insights, tag them, and deliver that information to relevant stakeholders throughout the company.

Reviewing the transcript for accuracy

If you choose to transcribe the interviews yourself, plan on spending at least two hours on each interview—unless you’re the fastest typist in the world.

The other alternative is to use a transcription service or software. These services can save you a lot of time, but they can be expensive and will still require you to manually review them for errors.

How do you keep track of the most important insights for each team?

As you review each deal, make sure you tag the most valuable quotes so you can find and categorize them later.

In order to do this productively, you should create a tagging taxonomy before you begin. Decide what “buckets” or Decision Drivers you want to categorize your discoveries into. Adjust those Decision Drivers as you go, but make sure to avoid making them too broad or too narrow.

As your library of win-loss interviews continues to expand, keeping track of all the insights you gather in your win-loss program gets more complicated. It’s common for in-house win-loss practitioners to have documents containing hundreds of pages of interview transcripts.

Here's a great workshop that lays out how we transcribe and track win-loss insights.

Step 6: Reporting your findings to stakeholders

The ultimate purpose of conducting a win-loss analysis program is to leverage buyer feedback to create positive change at your company. The best programs help teams improve their products, increase sales, reduce churn, and more.

But here’s the catch: You can unearth the most revelatory, groundbreaking data in the world, but if nobody in your organization knows about it, nothing will change—and all your efforts will be wasted.

Reporting your findings to people who can (and will) actually create change is where most win-loss programs most fall apart.

There are two major obstacles you’ll face when reporting your findings with a DIY program:

1. Organizing and reporting on the data you have housed in different formats on different platforms (CRM, interview transcripts, surveys, etc.)
   
2. Convincing internal stakeholders of the value of your discoveries
   

Organizing your data

Imagine that you’ve spent the last three months interviewing, surveying, and getting feedback on dozens of deals.

As the date of your program readout approaches, you realize you have to somehow make sense of 100+ pages of interview transcripts housed on Google Docs, survey responses from your Qualtrics or Surveymonkey platform, and your CRM data.

If you’re an Excel wizard or a former database administrator, you might not have much trouble with this.

But for the average program administrator who has seven other projects on their plate, making sense of all this data might feel like a task that borders on impossible—and it just gets harder as your program grows and evolves.

The beauty of going with a third party provider like Clozd is that we specifically build our software to make it easy for you to report on all of your win-loss data—from any source.

With the click of a few buttons, you can view all of the deals lost to a particular competitor and look at the macro reasons you’re winning and losing. Then you can click into a specific deal to read direct quotes from your buyers and hear—in their own words—what they liked and disliked about your product or sales process.

You can also check the accuracy of your sellers’ perspectives by comparing your reps’ closed-lost reasons logged in your CRM with the reasons identified by your buyers. (On average, buyer and seller data only match up about 18% of the time.)

There’s power in making your win-loss data accessible to everyone who needs it and equipping them with dynamic reporting capabilities so they can suss out the insights that are most valuable to them.

Getting others to take you seriously

The next challenge we see win-loss practitioners face is convincing leadership that their discoveries aren’t biased, insignificant, or incorrect.

You’ll get questions like:

• What questions did you ask in this interview? How do I know you weren’t leading the witness?
• Are the insights you’re showing us statistically significant? How many deals did you actually get feedback from?
• How do we know our buyers were telling you the truth? What if they were just being nice?

When your win-loss interviews are conducted by a third party, buyers tend to be more responsive and more honest with their feedback.

Your response rates on outreach will be higher, and the insights you glean will be more powerful.

Third-party providers also don’t have the baggage of internal politics to contend with. Their job is simply to surface the data and let the leaders do what they want with it—avoiding the politics that often come when a win-loss program is run out of a certain department.

Find out how we organize and report win-loss data to stakeholders.

So, do you really have the bandwidth to run a win-loss program internally?

Win-loss analysis can increase your win rate by up to 50% and significantly reduce churn, but (as this article illustrates) successfully running one internally requires a unique person with a specific skill set.

• They have to be a prepared, unbiased interviewer
• They have to be incredibly organized and tech-savvy enough to set up complex marketing automations
• They have to be ok with performing repetitive, menial tasks
• They have to have the technical and analytical capacity to identify trends in your data—and then build reports that integrate multiple data sources
• And they have to have enough trust within the company to be taken seriously when they present their findings to leadership

That’s a lot, and if you do end up having an employee like this, do you really want them to spend most of their time performing menial, administrative tasks like exchanging emails and sending out calendar invites?  Or do you want them to study dynamic reports, discover hidden problems in your business, and champion change? 

And what will you do when that employee leaves?

Ultimately, a lot of time and effort go into each buyer interview. It’s a complex and challenging task to administer an ongoing program that covers a significant portion of your sales pipeline and gives you a clear view of your strengths and weaknesses.

And that’s why win-loss analysis is so valuable! 

Most companies simply aren’t willing to go through the painstaking process to obtain this impactful but hard-to-obtain feedback from their buyers.

“You're likely to underestimate the time required to consistently schedule, conduct, transcribe, and analyze a meaningful volume of win-loss interviews.”
‍
Cameron England |“The Win-Loss Guy”

Those who succeed are able to build a huge competitive advantage.

Ultimately, if what you care about is having a deep and ongoing understanding of why your buyers are, or are not buying/churning, hiring a third party is the obvious solution.

Most of our clients find that if they can use win-loss data collected through a third-party provider to help them simply win one more deal, it more than pays for the cost of their annual program.

If you’re interested in calculating the potential ROI of working with a third party, check out our Win-Loss Analysis Revenue Calculator here.