How many interviews? An important question, without a simple answer.

One of the first questions we often hear from prospective clients is: “How many win-loss interviews should we conduct?” It’s an important question without a simple answer.

A few months ago, while listening to a panel of win-loss practitioners respond to this question, I was surprised by the variance in their answers. One self-proclaimed expert explained that as few as 8 interviews can be sufficient in certain circumstances - which may be true. But, he concluded by saying “It’s statistically significant, and I’m a statistician.” His comment raised a few eyebrows, including mine. In quantitative research, statistical significance typically isn’t achieved until you have around 400 responses for a given population. (Just for fun, go play around with this sample size calculator by Qualtrics).

When it comes to qualitative research (more in-depth, time consuming, and expensive than quantitative research) it’s rarely feasible to achieve so-called “statistical significance.” But that’s okay. The goals of each practice are different and each methodology has its unique benefits. To learn why qualitative interviews are so important for win-loss analysis, check out this previous blog post.

So, what’s the right number of interviews to conduct for win-loss analysis? To get to the best answer for your organization, here are some factors to consider:

‍

Consider your pipeline.

First, consider your pipeline to determine the raw number of potential interviews.

Let’s say you only have 200 total sales opportunities that flow through pipeline each year. Participation rates can vary widely, but let’s assume a participation rate of 40% for wins and 25% for losses -and- we’ll assume that your pipeline win rate is 25% (25% wins, 75% losses). That means that throughout the year you have the opportunity to conduct approximately 20 win interviews and 38 loss interviews - a total of 58 interviews. In a case like this, we’d recommend pursuing interviews with every opportunity.

But things are often more complicated. Let’s assume you work for a global company with multiple business units and thousands of deals flowing through pipeline each year per business unit. Following the same logic as above may yield a total number of potential interviews in the thousands. Exciting? Yes. Feasible? Probably not. So, that’s when we move to the next step.

‍

Consider your budget.

If interviewing every possible account is overly ambitious for the size and scale of your pipeline, that’s okay.

Start by approaching win-loss analysis at the business unit or product line level. Win-loss insights and themes will vary widely across units and product lines - so much so, that there is usually minimal incremental benefit to running a singular program across all divisions. So it’s often preferable to scope separate programs for each unit or product line - which is also easier from a budget allocation standpoint.

Then, consider the available budget for that unit or product line. In qualitative research, some data is always better than no data. As you’ll discover below, it’s possible for any business - of any size - to derive meaningful insights from as few as 10 interviews. But, the more interviews the better. Most Clozd clients eventually settle into ongoing programs of 50 to 200 interviews per year - based primarily on what they can afford. More interviews means richer insights and stronger conclusions; but, a few interviews is always better than none.

‍

Focus on saturation, not significance.

Once you determine how many interviews you can afford, the next consideration is how to deploy them most effectively. The goal, in qualitative research, is to conduct enough interviews to reach the point of “saturation.” If you narrow your area of focus, you can reach saturation faster. For an explanation of saturation, check out this previous blog post. Saturation is to qualitative research what statistical significance is to quantitative research.

For example, 10 interviews may not sound like very many; however, it may be possible to narrow your focus enough (e.g., product X deals in our newest sales region) to achieve saturation for a certain pocket of your business. It’s an art not a science. Our consultants help our clients determine (i) strategic segments of your business worth focusing on and (ii) identifying the point of saturation.

For example, while running a program for a large email marketing organization, it wasn’t financially feasible to interview every buyer at every account. So, we focused our efforts on competitive deals versus a single competitor. After several quarters, we felt that we had achieved saturation - there would be diminishing returns of additional interviews for that segment at that time. So, we shifted the program focus to a different competitor, until saturation was achieved there too. Following this approach we helped the client maximize their investment. The same approach can be taken in relation to geographies, customer segments, deal sizes, product lines, etc.

Ultimately, there’s not a one-size-fits-all answer to the question “How many interviews should we conduct?” Take the principles above into consideration, and then reach out. One of our partners would love to conduct a free consultation to help you determine the right number of win-loss interviews for your unique business and circumstances.

Talk With Us