Riley Martin:
Welcome to this session of WinLossWeek, the world's largest online event dedicated to win/loss analysis. This event is hosted by Clozd. We'd like to thank all of our partners for their contribution to making this event a success. The title of this session is, From Pilot to Company-wide Collaboration: How to Optimize the Impact of win/loss. We're joined by Eric Judd, who works as a research manager at Pluralsight, where he leads market research and insights that enable data-driven decision-making across the organization. He has sealed as a team leader and analyst with experiences as [inaudible] modeling, forecasting, research design and database management. Eric, thanks so much for joining us today.
Eric Judd:
Yeah. Thank you. Glad to be here.
Riley Martin:
I'm super excited to hear what you have to share. So let's go ahead and get started.
Eric Judd:
Okay. Jumping right in. We've had a great experience with Clozd and I've been able to get some good, consistent insights. Stuff I'm going to talk about today has to do with how we initially started working with Clozd, the steps we took to go from an initial first steps of few small interviews to building it out to involving stakeholders from across the entire organization. To set the scene with where we were at last year, 2019, we were in a place in the marketing department and a number of organizations where we needed to understand better what was going on with our customers. We'd had a miss on an earnings report and we had lost a few opportunities. We knew what was going on. We knew the general reasons why we had lost, but we didn't have a great grasp on the why.
We knew it was things like pricing or a number of other topics, but we weren't quite sure how the customers were feeling about that, what we could do, how we could pivot to best meet the needs of the customers and how to position ourselves moving forward. We already had a number of data sources informing us as to what was going on. In salesforce.com, reps would manually enter in from a pick list, reasons why we won and lost counts. So we had a general idea there, but it was still lacking in that we couldn't read between the lines. It was a little too static for us to make good data-driven decisions. And in addition to loss codes from salesforce.com, we had market research, user surveys, customer surveys, NPS surveys, a number of things that were giving some context, but it was insufficient.
And we needed something where we could have back and forth conversation, really to drill down to ladder down into some of the issues that customers were seeing and also to understand when we were being successful, why we were being successful and how we give us to capitalize on those types of opportunities moving forward. So in short, we recognized that we needed to hear it straight from horse's mouth. We needed to build a customer and doing win/loss interviews allowed for an environment for customers to give feedback and in a very effective, meaningful way.
So setting the scene, we had little budget available and we saw that small pilot program with Clozd would be the best way to kick things off and to get things moving. Okay. So we started off with Clozd. We got a small pilot program going, and as I initiated this, I did want to make sure I didn't make a mistake that I've seen done on prior research projects. And that's taking on too much, trying to do too much with too little resources. And so I really worked hard at the beginning of our engagement with Clozd to make sure that we just stayed on target. Focused on the right key issues, don't try to tackle too much. We don't try to solve too many problems. And at the same time, just at the get-go, we decided to focus on one segment of customers, one geography, just keep it simple and make sure that we solve the most pressing issues.
And we identified those issues as we knew we needed to possibly make some tweaks to pricing and packaging, make the packaging a little bit more clear for customers to understand. Make sure we were positioned well with pricing against competition. We weren't quite sure what perceptions were out there regarding our sales process. We wanted to understand that better and understand where in the sales process things were being successful, but also where any breakdowns were occurring. And in that sales process, we also wanted to get a sense of, are we talking to the right buyers, buyers in one department could act very differently than buyers in another department. So getting a sense for who's right the contact we would talk to. We wanted to really hone in on that. And then finally, we wanted to get a better sense for how we were doing with meeting the needs of the customer, with the jobs that they needed to have done with our product.
Okay. So we started with a small pilot program. Even with [inaudible] , I knew that sometime in the future, I wanted to extend this out to a full-scale program where we brought in individuals from across our entire organization, we're approaching 2000 employees. So we have a lot of stakeholders and a lot of opportunity to provide insights for many different teams. So I saw the opportunity from the get-go to get buy-in from a number of different organizations. Not only to get buy-in, but it was going to be impactful to get input from stakeholders, from a number of different teams. So I put together a Win/Loss Stakeholder Council. And with this team of representatives from a number of different organizations, we had a representative from portfolio marketing from our product and experience teams from a revenue organization. We had a number of representatives and also from our strategy team.
And together, we were able to work together as a team to identify the right research questions that we should be asking to get down on paper, hypothesis to test in our interviews and make sure that we were going after and trying to solve the right problems. And in addition to the design of the methodology that we did there with this Council, we were also able to utilize individuals from the revenue or to help us identify the right accounts target. So that was making sure we're focusing on accounts, opportunities that recently closed, that were in the right deal size, right company size, geography and make sure that they had had a sufficiently long sales cycle.
And with that, we're able to put together a list of accounts, most meaningful accounts with target for interviews. And in those sales leaders, CSM leaders also helped us make sure we should not reach out to any certain sensitive accounts. Then on the other hand, they also knew top of mind, that certain accounts would be really good to get feedback from because of unique experiences that have been had or where we had lost. Where one to a certain opportunity connecting with the sales and CSM leaders to put together that right distribution list was critical to making sure we had good interviews. So after having an initial kickoff call with the Stakeholder Council, we had a number of follow up communication lines set in Slack and Google docs, made it pretty easy to bring everyone together. Without a ton of meetings, we were able to get the collaboration needed, have a good discussion guide built out and make sure we targeted the right accounts.
And [inaudible] that we have the confidence going forward, make sure the pilot would be successful. After we were set up and had the right list, right discussion guide with the right research questions, we hit the launch button. And we went ahead. Clozd carried us there. We conducted the interviews, developed the report and with that Pluralsight, my team were then able to combine multiple data sources to tell a complete story about what was going on. So that included the salesforce.com loss codes included primary market research we'd already done. But then in addition to that, we then had context and the commentary from the Clozd interviews to really tell a cohesive and a meaningful data-driven narrative about what was going on.
And in addition to that, the Win/Loss Council we've put together, it created a very effective efficient platform for distributing insights gathered across the organization. Finally, we went out, we did the interviews, got the data back and what we're able to do with this process, with this Stakeholder Council that allowed us to begin to deliver a consistent insights each and every quarter. We were able to identify the extent to which we're meeting jobs to be done for the customer. We identified a sales approach that needed to be used to secure renewals, key accounts. And having this up and running, it also positioned us uniquely to respond to the COVID crisis. As COVID hit, we were able to quickly understand the effects of working from home on our customers, on the learners who use our platform and understand how that affected usage of our software.
It allowed us to understand budgeting that was taking place, whether you know, budgets were being canceled, whether they're being frozen, expectations for future spend, things like that. And it helped us identify opportunities to position ourselves against competitors moving forward. One of the big first wins that we had with our win/loss interviews was with something that we call FreeApril. And when the COVID crisis hit, we were looking at trying to understand the right strategy for Pluralsight as a company. And we decided to go with a campaign called FreeApril, where we opened up our entire platform for free, the entire month of April. We weren't quite sure whether we should go ahead with this. We knew it was a gamble. We have never done something like this before, giving away our entire service free for a month.
But with the context we had gained from win/loss interviews, from understanding what customers were going to be doing and understanding more precisely, I seen packaging needs, we had the confidence needed, hit the green light button on this little mark program. The result of free April, we gained over a million and a half new users. Cost per lead was significantly reduced. So there's no way we could have done that with the regular spend. And then also our B2C2B version efforts were greatly enabled. So this was an incredibly successful effort for Pluralsight. There's no way we could have had the confidence and the data needed to move forward with this type of a campaign. Had it not been for the win/loss analysis and the insights we were getting.
In addition to that, we've extended beyond that initial focus on one segment, one geography, one deal size, and we're moving into a broader range of interviews and that's helping us move forward to get more insights to meet the needs of the sales team throughout the whole company. Riley, I'm going to turn the time back over to you, but thanks for listening, thanks for tuning in. It's been great to present and [inaudible] all of that we'll be doing in the future with WinLoss.
Riley Martin:
Yeah. Thanks so much Eric, for your presentation and sharing about your experiences of win/loss analysis. I especially love to hear how you're able to create this Council to incorporate your key stakeholders and be able to combine these multiple data sources to strategize how to approach COVID-19 and how your business can not only stay afloat, but actually grow during this time. So I think that was really insightful. So thank you so much for sharing Eric and thanks so much everyone for attending the session of WinLossWeek. I hope you have a great day.
