[00:00:00] David Levin: They keep assuming that AI is going to be that talent challenge, but in the portfolio companies I’ve been at up close, that’s not the bottleneck. The bottleneck is that data plumbing. Outside engineers can try to stitch those systems together, and they can be very helpful. But at the end of the day, the operators have to own those calls.
[00:00:19] Kailin Noivo: Excited for today’s episode. Reconnecting to David. I’ve known David for quite some time. We chat all things AI, especially in this new age and what CEOs are thinking about. So, really excited for today’s episode.
[00:00:30] Kailin Noivo: Welcome to another episode of AI in commerce with Noibu. Today, we have David Levin, who’s currently an acting chief digital officer at w.kleinberg. Welcome, David.
[00:00:44] David Levin: Hey. Good to see you, Kailin.
[00:00:46] Kailin Noivo: Awesome. Alright, David. Kick us off. Tell us about your career journey. You’ve done some mighty impressive things, so start from the beginning.
[00:00:55] David Levin: Yeah. I think the TLDR on me is that I like to build things, and I like to build things that create value for companies. Like, putting, you know, numbers on the scoreboard is the fun part of, I think, why I fell in love with digital in the first place. Typically, when I get brought into a company, it’s when they know that digital should be a bigger part of their strategy and their future, but they just either don’t know or can’t get there. And so, there’s two problems that I’m usually trying to solve, but I’ll give you one for example today. The customer experience. The customer should experience one company, but the company is operating as 5 different organizations. The website team is different and operating differently than the store team. The customer service is fragmented from all of that. And each of these teams have their own data, their own definition of what the customer is, their own metrics, and what a good day looks like, and that can be a real problem. And so, I started my career on the agency side of all this. I spent about 20 years in digital agencies. I sold my digital agency to 360i, which we later sold to Dentsu. I then stayed a few years and then went and founded my own SaaS company, and I sold that. And then I spent 7 years transforming and helping grow Bob’s Discount Furniture from, you know, when I joined, the company was about a billion dollars backed by Bain Capital, and helped grow it to about $2 billion, prior to its IPO that happened recently. And like you said, I’m currently consulting for a few companies, one of which is w.kleinberg, which has been a lot of fun.
[00:02:30] Kailin Noivo: That’s pretty crazy, man. So, 2 IPOs, 5 exits, an MBO. Like, when you’re walking into these businesses for the first time, like, how are you assessing if, like, how did you know, for example, Bob’s, that, like, digital would be a growth lever? Yeah. How are you assessing, like, if you’re able to really drive that much growth through digital?
[00:02:50] David Levin: Yeah. I think the good thing in my career is I’ve been fortunate enough to be on both sides of the equation. I’ve looked at buying companies and been on the buy side, and I’ve been bought. And it’s taught me that there’s three things to look at, and one of them is not the website. Okay? First, can the company and the brand know and tell me what their customer is worth? Not the revenue that they get from the customer, but what a customer is worth over time. That lifetime value is critical, and whether that is different for customers acquired through different channels and through different ways. If nobody can answer that right away, digital isn’t yet a growth lever. They’re not ready. Because what it really is at that point is that it’s just a cost center that looks great on a dashboard. It’s also kind of the quickest diagnosis that I can do when I’m looking at a company or an opportunity because you learn it in the first meeting. If they can’t answer the question, you know exactly where they’re sitting from a maturation standpoint. The second thing I look at is, you know, where does the margin actually live? So, some businesses have real headroom to invest in acquisition, but other businesses don’t. And at Bob’s, the repeat curve really told us everything. We were really unclear about how often people were buying furniture. So, the first thing we did was model that repeat curve. And what we found was that 11% of new customers would repeat within the first 3 months, 24% within the first 2 years, and it went up to about 33% of customers would repeat within 5 years. But the most important number that I think we identified early that really helped us think through a go-to-market strategy was that the 3-month number, that 3-month repeat rate, was a reliable leading indicator for us of what the 12-month number was going to be. So, at the time, I think our 12-month repeat rate was around 16 and a half per cent, give or take a point, a tenth of a point. And we modeled that every single point we could increase was worth about $2.4 million in incremental annual sales. So, we were able to set targets of how can we improve that 3-month and 6-month repeat rate to drive up that 12-month repeat rate and then ultimately that 5-year repeat rate. And the third is who owns it. So, what I looked at, and what I looked at with Bob’s, was why I came in, and we built a customer experience and digital transformation specialization, or center of excellence: it was who actually owns it within the company. Not who sponsors it, but who actually owns it. And for us, for me, that’s the thing that kills most programs: You can talk all day long of we’re going to grow digital, we’re going to invest in this, we’re going to do this, but at the end of the day, if nobody actually owns it and nobody makes a decision of whether the program was successful and should be killed or not, you can’t actually succeed, and you can’t start really scaling digital as a lever.
[00:05:57] Kailin Noivo: Out of curiosity, what brought you from founding and selling companies to joining, like, a pretty established business? Like, a billion-dollar revenue business is pretty established.
[00:06:08] David Levin: It’s a great question. I think, you know, one of my favorite mentors and business partners said to me once, as you get older, you get more risk-averse - fear of starting another business - and so as I get older and want to spend more time with my kids, it’s harder and harder to go start a new business. So, I realized that I had a sense of this crystallized knowledge of what works in the playbooks, and they were just as applicable; maybe the orders of magnitude were different in that certain things matter more, like governance in a larger business, but being able to apply those things was a huge opportunity. And so I just kind of moved up and continued to scale what I had learned into larger and larger businesses with each move.
[00:06:52] Midroll: If you’re listening to The Ecommerce Toolbox, you’re entitled to a podcast exclusive website audit. Go to noibu.com/podcast-audit for a free scan that uncovers the hidden friction blocking your conversions and shows you where you’re leaking revenue.
[00:07:06] Kailin Noivo: Talk to us a bit about, you recently wrote that private equity is spending, like, billions of dollars to embed AI engineers across portfolio companies, and yeah, I mean, cost of capital’s gone up. So, it’s kind of like, increase costs, increase targets, implement AI. That’s what every PE firm is kind of pushing right now. Yeah. Talk to us a bit about why you think that’s potentially the wrong bottleneck.
[00:07:36] David Levin: For me, you can buy talent within a quarter. You can go out and hire great AI engineers. But what you can’t do is make sure that your data layer is ready, that your team is willing, and that you have the right exec empowered to make sure the deployment succeeds and has actually seen one that’s failed. Because if they haven’t seen when it’s failed, then what you have is someone who really can’t judge what’s working and not working in real time and make corrections. Just to give you some background here. So, as you were talking about, Anthropic announced a $1.5 billion venture with Blackstone, Apollo, Goldman, and General Atlantic, and others. OpenAI then launched a $4 billion deployment arm that’s backed by TPG, my old friends at Bain Capital, Brookfield, Advent, they keep assuming that AI is going to be that talent challenge, but in the portfolio companies I’ve been at up close, that’s not the bottleneck. The bottleneck is that data plumbing that I was talking about. Three definitions of customer living across 5 different systems. Outside engineers can try to stitch those systems together, and they can be very helpful. But at the end of the day, the operators have to own those calls. And if they’re not ready yet to own those calls, that’s going to be your first bottleneck. The second one is, and it’s kind of related, is change management. The team has to want to do this change. And so, if the analyst has been running their data models manually for a decade, the new way is a fight, not a deployment. And then the last piece of this is, like I was talking about earlier, it’s an exec team that really can tell the difference between a demo and a deployment. Because I’ve seen so many exec teams walk into a meeting with the CEO, show a curated set of data that’s been cleansed and it’s perfect, and the CEO walks out, and it’s like, “This is going to work.” But when the reality meets the road is at 2 AM, when your analyst is tired, and you find a weird edge case and a whole data model breaks. That’s where it all falls apart. That’s why I think you really need experienced people on the operating team to make those things work.
[00:09:50] Kailin Noivo: Man, this is a really interesting insight, especially when we’re starting to talk in the context of ecom. Because a lot of retail, especially the, retail is retail, right? It’s very correlated to, like, consumer sentiment and GDP, and when there’s inflation in your basket of goods, like, you’re probably going to stretch those jeans out a bit longer before getting an extra pair. So, there’s a lot of, like, interesting macro factors when it comes to commerce. And then you kind of, we’re still seeing some of the unwinding, especially where D2C businesses were being treated like tech companies for a little bit. So, there’s a lot of, at least a lot of our customers and prospects that are either owned by PE, they had investment, and it’s kind of a lot of the same talk tracks. It’s cut your team or cut your OpEx. Drop your OpEx now for the fourth year in a row by 10%, 15%, depending on how far you have to cut. Stop spending so much on ads, grow your LTV, drop your CAC, implement AI, do more with less. So, it’s kind of everyone’s kinda hearing the same thing.
[00:11:01] David Levin: Do more with less all at once everywhere.
[00:11:04] Kailin Noivo: While growing your targets year over year. Yes. And the thing I don’t envy, I mean, you had a SaaS company; the thing I don’t envy about, like, retail is come January 1, you’re at zero. And, like, that’s not the case with SaaS. Right? Like, you maybe lose 10% of your revenue. You know what I mean? Year over year, whatever, 5%. So, you start the year kind of 10% in the hole. But starting the year at zero is a complete different beast, right? So, it’s very interesting. And I guess where I was going kind of with all of this is, when it comes to data plumbing, like, do you think you can build autonomous AI agents, which is kind of the future everyone’s trying to get to? Everyone’s trying to build an AI agent to automatically go through their customer support tickets, to solve their bugs, to make their site faster, to investigate customer friction; like, everyone wants to do this. Like, how important is the data layer, and is that almost such an obvious that it’s a redundant question? But, yeah, I’m curious, especially given your kind of experience, how you’re now consulting with these businesses.
[00:12:09] David Levin: It’s a great question. I’d push back a little bit on the premise, just slightly, in that they’re both important. They’re not necessarily sequential. So, the data layer, I could talk about for hours how critical it is to stitch your data together and have a single view of customer, a single view of product, so that your AI, but also your teams, are all aligned around a single source of truth. It is critical. That said, you don’t have to finish that data layer to start working in AI. And so it’s really about knowing where the bodies are buried when you start. So, otherwise, you’d never start because the data layer is a journey, just like everything is a journey in retail. And so what I would do, and I would recommend to any of the operators listening to your podcast, is pick a use case where the data’s clean enough. Ship that use case and use the win to fund the plumbing for the next one, right, versus waiting to get the plumbing. At Bob’s, the clean-enough thing for us was delivery scheduling. And it was also a need. This was during COVID. Stores closed for a while. We all went into our homes and didn’t leave. And everyone, from a data perspective, agreed generally what a successful delivery was and what it meant. So, there was no real core definitional fight from a data plumbing perspective, and we could work with one system, which was the delivery scheduling system, and build a self-serve AI agent that could actually handle that delivery schedule without you having to call us, like the old legacy model in furniture. And so, that was where we sort of started, and that credibility allowed us to get even more advanced with customer support and other use case handling from there.
[00:14:01] Kailin Noivo: Do you see whether it’s in companies you’ve worked at or in companies you’re consulting with, call it, like, the apprehension around AI depending on how your team is structured? So, for example, like, if you’re super marketing heavy, then maybe you wanna lean in there less. If you’re super engineering-light, you’re okay to lean there more and then kind of vice versa. Like, have you seen that dynamic play out at all?
[00:14:24] David Levin: Yeah. It’s different in different scenarios. I think one thing you need to know when it comes down to casting and how you put a team together from a RACI perspective is if you’re light in engineering, can you go out and buy those engineers? So, in our use cases at Bob’s, the engineers for SAP and our internal delivery scheduling systems were strong, but our AI engineers - we didn’t invest in those. So, that’s where we went out and bought that through a SaaS provider and their professional services team. So, their PS teams could do the implementation, and our teams could focus on what we’re good at, which is just knowing our internal systems. And from the business side, knowing the customer experience, you know, know the target that we were going for as far as how we wanted this to feel and what was the amount of time we wanted you to spend. We wanted really less than two minutes to schedule or reschedule a delivery. That was our target. And so we were able to go out and cast for where we were weak to get that across the line.
[00:15:32] Kailin Noivo: Talk to me a bit more about at Bob’s, you led a generative AI strategy for customer service and workflows. You guys cut millions in costs. Like, talk to me a bit about where you guys were kind of drawing the line in terms of what you wanted the system to do, and what you still wanted heavy human involvement on. I know that that’s something that a lot of people are kind of struggling with right now: how much of the wheel do they hand over?
[00:15:58] David Levin: Yeah. And I think what you’re really getting at with that question is governance, especially in large companies, when you think about handing the keys to the car and saying, “Hey, AI agent, go drive my car.” You’re worried about what it’s going to do from a customer support and service perspective. So, to me, at a system level, at an organization the size of Bob’s, what we decided was that the model could understand what a customer was asking, could use large language models to really comprehend. But what we found was that it could not invent the answers. Because if we allowed it to do that, we were getting into real challenges. I’ll get specific and double-click on this for you. But the governing guardrail for us was “understand”, not “invent.” And so here’s how this worked: when we allowed it to generate its own answer to a customer-facing question, and these were in the early days of AI. It’s getting better and better. But this is also, I think, a great lesson in how you can step your way in and still test significantly, and I’ll talk about the return on investment that we got from this, test how you create change without maybe giving over the keys fully. What we found was that when we tested the generative content with, like, 5,000 store associates, and that’s how we tested it before we put it in market, we had an internal population with real stakes, and we weren’t just testing a QA script. We allowed it to sort of embarrass itself in front of the associates, and we found out for free where it wasn’t working and where it was. What we found was that it would hallucinate answers that didn’t show up in our demo testing. And so, here’s an example. We would ask it the questions that a marketing person would ask, but not the simple questions that you ask when you’re tired and frustrated or having typos, you know, because you’re typing quickly. And so, a salesperson asked it, “What are the discounts available on Labor Day?” Well, if you know about Bob’s Discount Furniture, it’s an everyday low-price retailer. There are no sales. The AI invented an entire answer out of thin air. It literally wrote.
[00:18:16] Kailin Noivo: Like a promotional calendar and everything.
[00:18:18] David Levin: And it invented a full promotional calendar. So, we immediately said, we are not ready to give it that level or manage that level of governance. So, we pulled it back and said, it can understand the question, but it can only suggest and write the written, approved answers directly. And we did that, and it still removed and deflected about 35% of all questions coming in, which is where we got that $4 million a year out of operating costs that you were quoting, which continued to scale as the store count grew.
[00:18:56] Kailin Noivo: There was actually a pretty famous case in Canada with Air Canada where they had to honor it, right? And it was just completely, like, out of thin air. It was in no FAQ, no training. So, no, that’s definitely a thing. And, obviously, with AI getting more and more, like, honestly, just the model’s getting better, hopefully, those things will decrease over time. And, David, as we’re looking to wrap up, like, if you were to give one piece of advice to CDOs in 2026, especially those interested in implementing AI and those that have all the same pressures we were just discussing, like, what would you say?
[00:19:32] David Levin: Look. Like any technology, I would say it starts with the problem statement, and you should always still be looking at what is the core KPI you’re looking to inflect. That’s number one. And then second, where are you comfortable and not comfortable using AI to effect that change? And how far are you willing to go? And then really have smart conversations with internal and external experts in AI about how could you achieve this while mitigating risk. I mean, full stop for me; that’s the key. And I think that once we figured out we could still just use the interpret model, as the example we were just talking about, once we figured out we could go there, at least the customer could talk in a very free-flowing way, in a very human way, and then the bot would respond with very specific text, but could at least repeat the question back that the person asked. We could show the customer we were listening and service them and reduce that risk. I think those are the two things you have to balance: how can you inflect the KPI and mitigate risk.
[00:20:35] Kailin Noivo: Makes sense. And I think a lot of your answers is really also framed in the context of, like, enterprise, whereas where we’re seeing a lot of traction too is a lot of SMBs didn’t have the ability to implement these systems before AI. Didn’t have the headcount, and now there’s, like, ways that they could effectively get the bulk of the outcome without having to hire massive teams around it.
[00:20:56] David Levin: Yeah. And the only difference there is the hallucination at a small scale is more painful because you have less customers you have to get it right with. So, it’s actually more risk if you can believe it at the SMB level to get it right, but you can, and you can execute at scale if you just think about what are the right guardrails and talk to your partner about that. You know, we’re running a pilot with you guys now. It’s more on the analytics side, but it’s going to move into the UX side. And I’m really excited to see what it can do, but this is at w.kleinberg. This is a smaller business. And so what I’m excited about with this is getting it right will have a huge impact for them. Getting it wrong will have greater downside. But I really believe it can do great things, what I’m seeing so far in the first few days of the pilot and what it can do on the analytics side. I can’t wait till we get to the activation side with you guys.
[00:21:47] Kailin Noivo: 100%. And, David, it was nice reconnecting again, and I appreciate you taking the time today.
[00:21:52] David Levin: Thanks for having me on.
[00:21:54] Outro: The Ecommerce Toolbox: AI and Retail is brought to you by Noibu. To find out more about Noibu and how we unify error monitoring, site performance, and experience analytics to uncover growth opportunities and skyrocket your revenue, visit www.noibu.com. That’s n-o-i-b-u.com. And then make sure to search for The Ecommerce Toolbox: AI and Retail on Apple Podcasts, Spotify, or anywhere else podcasts are found, and click subscribe so you don’t miss out on any future episodes. On behalf of the team here at Noibu, thanks for listening.