Episode 152

w.kleinberg: Don’t Automate What You Can’t Govern

David Levin
David Levin
Acting Chief Digital Officer

In this episode we talked about:

  • The three things David checks before deciding whether digital can be a growth lever: customer lifetime value, where the margin lives, and who owns it
  • How Bob’s Discount Furniture modeled its repeat-purchase curve, and why the 3-month repeat rate predicted the 12-month number
  • Why private equity’s billions in embedded AI engineers may be solving the wrong bottleneck
  • Why you can buy AI talent in a quarter but not a ready data layer, a willing team, or an exec who can tell a demo from a deployment
  • Why you do not have to finish the data layer before starting with AI: pick a use case where the data is clean enough, ship it, and fund the next one with the win
  • How Bob’s built a self-serve AI agent for delivery scheduling, with a target of under two minutes
  • Why the governing guardrail was “understand,” not “invent,” and how a Labor Day discount question exposed a hallucination
  • How the guardrail still deflected about 35% of inbound questions and took roughly $4 million a year out of operating costs

🎧 Listen now on Apple Podcasts, Spotify, or YouTube

Episode highlights:

00:55 David’s Career Journey: Agencies, SaaS Exits, And Bob’s Discount Furniture

02:50 Three Things To Check Before Digital Is A Growth Lever

07:36 Why Embedding AI Engineers Is The Wrong Bottleneck

12:09 Start With A Use Case Where The Data Is Clean Enough

15:58 Governance: Understand, Not Invent

19:32 Advice For CDOs Implementing AI In 2026

20:56 Why AI Guardrails Matter Even More For SMBs

David’s bottom line: AI talent is not the bottleneck. You can hire AI engineers within a quarter, but you cannot hire your way to a ready data layer, a team that wants the change, or an executive who can tell a demo from a deployment. David Levin, Acting Chief Digital Officer at w.kleinberg, spent seven years at Bob’s Discount Furniture helping take it from about $1 billion to about $2 billion in revenue. He says digital is only a growth lever if the company can answer what a customer is worth over time, knows where its margin actually lives, and has someone who owns the program. His advice on AI: start with a use case where the data is clean enough, ship it, and use the win to fund the plumbing for the next one. At Bob’s, the guardrail was “understand,” not “invent.” The AI could interpret a customer’s question, but it could only respond with approved answers. That still deflected about 35% of inbound questions and removed roughly $4 million a year in operating costs.

FAQ

David Levin is the Acting Chief Digital Officer at w.kleinberg. He spent about 20 years in digital agencies, sold his agency to 360i (later sold to Dentsu), then founded and sold his own SaaS company. He then spent seven years at Bob’s Discount Furniture, helping grow it from about $1 billion to about $2 billion in revenue ahead of its IPO. He now consults for a few companies, including w.kleinberg.
First, whether the company knows what a customer is worth over time, not just the revenue per order, and whether that differs by channel. If nobody can answer, digital is still a cost center that looks good on a dashboard. Second, where the margin actually lives, because some businesses have headroom to invest in acquisition and others do not. Third, who actually owns the program, not who sponsors it. In David’s view, a program without a clear owner who can decide whether it succeeded or should be killed cannot scale.
Bob’s was unclear about how often people bought furniture, so the team modeled the repeat curve. About 11% of new customers repeated within three months, 24% within two years, and about 33% within five years. The 3-month rate turned out to be a reliable leading indicator of the 12-month rate, which was around 16.5%. David says each point of improvement in the 12-month rate was worth about $2.4 million in incremental annual sales, so the team set targets on the 3-month and 6-month repeat rates.
He says you can hire strong AI engineers within a quarter, but you cannot buy a ready data layer, a team that wants the change, or an executive team that can tell a demo from a deployment. Outside engineers can help stitch systems together, but operators have to own those calls. In his experience the first bottleneck is data plumbing, such as three definitions of customer spread across five systems. The second is change management. The third is leadership that has seen a deployment fail and can judge what is working in real time.
No. David says the data layer and AI are both important but not sequential, and that waiting for perfect plumbing means you never start. His advice to operators is to pick a use case where the data is clean enough, ship it, and use the win to fund the plumbing for the next one. At Bob’s that use case was delivery scheduling: everyone agreed what a successful delivery meant, and it ran on a single system.
It was Bob’s governing guardrail. The AI could use a large language model to understand what a customer was asking, but it could not generate its own answer. It could only respond with written, approved answers, and repeat the question back so the customer knew they were heard. Bob’s tested the generative version with 5,000 store associates before launch. When one asked about Labor Day discounts at an everyday low-price retailer, the AI invented an entire promotional calendar, so the team pulled it back.
Even with the AI limited to approved answers, David says it deflected about 35% of all incoming questions. That is where roughly $4 million a year in operating cost savings came from, and the savings continued to scale as the store count grew.
Start with the problem statement and the core KPI you want to move. Then decide where you are comfortable and uncomfortable using AI to move it, and how far you are willing to go. Finally, have smart conversations with internal and external AI experts about how to reach the goal while mitigating risk. He sums it up as a balance between inflecting the KPI and managing risk.
David says a hallucination hurts more at small scale because there are fewer customers to get right. That makes guardrails even more important for SMBs, but he believes they can still execute well by choosing the right guardrails and talking to their partners about them. He also noted that w.kleinberg, a smaller business, is running a pilot with Noibu that starts on the analytics side and is expected to move into UX.
Toolbox listeners get an exclusive free trial of Noibu’s AI agents for CRO, A/B testing, performance, accessibility, ROAS, and bug resolution. The agents start by auditing your store and ranking the biggest opportunities, from slow pages to broken journeys. Then they build each change, wait for your approval, ship it, and measure what it did. The trial runs until you have at least two changes live and measured. Get started at noibu.com/podcast-audit.

Want to be featured on The Ecommerce Toolbox?

We feature senior ecommerce leaders with real things to say. If you’ve led something worth talking about, we want to hear from you.

Apply to be a guest →

Other episodes

See all episodes

Audit your site today

Supervised AI agents find what's broken, slow, or leaking revenue, do the work, and hand it back for your approval.