Noibu: Activating the Data & Building an AI-Native Ecommerce Platform

[00:00:00] Filip Slatinac: In the past, we collect the data. We display the data to you. And what we're building, and what most of it is already built, is that you can also do with the data. Not only do you get the error, but we can also solve the error. Not only do you have a performance degradation, but now you can also solve, automatically, the performance degradation.

[00:00:17] Kailin Noivo: Welcome to another episode of Noibu's AI podcast. This one's a little different because it's with a good friend of mine, my business partner, one of my business partners, the legendary Fil Slatinac. Welcome, Filip.

[00:00:32] Filip Slatinac: Thank you. Second time on the pod, finally, again.

[00:00:36] Kailin Noivo: Look, we just wanted to have a conversation. You're gonna be a very candid convo. Fil told me not to use my radio voice. So, really, what we're gonna talk about is what we've been obsessing over for the last 6 months. Do you guys know who Fil is? If you don't, just Google him. Known him for almost a decade. A really smart guy. He's been, obviously, leading a lot of different initiatives at Noibu, and he's our main brain. I just do podcasts. So, I'm really excited to chat about what we've been working on for the last 6 months in all seriousness. So, Fil, why don't you chew it up?

[00:01:08] Filip Slatinac: I think we've been working on, first and foremost, like, thanks for the amazing introduction. No need for all the pleasantries. What we've been working on for the last 6 months really is something that we've internally called activating our data. And for the people that may be listening to this, Noibu collects so much data. We understand which errors are happening, how much these errors are impacting people's websites, can they add to cart. And as a result, we know how much these errors are costing people. But we also collect all user behavior data, so what people are clicking and how much they're progressing through the funnel. We're also collecting performance data. Is your site slow? Is your cart page slow? Is your checkout slow? And for the longest while, our customers were telling us, this is amazing data, but we're under-resourced. We don't have time to go through all of this data and actually make the changes to fix them. And in some cases, the changes are really complicated. And so customers are telling us, this is great data. I don't have the time or the money to address any of this data. And when we say activate this data, internally, this is what we mean. We mean turning this data into something actionable. And what we've been doing for 6 months is we've been structuring our data in a way so that we can not only capture the data, but also act autonomously on the data. So, the contrast would be: in the past, we collect the data, and we display the data to you. And what we're building, and what most of it is already built is you can also do with the data. Not only do you get the error, but we can also solve the error. Not only do you have a performance degradation, but now you can also solve, automatically, the performance degradation. You have an idea about an A/B test. Well, we can also help you implement that a b test automatically. So, for the last 6 months, we've been really focused on, again, activating our data.

[00:03:01] Kailin Noivo: I love that, and I think this hits home. I do a lot of customer-facing calls, and there was a lot of tools that were procured during the COVID era and a lot of fragment teams using different tools. And first and foremost, we've been on a journey to help consolidate those data sources in the form of tooling, but I think what you're pulling out is how you use the data has dramatically changed a lot in the last 6 months. And we're gonna get into why we think we're in the best position and why we already have the best product, and we're just at the beginning. We're at the start line here, not the finish line.

[00:03:41] Filip Slatinac: Totally.

[00:03:42] Kailin Noivo: So, answer the question directly. Like, why did you advocate so hard internally to change the direction of how our data would be used? More directly, instead of humans using the data as the majority way of gaining value, AI would use the data as the majority way of gaining value.

[00:04:05] Filip Slatinac: I think a lot of people were pushing for this. It's not just me. I think a lot of our engineers, designers, and product people. The ultimate vision of Noibu was always to have your data work for you. So, it wasn't just me. It was really the work of the entire team. But the reason is actually quite simple. All of our customers were saying it. And sometimes they were saying it in a friendly way because they were getting a whole lot of value from the data itself, and they were just hypothesizing with us. Imagine a world where Noibu not only showed you the data, but also worked on the, like, fixed the data. And then the unhappy customers were kinda saying the same thing. They were saying, "I don't have enough time to go through all of this data, or I don't have the resources." And resources could be: I don't have the money to pay an agency, or I don't have the money to hire someone to go through this data. And if we looked at what the happy customers were saying, kind of what was on their wish list, and what the unhappy customers were saying, they were saying the same thing. We need a system that collects all of this data, that aggregates all of these tools that we're already spending so much money on, but not only that, we need these tools or this one tool, if you're paying for Noibu, to do the work for me. And it goes back to one of our founding principles. It's listening to the customer. The customers were going in this direction, some of them faster than others because they're AI-forward, but everyone was saying it. Like, you remember this. Even early on, people were saying, "Like, there are too many errors to fix." "There's too much work on." "I don't have enough time to go through everything", even when we were just error monitoring.

[00:05:44] Kailin Noivo: Yeah. And I think, as we added more modules, to your point, we've added more signal. And, like, the challenge with signals is you'll still need someone to actually act on the signal, right? The first problem we solved was signal-to-noise. We did it really well with bugs, then we did it with performance, then we did it with heat mapping data, doing it with ADA now; we're also doing it with A/B testing and a few different other categories, but there's been a few market forces. Retail's been quasi-flat for a few years. There's been a pretty big unwind on the P&L for digital budgets because, effectively, digital got shot for 10 years in demand and then kind of went down as a percentage of overall retail. So, at the same time, AI has been evolving from something that's been around for 10, 15 years to something that's actually being used, and you could tell by the Anthropics of the world's run rate. So, I know you won't take a lot of credit for this, but the bulk of the credit comes ultimately for you at the end of the day and your teams directly for kind of seeing around the corner on this. Now, what did it take organizationally to actually gain alignment? And what were some of the challenges that you had to face to be able to kind of ram something like this through the sausage maker?

[00:07:04] Filip Slatinac: Yeah. Again, it's really kudos to the teams, the various teams involved in this. It's really just not me or us, really. It's not just us three. It's really the teams that are really pushing this forward. It took a lot. The first thing that we need to be convinced of is that AI could be accurate, that it could digest an immense amount of data and come up with something accurate that is grounded in truth. And earlier, I suppose earlier last year, the models weren't there. The models were still hallucinating a whole lot. They were not great at tool calling. They were honestly just making stuff up. And as the models got better, the accuracy problem kinda solved itself. The models were able to reference the source material. They were able to stop hallucination or reduce hallucination. And that gave us confidence that it's a matter of time before the models, the LLMs, the AI providers, are good enough for the accuracy problem. Once we had conviction that we're getting there, then it was step two. Then it was: can we shape our data? Now, we know the world is going towards a place where we have automated systems working on the data. And so we needed to modify our own system so that we can power this new world. And it took a whole lot of work for us to structure our data in a way that it can be queryable by AI, and there's a whole amount of work necessary for us to do that. Then it's also distracting because we were on a mission to amalgamate as many tools as possible into Noibu to help our customers. And so everyone had a specific road map in mind. All of the product managers, the designers, and the engineers had a specific road map in mind. Our data was shaped so that AI can interact with it, and we knew that the models were good enough in terms of accuracy. It took a big push from everyone on the team to align on a brand new road map, a road map where AI is first. And Noibu really moves into being an AI-native product company. That took a long time as well. And then, as we kind of pass these two initial hurdles, we also used to think that Claude and Codex or ChatGPT are all-powerful. That everyone is going to be doing everything out of these tools, and they are powerful. They're so powerful. But I don't think they're all-powerful. They have flaws. It is difficult to orchestrate a difficult workflow, a workflow that requires a human in the loop, within just a chat box. And even though they're very powerful, there are systems to be built against protected workflows. Like, a great example is if you're solving a bug. If you're solving a simple bug, you can leverage Claude. No problem. But if you're solving a complicated bug where you need a preview URL, and the change needs to go to staging, and it's touching complicated code in your checkout or your cart, you don't wanna just blind-approve the PR. You don't wanna just let it go. And there is no way of controlling these things with just Claude. And that thing, it needs to be built, and that's exactly what we're working on right now. And we hope to have something delivered in the short future.

[00:10:12] Midroll: If you're listening to The Ecommerce Toolbox, you're entitled to a podcast-exclusive website audit. Go to noibu.com/podcast-audit for a free scan that uncovers the hidden friction blocking your conversions and shows you where you're leaking revenue.

[00:10:26] Kailin Noivo: No. It's interesting. And, like, this is all moving pretty quickly. Like, one thing that I can say from my front is that I just keep hearing from customers or prospects that everyone's being asked to consolidate tools, drop spend, in some cases, decrease their reliance on external agencies. And at the end of the day, everybody's solution is trying to implement AI, and people don't really know how. D2C businesses that are owned by private equity or other investors, like, they're just straight up like, implement AI, implement AI, is one of the most common things that people are being told to do. They don't know how. And what we're really seeing in the market is what you're flagging right now. People want to implement AI. They don't know how. They're afraid of, like, the Terminator scenario on their sites. They're ultimately accountable for P&L. So, they're trying to find a surefire way of integrating it. And the promise is there, but it's not an out-of-the-box solution that you just plug something in. You don't plug Shopify into Claude. Next thing you know, everything's automatic. We chatted a lot about kinda how we got here, and I alluded a bit to where we're going. But I'd be curious from your perspective. What are we really, really, really good at automating today end-to-end, and how do you envision the next, I was gonna say, 6 to 12 months, but really, AI is moving so fast that I'm gonna say like 0 to like 3 months, because who actually knows? I have no idea. I used to think I knew what 6 to 12 months look like. I have no clue now. I have a pretty good idea, I think, but I'm curious to hear your opinion on the next 0 to 3. So, two questions. What are we good at end-to-end today? What are we the best in the world at end-to-end today? And then where are we taking it next?

[00:12:15] Filip Slatinac: That's a good question. I think what we're really good at today is true to our core, which is we're really good at bug identification and bug resolution. So, basically, the end-to-end workflows related to bugs that happen on ecommerce stores. We're also really good at performance, performance problem identification and resolution. And then the third thing that we're also very good because we're deployed on customers' websites, and we collect 100% of the traffic on ecommerce websites; we're also good at identifying ADA problems and solutions against those ADA problems, which is something that is gaining in popularity. Like, ADA compliance wasn't that big of a deal two years ago. It's becoming a huge problem right now. People are getting sued left, right, and center for ADA noncompliance, and our software can actually pick up on those problems and automate the resolution of ADA problems. Have you noticed that ADA compliance is something that's becoming more popular?

[00:13:19] Kailin Noivo: Yeah. I don't know if it's becoming more popular or if there's just law firms that are funding, like, I think there's some sort of business situation going where a lot of brands are coming. In terms of where it's becoming more popular is I searched our calls, and the percentage of calls in which ADA compliance is mentioned is through the roof. So, like, the lawsuits are gaining popularity, which is something that's interesting. We don't have to get into the why or the who's behind it, but yes, absolutely. From a high level, brands are being sued for ADA compliance issues at a much higher clip than we've ever seen before.

[00:13:54] Filip Slatinac: Right. And then to get to the second part of your question, like, where we're going in the next 0 to 3 months, we really wanna take it a step further. Of course, in ecommerce, you're worried about your bugs, you're worried about your performance, you're worried about ADA compliance, but those are the things that you just, honestly, in a lot of cases, want to be doing but just don't have enough time to be doing. So, what are you doing in ecommerce that we can help with? And the simplest thing to do is you're optimizing revenue, you're improving conversion rate, and you're improving your ROAS. And where we're going is exactly in this direction. We're going to be building tools and automations and end-to-end workflow automations to help people increase or decrease their ROAS, increase the conversion rate, run automated A/B tests, suggest and implement A/B tests automatically, and so not only are we able to help with the work that people know they have to do but just don't have the time to do, but we will also help our customers do the work they already are doing, and they need to do.

[00:14:58] Kailin Noivo: Yeah. It's pretty insane, to be honest. Like, the fact that if we look at it from a roadmap standpoint, like, the fact that we've completely end-to-end done those three things that you mentioned, like automated today, bugs, performance, ADA. We do it. It works. And then, to your point, we've had digital experience analytics for a while, but the ability to go in and find hypotheses and then test them is very manual. Time to value is quite...

[00:15:23] Filip Slatinac: Very difficult.

[00:15:24] Kailin Noivo: Very difficult. You need to have a theory. Same with optimizing your ad campaign. So, really, at the end of the day, like, being able to launch those agents is pretty crazy. Like, if you zoom out by the end of the year, we're talking here that we're going to have end-to-end agents that will send you a message in the morning saying, "Hey. I noticed this trend in this friction, and I want you to launch an A/B test. Here are some ideas. Which one do you think is a good idea?" Automates the creation and the reporting backwards; like, it's so clearly where our customers are pulling us. Yeah. It's been phenomenal, and it's honestly the most exciting time in the last decade for me, at least in the context of Noibu.

[00:16:08] Filip Slatinac: A/B testing is one of those things that is incredibly fascinating to me. And, like, for example, if you take a look at the traditional DXA tools, they're really good at picking up an insight. People aren't scrolling below the fold, and your add to cart is below the fold. And so what the traditional DXA tooling will tell you is, like, oh, just put the add to cart above the fold. That's not necessarily true. Right? Like, there could be so many other factors as to why your people are just not interested in clicking add to cart on this product. And so when it comes specifically to DXA insights or user behavior insights, there is no concrete solution. It's always an idea. It's not like a bug. A bug is a thing. You can solve the thing. For user behavior insights, it always comes down to a hypothesis. I believe that if we were to, on this specific page, put the add to cart above the fold, I believe that it will increase conversion. Let's go test this. Okay. It fails. Let's run another A/B test based on an insight. And so, in order to really lean into the DXA insights and opportunities, you need a way to constantly, with ease, be able to run A/B testing. Otherwise, you can't find the right solution, the correct solution.

[00:17:30] Kailin Noivo: It's true because the answer is different depending on where you work. If you work at Guest Genes or if you work at Samsonite.

[00:17:39] Filip Slatinac: Totally different market.

[00:17:40] Kailin Noivo: Two different answers.

[00:17:41] Filip Slatinac: Of course. Right?

[00:17:42] Kailin Noivo: Which is kinda crazy.

[00:17:44] Filip Slatinac: Different answers, different shoppers, different products. And so there is no one-size-fits-all solution to user behavior insights, and so you need to test. And the reason our customers in the market aren't testing, it's so difficult to create an A/B test, manage your A/B tests, make sure it's not breaking your website, and report on this A/B test. But that's exactly what we're building, and that's exactly what we're making incredibly easy for our customers to do.

[00:18:10] Kailin Noivo: I love it. I love it. And as we look to wrap up, appreciate your time, Fil. Always great catching up.

[00:18:16] Filip Slatinac: Pleasure.

[00:18:17] Outro: The Ecommerce Toolbox: AI and Retail is brought to you by Noibu. To find out more about Noibu and how we unify error monitoring, site performance, and experience analytics to uncover growth opportunities and skyrocket your revenue, visit www.noibu.com. That's n-o-i-b-u.com. And then make sure to search for The Ecommerce Toolbox: AI and Retail on Apple Podcasts, Spotify, or anywhere else podcasts are found, and click subscribe so you don't miss out on any future episodes. On behalf of the team here at Noibu, thanks for listening.