What Should You A/B Test Next on Your Ecommerce Site?
Test next whatever your own data shows is costing the most: a page where many shoppers arrive but few move on, a step that behaves differently by device, or an element they ignore. Rank candidates by shoppers affected, evidence strength, and effort. Clear errors and slow pages first, because they can distort any test.
Most teams do not lack test ideas. They lack a reliable way to choose among them, so the loudest opinion or the latest competitor screenshot wins. A better method starts from evidence: where your own shoppers hesitate, give up, or never look. This guide gives a five-step way to turn that evidence into a ranked list of tests.
TL;DR
- Start from behavior data, not opinions or competitor screenshots.
- Score each idea on reach, evidence, and effort.
- Clear errors and slow pages before testing, since they can decide a result on their own.
- Test one change with one success metric and run it for at least a full weekly cycle.
How do you find test ideas in your own data?
Look for gaps between attention and action. Pages with heavy traffic and a weak next step, elements shoppers tap that do nothing, sections that few shoppers scroll to, filters that are ignored, and journeys that split by device all point to something worth testing. Each of these is an observation about behavior, which is a much stronger starting point than a guess about taste.
How do you rank the ideas?
Score every candidate from 1 to 5 on three questions, then multiply the first two and divide by the third. A higher result means test it sooner.
| Question | What a high score means | Where the answer comes from |
|---|---|---|
| Reach: how many shoppers does this touch? | A high-traffic page or a step most shoppers pass through | Page and journey traffic |
| Evidence: how clear is the problem? | Several signals agree, such as drop-off, ignored elements, and session behavior | Click, scroll, and session data |
| Effort: how hard is the change to build and check? | A high score means more work, so it lowers the priority | Design and development estimates |
This keeps small, high-reach changes ahead of big redesigns that touch few shoppers, which is usually the right order for stores without huge traffic.
What should you rule out before you test?
Check the page for errors and slow loads first. If a script fails or a page loads slowly for one variation more than the other, the result reflects the fault and not the design. Fixing those problems is often a better first move than a test, and it protects every later test from false results.
How do you set up the test so the answer holds?
Write the hypothesis in one sentence, choose a single success metric before you start, and change one thing at a time. Run the test for at least one full weekly cycle and long enough to collect a reliable sample, since shopper behavior differs between weekdays and weekends. Resist calling a winner after a strong first few days, which is a common source of false wins. Stores with limited traffic should test fewer, bigger changes. For the mechanics, see how to run A/B tests without a lot of developer work.
How do you learn from each result?
Record what you expected, what happened, and what it tells you about your shoppers, whether the test won or lost. Each result narrows the next round of ideas, so the list ranks itself better over time. A losing test that explains why shoppers behave as they do is still progress.
How do Noibu's agents fit in?
This is the loop Noibu's agents run. The CRO Agent finds underperforming product pages, and opportunities like these can become experiments. The A/B Testing Agent builds and runs experiments on your store, measures the lift, and reports back with the winning variant to ship. As the A/B Testing Agent post explains, it writes each test with a hypothesis, a success metric, and variations, serves it after a person approves, and flags errors or slow pages that could skew a result. You can work with the agents in the Noibu console or from Claude through the Noibu plugin for Claude. Noibu's agents work in the Noibu console and in AI tools such as Claude Cowork, Microsoft Copilot, and OpenAI Codex, and every agent follows the same loop: find, build, your approval, ship, measure. Nothing ships until a person says so.
Noibu is AI for ecommerce operations, the always-on optimization system that keeps online stores running at their best. A team of supervised AI agents finds what's broken, slow, or leaking revenue, does the work, and hands results back for human approval. Trusted by retailers like Free People, Guess, Liquid I.V., and Ariat.
Frequently asked questions
How do I decide what to A/B test next?
Start from your own behavior data, list the gaps between attention and action, and score each idea on reach, evidence, and effort. Test the highest-scoring idea that has no errors or slow loads on the page.
How many things should I change in one test?
One. A single change with a single success metric gives an answer you can trust. Testing several changes at once makes it unclear which one moved the result.
How long should an ecommerce A/B test run?
At least one full weekly cycle, and long enough to collect a reliable sample. Calling a test after a strong first few days is a common source of false wins.
Why do errors matter for A/B tests?
If a script error or slow load affects one variation more than the other, the result reflects the fault instead of the design. Check the page for errors before and during a test.
What if my store has low traffic?
Test fewer, bigger changes and run them longer. For very low traffic, shipping a lower-risk change and measuring before and after can be more practical than a split test.
Related topics
- Noibu's A/B Testing Agent
- How to Run A/B Tests Without a Lot of Developer Work
- Noibu's CRO Agent
- Product Page Optimization: How to Find What Stops Shoppers Adding to Cart
See what the agents find on your store: run a free website audit →
