How Much Traffic Do You Need for an A/B Test?
An A/B test needs enough visitors per variant to detect the smallest lift you care about. At a 2% conversion rate, spotting a 10% relative lift takes about 80,700 visitors per variant at 95% confidence and 80% power, and a 20% lift takes about 21,100. Lower conversion rates and smaller lifts need far more traffic.
Last updated November 17, 2026
The question usually comes up the moment a team considers testing: is there enough traffic on this store for an A/B test to say anything? The honest answer comes from a sample size calculation, not from a tool or a rule of thumb about monthly sessions. This playbook covers the inputs behind that calculation, the standard formula, a worked table of visitors per variant for common store conversion rates, and what to do when the number is out of reach.
Key takeaways
- Sample size depends on four inputs: baseline conversion rate, the smallest lift you want to detect, significance level and statistical power.
- At a 2% conversion rate, detecting a 10% relative lift takes about 80,700 visitors per variant; a 20% lift takes about 21,100.
- Halving the lift you want to detect roughly quadruples the traffic you need.
- Size the test on one primary metric before launch, and treat guardrail metrics as checks, not as extra targets to power.
- If the number is out of reach, test a bigger change, a busier page or a higher-rate metric, rather than stopping early.
Part of: What Are the Best A/B Testing Tools for Ecommerce Brands?
What decides how much traffic an A/B test needs?
Four inputs set the sample size of an ecommerce A/B test. The baseline conversion rate is the current rate of the metric being tested on the page in question, such as purchase conversion on a product page. The minimum detectable effect (MDE) is the smallest change the test is built to catch. The significance level, usually 5%, caps how often the test declares a winner when there is no real difference. Statistical power, usually 80%, is how often the test catches a real effect of the planned size.
The minimum detectable effect drives the result more than anything else. Sample size grows with the inverse square of the effect, so halving the lift you want to detect roughly quadruples the visitors you need. A lower baseline conversion rate also pushes the number up, because conversions are rarer and each variant needs enough of them to separate a real difference from noise.
The MDE can be stated two ways. An absolute MDE is a change in percentage points, such as 2.0% to 2.5%. A relative MDE is a change as a share of the baseline, such as a 10% relative lift, which moves 2.0% to 2.2%. Most store teams think in relative lift, and the tables in this post use relative lift. Mixing up the two is a common reason calculator results look wildly different.
What is the sample size formula for an A/B test?
A conversion-rate A/B test compares two proportions, so it uses the standard two-proportion sample size formula. Penn State's STAT 509 course notes on comparative studies give the normal-approximation version. With an equal split between control and variant, the visitors needed per variant are:
n per variant = 2 × (z1−α/2 + z1−β)² × p̄(1 − p̄) ÷ (p2 − p1)²
In that formula, p1 is the baseline conversion rate, p2 is the conversion rate you want to be able to detect, and p̄ is their average. For a two-sided test at 95% confidence, z1−α/2 is 1.96, and for 80% power z1−β is 0.84, so the squared term is about 7.85. Evan Miller's essay How Not To Run an A/B Test gives a quick approximation of the same calculation, n = 16σ²/δ², where σ² = p(1 − p) and δ is the minimum effect to detect.
A worked example shows the scale. A store converts at 2% and wants to detect a 10% relative lift, from 2.0% to 2.2%. The average rate is 2.1%, so p̄(1 − p̄) is about 0.0206, and the difference squared is 0.000004. That gives 2 × 7.85 × 0.0206 ÷ 0.000004, or about 80,700 visitors per variant, and about 161,400 visitors across a two-variant test. Evan Miller's sample size calculator runs a similar calculation if you prefer not to do the arithmetic by hand.
How to size an A/B test before launch
- 1
Pick one primary metric
Choose the conversion the change should move, such as purchase or add-to-cart, and note its current rate on the tested page.
- 2
Set the smallest lift worth acting on
Decide the minimum detectable effect as a relative lift, based on what would justify shipping the change.
- 3
Fix confidence and power
Use 95% confidence (two-sided) and 80% power unless you have a reason to change them.
- 4
Calculate visitors per variant
Run the two-proportion formula or a calculator and multiply by the number of variants for total traffic.
- 5
Compare with real traffic
Divide by weekly sessions to the page. If the number is out of reach, change the test, not the math.
How many visitors per variant do typical store conversion rates need?
The table below applies the two-proportion formula to a range of baseline conversion rates that cover most online stores' purchase conversion, plus a 10% baseline that fits a higher-rate metric such as add-to-cart. The values were computed in Python using the formula above, rounded up to the next whole visitor.
Assumptions: two variants (control and one challenger), a 50/50 traffic split, a two-sided test at a 5% significance level (95% confidence), 80% power, and a minimum detectable effect stated as a relative lift on the baseline. Counts are visitors per variant; double them for the total traffic the test needs.
| Baseline conversion rate | 5% relative lift | 10% relative lift | 20% relative lift | 30% relative lift | 50% relative lift |
|---|---|---|---|---|---|
| 1% | 637,012 | 163,096 | 42,694 | 19,828 | 7,751 |
| 2% | 315,208 | 80,683 | 21,110 | 9,799 | 3,827 |
| 3% | 207,940 | 53,212 | 13,915 | 6,456 | 2,519 |
| 5% | 122,125 | 31,235 | 8,159 | 3,781 | 1,472 |
| 10% (for example, add-to-cart rate) | 57,764 | 14,752 | 3,843 | 1,776 | 687 |
Three patterns stand out. A 1% conversion rate needs roughly twice the traffic of a 2% rate for the same relative lift. Going from a 20% lift to a 10% lift multiplies the requirement by almost four. And a higher-rate metric like add-to-cart needs far fewer visitors than purchase conversion, which is why many teams measure the step closest to the change.
What lift can your traffic actually detect?
Turning the calculation around is often more useful: start from the traffic a page gets and ask what size of lift a test could detect. The table below uses the same assumptions and a four-week test window, with traffic split evenly between two variants. How long a test should run depends on more than traffic, including full weekly cycles and sale periods, which is covered in how long an ecommerce A/B test should run.
| Weekly sessions to the page (both variants) | Visitors per variant after four weeks | At a 2% baseline | At a 3% baseline |
|---|---|---|---|
| 1,000 | 2,000 | about 72% | about 57% |
| 5,000 | 10,000 | about 30% | about 24% |
| 10,000 | 20,000 | about 21% | about 17% |
| 25,000 | 50,000 | about 13% | about 10% |
| 50,000 | 100,000 | about 9% | about 7% |
| 100,000 | 200,000 | about 6% | about 5% |
A product page that gets 10,000 sessions a week and converts at 2% can reliably detect a lift of about 21% in four weeks. Many single changes, such as new button copy or a reordered page section, are unlikely to move conversion that much. That page can still test, but it needs bold changes or a higher-rate metric to get a clear answer.
Is there enough traffic on my store for A/B testing?
Teams often ask this before adopting any testing tool, sometimes phrased as whether there is a minimum number of sessions. No tool changes the math. The sample size is set by the store's own conversion rate and the size of lift it is chasing, and that holds for every A/B testing platform on the market.
Take a store page with about 200 visits a day, or 1,400 a week, converting at 2%. Detecting a 20% relative lift needs about 21,100 visitors per variant, or about 42,200 in total. At 1,400 visits a week, that is roughly 30 weeks of traffic for one test, long enough for seasonality and promotions to cloud the result. The same page could detect only very large lifts within a month.
When the numbers do not work, there are honest options. Test a bolder change that could plausibly move conversion by 30% or more. Move the test to the busiest pages, such as the home page, top collection pages or the cart. Measure a higher-rate step close to the change, such as add-to-cart instead of purchase. Or skip the test and remove clear friction first, the approach laid out in what a low-traffic store should fix before running A/B tests. Stopping a test early because it looks good is not one of the options: it raises the chance of shipping a false winner.
Which metric should the sample size be based on?
Size the test on one primary metric, chosen before launch: the conversion the change is meant to move. A change to a product page gallery might target add-to-cart rate, while a change to checkout messaging might target checkout completion. Using the baseline rate for that metric on that page, not the sitewide conversion rate, keeps the calculation honest.
Guardrail metrics, such as revenue per visitor, average order value or error rates on the tested page, are watched to make sure the winning variant does not quietly harm something else. They are not usually powered on their own; trying to size a test to detect small changes in every metric multiplies the traffic required. Revenue-based metrics in particular vary much more from visitor to visitor than a yes-or-no conversion, so they need larger samples to detect the same relative lift.
What else changes the sample size?
Adding variants adds traffic. Every variant needs its own full sample, and comparing several variants against one control usually calls for a stricter significance level to keep false winners in check. With a Bonferroni correction, a control and two challengers at a 2% baseline and a 10% lift need about 97,700 visitors each, or about 293,100 in total, compared with about 161,400 for a simple A/B test. The tradeoffs between running more variants and running a clean two-way test are covered in multivariate testing vs A/B testing.
An uneven split, such as sending 90% of traffic to the control to limit risk, also raises the total traffic needed, because the smaller group sets the pace. Raising power to 90% or confidence to 99% raises the requirement too. Each choice is reasonable in the right situation, but each one should be made before launch and written into the test plan alongside the sample size.
How does Noibu's A/B Testing agent handle the test from idea to result?
Noibu's A/B Testing agent builds and runs experiments on your store, measures the lift and reports the winning variant to ship. It runs the same loop as every Noibu agent. Find: the agents watch revenue and experience signals on the store, and Noibu's CRO agent proposes tests from heatmap, click and scroll context on poorly converting pages. Build: the A/B Testing agent writes the experiment and its variations. Your approval: a person reviews the test, and nothing goes live without that approval, which is the point where a team can check the proposed test against the page's traffic and the visitors per variant it needs. Ship: the test goes live on the store. Measure: the agent measures the lift and reports the result. Compound: every result informs what gets tested next. More detail is in Noibu's A/B Testing agent and what to A/B test next on an ecommerce site.
The A/B Testing agent is one of six supervised agents, alongside Bug Resolution, Performance, CRO, ADA and ROAS. Teams work with them in the Noibu console and through Noibu MCP in AI tools such as Claude, so a proposed test can be reviewed and approved where the team already works.
The short version
How much traffic an A/B test needs comes down to the baseline conversion rate, the smallest lift worth detecting, 95% confidence and 80% power. At a 2% conversion rate that means about 80,700 visitors per variant for a 10% lift and about 21,100 for a 20% lift. Calculate it before launch, compare it with real weekly traffic, and change the test rather than the math when the number is out of reach. To see which pages on your store have the traffic and the problems worth testing, start with Noibu's free website audit.
Noibu is AI for ecommerce operations, the always-on optimization system that keeps online stores running at their best. A team of supervised AI agents finds what's broken, slow, or leaking revenue, does the work, and hands results back for human approval. Trusted by retailers like Free People, Guess, Liquid I.V., and Ariat.
Frequently asked questions
How many visitors do I need for an A/B test?
It depends on your baseline conversion rate and the smallest lift you want to detect. At a 2% conversion rate, with 95% confidence and 80% power, a test needs about 80,700 visitors per variant to detect a 10% relative lift and about 21,100 per variant to detect a 20% lift.
What is a minimum detectable effect?
The minimum detectable effect is the smallest change in conversion rate a test is designed to catch reliably. It is usually stated as a relative lift, such as 10% on a 2% baseline, which means detecting a move from 2.0% to 2.2%. Smaller effects need much more traffic.
Why use 95% confidence and 80% power?
They are the common defaults. A 5% significance level limits false winners when there is no real difference, and 80% power means the test catches a real effect of the planned size four times out of five. Raising either one increases the sample size.
Is there a minimum amount of traffic for A/B testing?
There is no single cutoff, because the answer depends on conversion rate and the lift you are chasing. As a rough guide, a page that cannot reach the visitors per variant for a 20% to 30% lift within about a month is better served by removing friction and making bold changes than by running small tests.
Does testing more variants need more traffic?
Yes. Every variant needs its own full sample, and comparing several variants against one control usually calls for a stricter significance level. At a 2% baseline and a 10% lift, a control and two variants need about 97,700 visitors each with a Bonferroni correction, compared with about 80,700 each for a simple A/B test.
How does Noibu's A/B Testing agent work with sample size?
Noibu's A/B Testing agent builds and runs experiments on your store, measures the lift and reports the winning variant to ship. Nothing goes live without human approval, so a team can check each proposed test against its traffic and the visitors it needs before approving it.
