Most A/B tests get called too early, on too little data — and the "winner" was just noise. Enter the visitors and conversions for each variant to see the real conversion rates, the uplift, and how confident you can actually be.
Enter totals for each variant.
The share of each variant's visitors that converted.
How much better (or worse) B converts than A, in relative terms.
— A p-value under 0.05 means: if the two variants truly performed the same, a gap this large would appear less than 5% of the time. It is not the probability that B is better.
The plausible range for the true difference in conversion rate, in percentage points. If this range includes zero, you can't yet rule out "no real difference".
Significance isn't the whole story. Even below p = 0.05, run the test for full business cycles (weekdays and weekends behave differently), don't stop the moment it crosses the line, and make sure both variants got enough conversions — a handful can look significant and still be a fluke. A low p-value says the gap is unlikely to be a fluke; it can't tell you it's worth shipping.
Two-proportion z-test using the normal approximation. The test statistic uses the pooled conversion rate for its standard error; the confidence interval uses the unpooled (Wald) standard error of the difference. The two-sided p-value is 2 × (1 − Φ(|z|)), where Φ is the standard normal CDF. Below roughly 30 conversions per variant the normal approximation is unreliable and a warning is shown. This tool does not correct for repeated "peeking" or for testing many variants at once — decide your test length in advance, and treat a very uneven traffic split as a possible sample-ratio problem.
If your conversion events fire inconsistently between variants, the whole test is compromised before the maths begins. A free foundation audit verifies your tracking is measuring the same thing, the same way, everywhere — so your experiments are trustworthy.
Request a free audit