The standard test
A pooled two-proportion z-test, which is what most A/B testing platforms report under the hood.
Tools / Statistical Significance Calculator
Statistical Significance Calculator
Enter visitors and conversions for both variants. Get the p-value, the relative lift, and a straight answer on whether the difference is significant.
z = (p̂B − p̂A) ÷ √( p̄(1 − p̄)(1/nA + 1/nB) )
Enter values to see the result
- 1
Enter variant A
Visitors who saw the control and how many of them converted.
- 2
Enter variant B
Same two numbers for the challenger. Conversions cannot exceed visitors.
- 3
Read the verdict
You get both conversion rates, the relative lift, the z-statistic, the p-value, and whether it clears 0.05.
Why test significance
Variant B being ahead is not the same as variant B being better. This separates a real difference from ordinary noise.
Lift alongside significance
A significant 0.2% lift and a non-significant 40% lift call for very different decisions. Both numbers are shown.
Honest about the threshold
The verdict uses the conventional 0.05 cutoff and says so, rather than hiding the p-value behind a badge.
What to test
Landing page A/B tests
Compare signup or purchase rates between two page versions.
Email subject lines
Check whether an open or click rate difference is more than noise.
Survey question wording
Compare the yes rate between two phrasings of the same question.
Pooled two-proportion z-test
z = (p̂_B − p̂_A) ÷ √( p̄(1 − p̄) × (1/n_A + 1/n_B) ) where p̄ = (x_A + x_B) ÷ (n_A + n_B)
p̄ is the pooled conversion rate across both variants, used because the null hypothesis is that the two rates are equal. The p-value is the two-tailed normal probability for that z.
Before you call a winner
- Decide the sample size before the test starts, then leave it alone until you reach it.
- Checking daily and stopping the moment it turns green is how false positives get shipped.
- Run for whole weeks so weekday and weekend traffic are both represented.
- A significant result on 30 conversions is fragile. Look at the absolute counts, not only the p-value.
Significance vs conversion rate
Significance (this page)
- Compares two variants
- Answers whether the gap is real
- Needs both visitors and conversions
Conversion rate calculator
- Describes one variant
- Answers what the rate is
- Needs one pair of numbers
Related calculators
Frequently asked questions
How do you calculate statistical significance for an A/B test?
Use a two-proportion z-test. Pool the conversion rates across both variants, compute the standard error of the difference, divide the observed difference by it, and convert that z to a two-tailed p-value.
If p is below 0.05, the difference is conventionally called statistically significant.
What does statistically significant actually mean?
That a difference this large would be unlikely if the two variants really performed identically. It does not mean the difference is large or that it matters commercially.
Always read the lift next to the p-value before deciding to ship.
How many visitors do I need per variant?
It depends on your baseline rate and the smallest lift worth detecting. Detecting a small relative lift on a low base rate can take tens of thousands of visitors per variant.
The sample size calculator gives a starting figure for a target margin of error.
Can I stop the test as soon as it hits significance?
No. Peeking repeatedly and stopping at the first green result inflates your false positive rate well above 5%.
Set the sample size in advance, run to it, then read the result once.
What confidence level does this use?
The verdict uses the standard 95% level, meaning p < 0.05. The exact p-value is always shown, so you can apply a stricter threshold if your context calls for it.
What if my test is not significant?
You do not have enough evidence that the variants differ. That is not proof they are the same. Either keep running to a larger sample or accept that any real difference is probably too small to be worth chasing.
Does this work for more than two variants?
Not directly. Comparing several variants at once raises the chance of a false positive, so you need an adjustment or an omnibus test. The chi square calculator handles a multi-variant conversion table.
Are my numbers stored?
No. Everything runs in your browser.
Collecting fresh survey responses? Try Formms for a form with a short link and QR.
Collect the responses on your phone next.
Formms builds the survey from a prompt, then you can paste counts back into these calculators anytime.
View a demo form →