A/B Test Calculator

How the A/B test significance calculation works

This calculator runs a two-proportion z-test to determine whether the difference in conversion rates between Variant A and Variant B is statistically significant, or could simply be due to random chance.

First, a pooled conversion rate is calculated by combining conversions and visitors from both variants: p = (conversions_A + conversions_B) / (visitors_A + visitors_B). This pooled rate is used under the null hypothesis that both variants truly convert at the same rate.

The standard error of the difference is SE = sqrt(p * (1 - p) * (1/n_A + 1/n_B)), and the z-score is z = (rate_B - rate_A) / SE. The z-score is converted to a p-value using the standard normal cumulative distribution function, and confidence is reported as (1 - p-value) * 100%. A result is generally considered statistically significant once confidence reaches 95% or higher — but larger sample sizes always give more reliable results than smaller ones, so treat a low-traffic test's result with caution even if the confidence number looks high.