What you’ll need
CSV: experiment, visitors_a, conversions_a, visitors_b, conversions_b. Planning uses baseline rate, relative minimum detectable effect and statistical power.
Performance TOOL · FREE TO USE
A/B Test Analyzer
Review independent conversion experiments in a batch, estimate sample requirements and export readable evidence.
Runs on your device · Files stay private01Your workspace
- 1 Add your input
- 2 Run the tool
- 3 Get your results
CSV: experiment, visitors_a, conversions_a, visitors_b, conversions_b. Planning uses baseline rate, relative minimum detectable effect and statistical power. New here? Start with “Try an example”.
Or paste your data below. You can edit it before running the tool.
Your files stay on this device. No sign-up.
Plan a fixed-horizon experiment
8,158 visitors per arm
Approximate two-sided plan at 5% significance. A 20% relative lift on a 5% baseline means a 6% target. Set the stopping rule before starting; this planner does not track experiment progress.
How it works
Fixed-horizon two-sided pooled two-proportion z tests; Newcombe/Wilson difference intervals; optional Holm adjustment across the batch. Low-count tests stay inconclusive.
Know the limits
1,000 independent two-arm binary experiments. No revenue, sequential or causal-certainty claims. Planning is an approximation; do not repeatedly peek at fixed-horizon tests.
GET FROM A QUESTION TO A NEXT STEP
Find out whether a conversion-rate difference is convincing
Variation B has more conversions, but that alone does not tell you whether the difference is reliable. This A/B test calculator compares independent two-arm binary conversion experiments using supplied visitor and conversion counts. It is useful after a planned test or when estimating the sample needed for a future experiment. It keeps statistical evidence separate from the business decision about which experience to ship.
How to use A/B Test Analyzer
Enter experiment names, visitors and conversions for each arm using the example CSV structure. Use independent groups and a consistent conversion definition.
Select the available analysis settings and run the comparison. Review rate differences, confidence intervals, p-values and any batch adjustment together.
For planning, supply the baseline rate, relative minimum detectable effect and desired power. Decide the test horizon before monitoring the outcome.
What do the results mean?
A small p-value is evidence under the test assumptions, not a probability that a variation will win forever. Low-count results remain inconclusive. Repeatedly stopping whenever a fixed-horizon test looks favorable can invalidate the intended interpretation, and statistical significance does not establish commercial value.
What to do next
Use Campaign Forecast Calculator to explore whether a plausible conversion improvement is large enough to matter to your campaign economics.
Keep the scope in mind
1,000 independent two-arm binary experiments. No revenue, sequential or causal-certainty claims. Planning is an approximation; do not repeatedly peek at fixed-horizon tests.
Common questions about A/B Test Analyzer
Why can a large percentage lift still be inconclusive?
A large lift from very few visitors or conversions can be highly uncertain. Review the counts and interval instead of choosing a winner from the percentage alone.
Can this analyze revenue per visitor or multiple variants?
This implementation analyzes independent two-arm binary conversion counts. Revenue distributions, sequential tests and multi-arm experiments require methods outside this tool’s scope.