Using the Humblytics A/B Sample‑Size Calculator

A/B (split) testing compares a control (Variant A) with a variation (Variant B) so you can make evidence‑based improvements to your website or app. The Humblytics Sample‑Size Calculator tells you exactly how many visitors each variant needs before you can trust the result.

---

1. Why Sample Size Matters

Too Small	Just Right	Too Large
Results look erratic; you risk acting on noise.	Detects real differences with high confidence.	Wastes time and traffic without adding precision.

Choosing the correct sample size balances statistical rigour with business velocity.

---

2. Input Definitions

Field	What It Means	Example
Baseline Conversion Rate	Your current conversion rate.	5 % (5 of every 100 visitors convert)
Minimum Detectable Effect (MDE)	The smallest lift you care about.	+1 % absolute (from 5 % → 6 %)
Statistical Significance	Confidence level that the observed lift is real, not random.	95 % (industry default)
Statistical Power	Probability of detecting an effect if it exists.	80 % (common default)

Tip: Lower MDE or higher confidence / power settings will increase the required sample size.

---

3. Step‑by‑Step

1. Open the Humblytics Sample‑Size Calculator.

2. Enter each value defined above.

3. Click Calculate.

4. Record the required visitors per variant shown in the results panel.

5. Plan your test window so you can realistically hit those numbers.

---

4. Worked Example

• Baseline Conversion Rate: 5

• MDE: 1

• Significance: 95

• Power: 80

▶︎ Result: ≈ 4,000 visitors per variant (total ≈ 8,000 sessions). Run the experiment until both A and B have reached these counts before analysing.

---

5. Best‑Practice Reminders

• One Variable at a Time — isolate the element you’re testing.

• Run Full Business Cycles — capture weekday/weekend traffic differences.

• Don’t Peek Early — premature stops inflate false‑positive risk.

• Look Beyond Win/Loss — examine bounce rate, engagement, revenue per visitor.

• Iterate — document learnings and queue up the next hypothesis.

Following this workflow ensures every Humblytics test is powered correctly, statistically sound, and focused on meaningful business impact.