Humblytics
  • Overview
    • Introduction to Humblytics
    • Features Overview
    • Frequently Asked Questions
  • How to Get Started
    • Create a New Account
    • Add Humblytics Analytics to a Custom / Self-Hosted Site
    • How to Add Humblytics Analytics to Your Weblow Site
    • How to Add Humblytics Analytics to Your Framer Site
  • Split Testing Overview
    • How Humblytics Split Testing Works - Under the Hood
    • How to Setup a Split Test
    • Creating A/B Page Variants for Split Testing
    • How to Analyze Split Test Data
    • Using the Humblytics A/B Sample‑Size Calculator
    • Deciding How Long to Run an A/B Test
  • How to Track Custom Form Submissions
    • GoHighLevel
    • Tally.so
      • Webflow – How to Track Tally.so Form Submissions
      • Framer – How to Track Tally.so Form Submissions
      • Custom/Self-Hosted – How to Track Tally.so Form Submissions
    • Typeform
      • Webflow – How to Track Typeform Submissions
      • Framer – How to Track Typeform Submissions
      • Custom/Self-Hosted – How to Track Typeform Submissions
    • Cal.com
      • Webflow – How to Track Cal.com Booking Submissions
      • Framer – How to Track Cal.com Booking Submissions
      • Custom/Self-Hosted – How to Track Cal.com Booking Submissions
    • Hubspot
      • Framer – How to Track HubSpot Form Submissions
      • Custom/Self-Hosted – How to Track HubSpot Form Submissions
      • Webflow – How to Track HubSpot Form Submissions
  • How to Track Custom Click Events
    • How to Track Click Events on Custom / Self‑Hosted Site
    • How to Add Custom Event Tracking for Webflow Sites
    • How to Add Custom Event Tracking for Framer Sites
  • Understanding Your Data
    • Understanding Site Traffic
    • Understanding Pages Data
    • Understanding Click Data
    • Understanding Forms Data
    • Understanding Heatmap
    • Understanding Funnels
  • Campaign Tracking with UTM Links
Powered by GitBook
On this page
  • 1. Why Sample Size Matters
  • 2. Input Definitions
  • 3. Step‑by‑Step
  • 4. Worked Example
  • 5. Best‑Practice Reminders

Was this helpful?

  1. Split Testing Overview

Using the Humblytics A/B Sample‑Size Calculator

A/B (split) testing compares a control (Variant A) with a variation (Variant B) so you can make evidence‑based improvements to your website or app. The Humblytics Sample‑Size Calculator tells you exactly how many visitors each variant needs before you can trust the result.

1. Why Sample Size Matters

Too Small

Just Right

Too Large

Results look erratic; you risk acting on noise.

Detects real differences with high confidence.

Wastes time and traffic without adding precision.

Choosing the correct sample size balances statistical rigour with business velocity.

2. Input Definitions

Field

What It Means

Example

Baseline Conversion Rate

Your current conversion rate.

5 % (5 of every 100 visitors convert)

Minimum Detectable Effect (MDE)

The smallest lift you care about.

+1 % absolute (from 5 % → 6 %)

Statistical Significance

Confidence level that the observed lift is real, not random.

95 % (industry default)

Statistical Power

Probability of detecting an effect if it exists.

80 % (common default)

Tip: Lower MDE or higher confidence / power settings will increase the required sample size.

3. Step‑by‑Step

  1. Open the Humblytics Sample‑Size Calculator.

  2. Enter each value defined above.

  3. Click Calculate.

  4. Record the required visitors per variant shown in the results panel.

  5. Plan your test window so you can realistically hit those numbers.

4. Worked Example

  • Baseline Conversion Rate: 5

  • MDE: 1

  • Significance: 95

  • Power: 80

▶︎ Result: ≈ 4,000 visitors per variant (total ≈ 8,000 sessions). Run the experiment until both A and B have reached these counts before analysing.

5. Best‑Practice Reminders

  • One Variable at a Time — isolate the element you’re testing.

  • Run Full Business Cycles — capture weekday/weekend traffic differences.

  • Don’t Peek Early — premature stops inflate false‑positive risk.

  • Look Beyond Win/Loss — examine bounce rate, engagement, revenue per visitor.

  • Iterate — document learnings and queue up the next hypothesis.

Following this workflow ensures every Humblytics test is powered correctly, statistically sound, and focused on meaningful business impact.

PreviousHow to Analyze Split Test DataNextDeciding How Long to Run an A/B Test

Last updated 2 days ago

Was this helpful?