How to Analyze Split Test Data

Interpreting Experiment Results

Once a test has reached significance, the real value comes from understanding why a variant won and what to do next.

1. Access Your Results

• Open Dashboard → Split Testing and click the experiment you’d like to review.

2. Overview Panel

• Status & Dates – confirms the experiment life‑cycle.

• Randomization – Session‑ vs User‑Level.

• Primary Goal – the metric that decides the winner.

• Lift & Confidence – headline stats that tell you if the variant outperformed the control and by how much.

3. Insights & Metrics

Humblytics auto‑calculates a core set of engagement and conversion KPIs. Typical metrics include:

• Click‑Through Rate (CTR)

• Form Submission Rate

• Bounce Rate

• Average Session Time

• Scroll Depth

Pro‑tip: Focus first on the metric that maps directly to your business goal, then sanity‑check secondary metrics for unintended trade‑offs.

4. Target Interactions

• The Top Clicked Elements table surfaces high‑impact buttons, links, or CTAs.

• Pin critical elements so they remain fixed at the top for faster reviews.

5. Real‑Time Monitoring

• Live charts show performance trajectories as data streams in.

• The Race to the Best bar visually indicates when a variant crosses the 95 % confidence threshold.

6. Declaring a Winner

1. Confirm statistical significance (default 95 % confidence).

2. Verify minimum‑sample thresholds are met for each variant.

3. Inspect secondary metrics to ensure there are no adverse effects (e.g., CTR ↑ but Bounce ↑ as well).

7. Rolling Out the Winning Variant

• Deploy the successful design or flow in your CMS, site builder, or custom codebase.

• Optionally keep the experiment running a few extra days to catch any regressions.

8. Iterating Forward

• Log the hypothesis, outcome, and lessons learned in your testing backlog.

• Use insights to craft the next hypothesis—optimization is a continuous loop.

Happy testing! Measure, learn, and iterate with confidence using Humblytics.