# How Humblytics Split Testing Works - Under the Hood When you run a split test with Humblytics, the logic is seamlessly integrated into our standard analytics script—no extra setup required. If a visitor qualifies for an experiment, they'll be automatically and smoothly directed to the correct version of the page based on your test configuration. We've designed our split testing engine to prioritize performance, SEO, and user experience: Performance-first optimization without trade-offs | Capability | What it means | | --------------------------------- | ------------------------------------------------------------------------------------------------- | | Lightning-fast & lightweight | Adds only a few kilobytes and loads asynchronously, so it never blocks rendering | | Seamless redirect handling | Variant redirects occur before paint to avoid visual flicker or flashes of original content | | Analytics-aware event suppression | Silences events for the original page on redirect, preventing double-counting | | Bot-safe by design | Skips testing logic for bots and crawlers, protecting search-engine visibility and metrics | | SEO-friendly canonical control | Automatically points variant pages to the control with canonical tags; index only what you choose | | Cookie-free audience assignment | Uses short-lived query parameters instead of cookies, preserving privacy and compliance | | Flexible targeting | Choose session-level (new assignment each visit) or user-level (sticky experience) splits | | SPA-compatible | Works across multi-page sites and single-page apps, tracking navigation changes automatically | Build, launch & learn—without slowing down your site or compromising SEO We've built Humblytics Split Testing to give you powerful optimization tools with zero performance trade-offs. Whether you're testing headlines, layouts, or full-page experiences, you can accomplish it all without slowing down your site or compromising your SEO. ### Measuring Split Test Performance Once your test is live, Humblytics automatically tracks how each variant performs against your selected goal—whether that's a button click, form submission, or page visit. We handle all the heavy lifting in the background, so you can focus on results, not statistics. #### Goal Tracking Each split test has a primary goal—the action you're trying to optimize. For example: * Clicking a "Sign Up" button * Reaching a confirmation page * Submitting a contact form You'll define this goal when setting up your test, and we'll track how often it occurs for each variant. #### Conversion Rate & Lift We calculate the conversion rate for each variant by dividing the number of goal completions by the number of views. From there, we show you: * **Absolute performance** (e.g., 12% vs. 10%) * **Relative lift** (e.g., Variant B is performing 20% better than the control) #### Confidence & Declaring a Winner To ensure differences aren't just due to random chance, we apply standard statistical techniques to estimate confidence. When one variant performs significantly better than the others with enough data behind it, we flag it as the likely winner. While we aim to give you actionable results quickly, we're also cautious about jumping to conclusions too early. In general: * The more traffic you have, the faster we can detect a winner * If results are close, we'll wait for more data to improve accuracy * We visually show when a result is trending better—but not yet statistically significant You'll always see a clear summary of which variant is winning, by how much, and how confident we are in the result. #### Continuous Monitoring You don't need to manually calculate anything - our dashboard keeps everything up to date in real time. You can check in at any time to see how your test is performing and decide whether to: * Let it run longer * Manually pick a winner * End the test and apply the changes ### How Confidence Is Calculated (Under the Hood) For each variant, we track: * **Number of views** (visitors) * **Number of goal completions** (conversions) * **Conversion rate** = conversions ÷ visitors To compare performance between two variants (e.g., Control vs. Variant B), we calculate the confidence level using a two-proportion Z-test, which tells us how likely the observed difference in conversion rates is due to chance. #### The Steps: 1. **Define conversion rates for both groups:** * p₁ = conversions\_A ÷ visitors\_A * p₂ = conversions\_B ÷ visitors\_B 2. **Calculate pooled probability** (the average conversion rate across both groups): * p = (conversions\_A + conversions\_B) ÷ (visitors\_A + visitors\_B) 3. **Compute standard error (SE):** * SE = √\[p × (1 - p) × (1/visitors\_A + 1/visitors\_B)] 4. **Calculate Z-score:** * Z = (p₁ - p₂) ÷ SE 5. **Convert Z-score to confidence level** using the cumulative distribution function (CDF) of the normal distribution. The resulting confidence level represents the probability that the observed difference is statistically significant (i.e., unlikely to be due to random chance). For example, a Z-score of ±1.96 corresponds to approximately 95% confidence. In the UI, we surface this confidence level with visual indicators (e.g., "95% confidence this variant performs better") and show trending results when the confidence threshold hasn't been reached yet. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.humblytics.com/split-testing-overview/how-humblytics-split-testing-works-under-the-hood.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.