# Incrementality Testing > Geo holdout experiments with synthetic control groups that measure what your ads actually add — not what attribution claims they do. Are your ads driving sales — or just taking credit for them? Find out with SegmentStream GeoLift Experiments. *8 min read · March 2026* ## 01 — The Case for Incrementality Testing [Cross-channel attribution](/measurement-engine/cross-channel-attribution) is the foundation of daily optimization. It connects clicks to conversions, provides granular performance insights, and feeds platform bidding algorithms. But attribution has blind spots. Take Google Brand Search. You bid on your own brand name. In analytics, the campaign shows outstanding ROAS. But would those conversions have happened anyway? If someone searches your brand name, they already know you. Without the paid ad, they'd likely click the organic link and convert regardless. You might not lose a single sales dollar — while saving a huge portion of your paid media budget. This is exactly why incrementality tests were invented — to measure what ads actually add to your overall sales. --- ## 02 — Two Types of Incrementality Tests ### In-Platform Lift Studies (and Why to Avoid Them) Meta, Google, TikTok, and Snap offer built-in "lift studies" that randomly split users into test and control groups. Two problems make them unreliable: **Unequal tracking.** Test groups get pixels, click IDs, and client-side identifiers. Control groups rely on server-side CAPI only. Test groups show more observed conversions because measurement is better — not necessarily because ads worked. **Blackbox.** You can't see how audiences were split, how conversions were counted, or what methodology was used. The platform selling ads is also grading them. > The platform selling you ads is also measuring whether those ads work — using methods you cannot audit. Avoid in-platform lift studies at all costs. ### Geo Holdout Experiments Geo holdouts split at the regional level — countries, states, cities, DMAs, or ZIP codes. Ads run normally in control regions and are paused in holdout regions. Geography is a clean boundary: no cross-contamination, no audience overlap, and you control the entire experiment. Instead of analyzing performance on a user level, you look at total sales across regions — without any attribution at all — and check: did sales go up or down because certain ad activity was turned off? This is the most accurate methodology for measuring incrementality — and it's exactly what SegmentStream offers. --- ## 03 — SegmentStream's Approach to Rigorous Geo Holdout Testing > **[Illustration: Experiment Phases]** > End-to-end experiment flow showing the sequential phases of a geo holdout test. ### Market Selection SegmentStream analyzes historical sales data across all available regions — factoring in seasonality, trends, and patterns. This surfaces regions that historically behave similarly, so they can be used for a valid experiment. Once the correlated regions are identified, SegmentStream splits them into two groups — test and control — so that each group's aggregate sales track in parallel: same peaks, same dips, same growth rate. Pause ads across the test group, keep them running across the control group, and you have a clean comparison. > **[Illustration: Region Correlation]** > The system identifies regions with correlated sales trends, then assigns them to test and control groups. ### Synthetic Controls Even correlated regions almost never have the same absolute sales volume. California might do $800K/month while Ohio does $420K. Comparing raw numbers would be meaningless. Synthetic control modeling solves this by applying weighted coefficients to each region, scaling them to a common baseline. The result: an apples-to-apples comparison where the only variable is whether ads were running or not. > **[Illustration: Synthetic Control]** > Raw sales at different absolute levels, then weighted to a common scale for valid comparison. ### Minimum Detectable Effect Every experiment has an MDE — the smallest lift it can reliably detect. SegmentStream calculates MDE upfront based on the number of regions, volume per region, and baseline variance. If the expected effect is smaller than MDE, the system flags it before the test runs — preventing wasted experiments that were never powered to produce a result. > **[Illustration: MDE]** > An 18% lift clears the 15% MDE threshold. A 5% lift is invisible — the test can't detect it. ### A/A Validation Before pausing any ads, SegmentStream runs an A/A period — both groups receive identical treatment. If the groups track closely, the experiment design is confirmed valid. If they diverge, the system flags the issue and adjusts region selection automatically. This catches competitor launches, seasonal anomalies, and assignment bias before they contaminate results. > **[Illustration: A/A Validation]** > Left: groups track together — valid design. Right: groups diverge — fix selection before proceeding. ### Sales Cycles SegmentStream accounts for your sales cycle length when designing the experiment. If your average cycle is 14 days, conversions at the end of the holdout were influenced by ads that ran before it started. The system automatically extends the test window or excludes the contaminated tail from evaluation. > **[Illustration: Sales Cycle Lag]** > The lag zone at the end of the holdout is contaminated by pre-test ad exposure. ### Evaluation Once the holdout period ends, SegmentStream doesn't just compare raw sales numbers between test and control groups. It applies the same modeling coefficients that were used to create synthetic controls — scaling sales from each region up or down to ensure incrementality is calculated on a normalized basis. This is a common mistake when running geo tests manually: marketers look at raw sales in their analytics reporting, see different numbers between regions, and draw conclusions — completely ignoring the synthetic control modeling that makes the comparison valid in the first place. > **[Illustration: Geo Experiment Chart]** > Control and holdout track closely pre-test. During the experiment, holdout drops. The gap is incremental lift. But a single incrementality number by itself is meaningless without knowing how confident you can be in it. That's why SegmentStream always reports results with **confidence intervals** — the range within which the true effect most likely falls. If a test shows +35% incremental lift with a confidence interval of [22%–49%], the entire range is above zero — the result is statistically significant. But if another test shows +5% with an interval of [−8%–+18%], the range includes zero, meaning the result is inconclusive. The ads might be incremental, or the observed difference might be noise. > **[Illustration: Confidence Interval]** > Google brand search: significant (CI above zero). TikTok awareness: inconclusive (CI spans zero). --- ## 04 — Geo Lift Experiments Made Easy Before, running or analyzing an incrementality test wasn't possible without a data science expert. With SegmentStream, you can design, launch, and evaluate incrementality tests — without any technical skills, knowing you will get trustworthy insights. Manage Geo Holdout Experiments from the SegmentStream UI: Or directly from your favourite AI tools like Claude Code, Claude Cowork, Cursor, or Codex: > **[Interactive: Incrementality Cowork Demo]** > AI conversation demo showing how to design and evaluate an incrementality test through natural language. --- ## 05 — Important Considerations Incrementality testing often sounds great in theory, but in practice — most tests are done wrong, or shouldn't be launched in the first place. Here are the most important things to understand before running one. ### Geo Holdouts Can't Measure Long-Term Brand Effects Many teams think that if last-click attribution can't measure upper-funnel, they need to jump straight to incrementality tests. The problem: most geo holdouts run for 2–4 weeks. If your goal is to measure how upper-funnel ads influence brand recognition and organic demand over time, a short holdout is simply the wrong methodology. Brand effects compound over months — a 4-week test won't capture them. ### Geo Holdouts Have Hidden Costs The primary cost isn't the platform fee or the setup effort. Tests are invasive — you stop showing ads to a large chunk of your audience. That means a noticeable drop in sales during the test period. You should only run an incrementality test if *not knowing* incrementality could lead to even bigger losses over time. ### Incrementality Means Nothing Without Confidence Intervals A headline number like "+12% lift" is meaningless if the confidence interval is [−5%, +29%]. The range includes zero, so you can't tell whether the ads had any effect at all. Always look at the interval, not the point estimate. ### Forgetting to Adjust Budgets in the Control Market When you remove test regions from targeting, the freed budget overflows into control regions — making them spend more than they should. This breaks the entire purpose of the test. It's one of the single biggest reasons why geo holdouts produce unreliable results. > At SegmentStream, we take incrementality testing seriously — and would never recommend running a test unless the specific use case requires it and there is no substitute methodology. But when a test makes sense to run, you can be confident the results are ones you can actually trust.