Using Geo Testing to measure Incrementality

As advertising evolves to include greater customer privacy, measurement methods that focus on aggregation and modelling can aid in understanding of true lift. This is because modifications like cookie removal have no impact on the data utilised to develop such procedures. Geo testing is one such method that marketers must use to compute lift for a range of channels.

In order to successfully carry out geo testing and comprehend incrementality, marketers could turn to SYNC MOS for expertise. It is adaptable to changes in the marketing landscape and uses aggregated data to quantify the incremental impact on marketing initiatives. SYNC MOS is optimized for cross-media applications, repeatability, and scalability. More on this later.

So What is Incrementality in Marketing?

The lift or increase in the targeted outcome delivered by marketing effort is referred to as incrementality. Profitability, revenues, conversions, site views, or brand exposure could all be these targeted outcomes. At the ad, campaign, strategy, or channel level, incrementality testing and measurement deliver a true incremental contribution of your paid media making incrementality in marketing is especially important for channels where ad impressions are difficult to map and measure, such as social channels like Facebook, Snap, Pinterest, and others. You can use incrementality measurement to determine:

• There is waste to be eliminated, as well as chances to scale, expand, and shift media spend for optimal growth.
• Contributions to marketing goals from media outlets, publishers, and campaigns
• Whether to launch additional campaigns or advertising to boost the portfolio’s contribution to conversions.

In a nutshell, Incrementality isn’t about assigning credit to a conversion; it’s about identifying the interaction that moves a user from passive to active. The interaction that effects the actual outcome is referred to as incremental.

Using Geo Lift for Incrementality: The Gold Standard of Measurement

AB Testing has recently become the gold standard for product development teams. Experimenting with the product allows teams to understand the incremental impact of each change on important KPIs and gradually enhance their products.

Problem: Yet, arbitrarily deploying a feature to some consumers but not others can result in biassed results. This could be due to network effects, inventory overflow, or, more broadly, conditions in which the Stable Unit Treatment Value Assumption (SUTVA) is violated. This is the notion that how one randomisation unit responds to a product change is unrelated to how any other unit responds to the same product change. The randomisation unit is the level at which we assign each variant at random. In most cases, this is a single user, but it might also be a device, a session, a social cluster, or something else.

Several components of a social product, for example, require two-way involvement to function. Because not all of their friends have access to the same feature change, randomly presenting users a feature change may result in some of those users not getting a proper chance to engage with it. As a result, the test success numbers may not be genuinely representative of the effect of rolling out the feature to all of your users.

Quick Solution: We could control for these in a variety of ways by changing the design of our experiment. In the case of a social network, for example, we may divide the network into clusters of individuals who frequently engage with one another and do the random assignment at the cluster level. But, executing such complex experiment designs can incur significant development costs. As a result, we may not always have the resources to carry out the experiment.

Better Solution: Geo Lift Experiment

One possible solution to this difficulty would be to do a geo lift experiment as a quasi-experiment. The plan would be to roll out the feature change to all users in a defined geographic area. The key success metrics from this geo are then compared to those from a Control geo where the feature modification was not released. We then use this comparison to determine the feature’s causal effect on our key success measures.

Assumptions: It’s important to note that this only works as a solution if you believe that limiting the trial to a specific area represents releasing the functionality to your whole user base. With the social product example, this only works as a solution if the majority of consumers assume that all other users they engage with on the product are from the same geographic area as them.

The method is founded on the Difference in Differences method for causal inference. The difference in differences approach involves utilizing two comparable user segments and modeling the key success metrics before and after the implementation of a feature change. One of the user segments is designated as the Control group and does not receive the feature change during the quasi-experiment, while the other user segment is the Treatment group and does receive the change. For each success metric, we create a model that considers the following:

We calculate the difference between the actual treatment performance (solid orange line) and the counterfactual

Benefits of Using GeoLift to Measure Incrementality

1. Help Brands Invest in Data-Based Decision-Making

Measurement frameworks can present numerous challenges for advertisers, and one of these challenges is the tendency for brands to allocate a large proportion of their budgets to one or two digital channels. This makes it more difficult to determine the return on investment (ROI) of investing in newer platforms. GeoLift can be an effective tool to overcome these challenges. A data-based approach of ad performance can be achieved because ads are served to a subset of people and not a similar subset. By controlling other factors that could influence performance, lift tests enable advertisers to measure instrumentality and determine the direct impact caused by specific marketing activity.

2. Evaluate Media Mix More Effectively

When conducting lift studies, brands tend to concentrate their spending on one or two approaches, which can limit the advertiser’s understanding of channel performance and overall performance. However, GeoLift tools can help advertisers to maximise the benefits of lift testing by providing a more comprehensive view. By using geographical boundaries to create test and control groups, GeoLift offers advertisers the flexibility to better understand the performance of different media sizes, rather than just individual channels. This approach enables advertisers to gain a more complete understanding of how various media channels work together to drive conversions and optimize their overall marketing strategies.

3. Help Understand the Incremental Impact

GeoLift is a powerful tool that can help marketers gain insights into the incremental impact of their campaigns relative to their budgets. Determining the return on investment (ROI) of investing in new marketing approaches can be a significant challenge for most advertisers when using traditional techniques. However, lift tests enable brands to better understand relative channel performance and effectively compare diverse media mixes. This approach can help advertisers to optimize their marketing strategies, by determining the most effective media channels for driving conversions and maximizing their ROI.

SYNC MOS Solution

Did you know that difference-in-difference is not the only geo-lift experiment that can be conducted, in fact there are three types of GeoLift Experiments:

1. Difference-in-Difference
2. Market Match Tests
3. Synthetic Control Method

SYNC MOS is helping advertisers calculate lift at a geographical level but not with one solution say difference-in-difference fits all. Our methodology applies the custom synthetic control method to generate geographic quasi-experiments that measure the true incremental value of marketing campaigns.

Unlike conventional geo-experimentation techniques, SYNC MOS chooses the best treatment design for the experiment via simulations and power calculations. SYNC MOS provides accurate and actionable lift results from robust inference capabilities that can detect lift with smaller samples and effect sizes. This solution helps make otherwise complicated process very simple and executable.

Why SYNC MOS incrementality is one stop solution

• Gold Standard Measurement
• Fundamentally incremental
• End to end solution from data to statistics to business insights and decision
• Works with any and every channel
• Flexible and customizable as per business requirement
• Not impacted by industry changes