In Attribution Settings > Advanced, there are a few Incrementality-specific tweaks that can be made for how the data is displayed. In order to adjust any of those settings, incrementality must be enabled. You can do that by going to the Overview tab on Podscribe, opening up the Attribution Settings, and toggling ‘Incrementality’ on:
Next, expand ‘Advanced’ to see the ‘Incrementality’ settings:
Each toggle can change the way metrics are calculated on the dashboard.
Show Ranges: Podscribe’s incrementality feature calculates confidence intervals, or the range where the true value of incrementality exists. Typically it is represented as a ‘+/-’ value, like +/- 3%. With Show Ranges toggled on, the metrics will show the range of possible values based on the confidence interval.
For example, if the incrementality in Podscribe shows 20% with a confidence interval of +/- 3%, it means the true incrementality is between 17% and 23%. Let’s also say that standard attribution said there were 100 attributed conversions. That means the true incremental number is somewhere in the range of 17 and 23 incremental attributed conversions. This can be useful when making strategic decisions, as the performance of the channel or a campaign could span above and below your goal, while those that span below your goal are easier to optimize away from.
Adjust for Bias: When generating control groups using our synthetic methodology, it is possible that the control group may perform better or worse than the exposed group in the month prior to the testing month. This is called bias, and Podscribe can remove this bias from the results by toggling this feature on.
Action Override: By default, incrementality is calculated separately for each action. However, if an action has low statistical significance, you can select a different action from this dropdown and apply its incrementality to all actions.
Compute By: Podscribe runs two separate incrementality tests; one at the channel level, and one at the campaign level. Note that channel-level control groups are mutually exclusive to the exposed group at the channel-level. Campaign-level incrementality uses control groups that are mutually exclusive to the exposed group at the campaign-level. This means that a household in the control group in one campaign could exist in the exposed group in another campaign.
The dropdown determines which test the dashboard uses to calculate the incrementality numbers (both the percentages and absolute numbers).
- Channel - All incrementality numbers are calculated using the synthetic channel-level test, including campaign incremental numbers.
- Channel + SmartServe - All campaign results use the synthetic channel-level test to calculate results, except for campaigns that are using SmartServe incrementality (either PSA or Ghost hold out groups). At the channel level, results will combine the synthetic channel-level test results and SmartServe incrementality results to inform the channel incrementality results.
- Campaign - All campaign incrementality numbers are calculated using the campaign-level incrementality test, whereas the channel results use the channel-level test. When a campaign’s metric has not reached statistical significance, the results we fall back to using the channel-level results, which are generally more stable.
- Campaign + SmartServe - All campaign results use the synthetic campaign-level test to calculate results except for the campaigns using SmartServe incrementality (either PSA or Ghost hold out groups). At the channel level, results will combine the synthetic campaign-level test results and smartServe incrementality results to inform the channel incrementality results. When a campaign’s metric has not reached statistical significance, the results we fall back to using the channel-level results, which are generally more stable.
Powered by