Repro App
Campaign Mesurement
How to measure the campaigns
Please go to MARKETING>MESSAGING
On this screen, click the campaign you want to check.
On the next screen, you can see the result of campaign delivery.
The graph shows the number of in-app messages delivered and the conversion rate for each pattern.
Deliveries
Total number of users who saw the delivered message on a daily basis.
Conversions
Number of total users who have performed the campaign goal event set from the creation page within 24 hours on a daily basis.
Conversion Ratio
Number of conversions divided by deliveries.
Campaign Progress
Results of the deliveries, opens and occurrence of the campaign goal event on a daily basis: Note that the results represent the number of users.
How to conduct A/B test
If the A / B test is being executed, the execution result of the A / B test is displayed.
You can select each pattern including the control group as the baseline and change the targeted indicator.
Pattern
Each pattern used to deliver the A/B test, including the control group, will be displayed here.
Baseline
Here, you will be able to select the baseline pattern that will be used as the standard when comparing each pattern. By changing the baseline, you will be able to compare between a variety of different patterns.
Targeted indicator
An indicator is used when comparing patterns to baselines in A/B tests. You can select a ratio value (i.e. Button1 CTR, Button2 CTR, CVR) as the indicator. Also, the indicators listed can be changed through the editing page.
90％ confidence interval
The 90% confidence interval will be displayed when each pattern is compared to the baseline. For more details on 90% confidence interval, please see the development guide.
Use case 1
When displayed as such, it means that the difference between the pattern and the baseline is significant. We recommend moving onto the winner pattern if enough time has passed since delivery.
Use case 2
When displayed as such, it means that the difference between the pattern and the baseline is not significant. Either wait until the results appear or consider moving on to the next campaign if enough time has passed since the delivery.
Use case 3
When displayed as such, it means that the difference between the pattern and the baseline is significant. If this is the only window displayed, we recommend determining the baseline pattern as the winner pattern.
NOTE
Below is an example of when the winner pattern is selected from the A/B test significant difference results. The comparison results of the rest of each pattern will not be displayed.
Status List Screen
You will be able to check the status of the A/B test for each campaign here, when the operating campaign is either live or not. Icon display conditions are as follows.
NOTE
In the instance of a campaign that has a significant difference with the baseline having both exceeding and falling below results, it will be displayed as the results of the pattern with at least one exceeding the baseline.
Select the winner
Once you select the winner, all after deliveries of the same in-app message will be using the selected pattern to all target users.