Repro Web
Measuring Effectiveness.
Measurement of Effectiveness
Marketing > Click on the message.
Click on the campaign of your choice on this screen.
On this screen, click on the campaign of your choice.
On the next screen, you can see the delivery results.
On the next screen, you can see the results of the delivery.
The graph shows the number of web messages delivered, the number of button 1 presses, the number of button 2 presses, the main conversion rate and the sub-conversion rate for each pattern.
Number of Distributions
This is the total number of users whose web messages were displayed on a daily basis.
This is the total number of users who received a web message.
Number of conversions
The number of users who converted within 24 hours of delivery, summed up on a daily basis.
This is the number of users who converted within 24 hours of delivery.
Conversion Rate
Conversions / Deliveries.
Trend of Results
The results of daily totals of the number of deliveries, the number of button 1 clicks, the number of button 2 clicks, the main conversion rate, and the sub-conversion rate during the campaign period are output as graphs. The unit of measurement is users.
How to measure the effectiveness of A/B testing
When an A/B test is running, the results of the test run will be displayed.
When the A/B test is running, the results of the test run will be displayed.
You can select a baseline for each pattern, including the control group, or change the indicator of interest.
Pattern
Show each pattern that is sent in the A/B test, including the control group.
Baseline
When comparing each pattern in the A/B test, select a baseline as the reference pattern. By changing the baseline, you can compare various patterns.
Indicators to be covered
Select the metrics to use when comparing each pattern to the baseline in the A/B test. You can choose from a set of percentage values (button 1-click rate, button 2-click rate, main conversion rate, and sub-conversion rate).
90% confidence interval
The 90% confidence interval for each pattern compared to the baseline is displayed. html#id1" title="FAQ">FAQ in the development documentation.
.
- Plus compared to baseline
Such an indication means that the difference between the pattern and the baseline is significant. If enough time has elapsed since the delivery started, it is recommended to proceed to determining the winning pattern.
- No significant difference
This indication means that the difference between the pattern and the baseline is not significant. We recommend waiting a little longer to see results, or if enough time has passed since you started delivering, assume there is no difference and consider your next campaign.
- Negative compared to the baseline
If this is the indication, it means that the difference between the pattern and the baseline is significant. Also, in such displays, it is recommended that the pattern set as the baseline be determined as the winning pattern.
- There is no value in the indicator
In such a display, the pattern means that the indicator's value does not exist and therefore it is impossible to calculate significant differences.
Points to keep in mind
When you select a winning pattern, the A/B test result will be displayed as shown below, and the comparison results for each pattern will not be displayed.
List screen
When a campaign running an A/B test is public or private, you can check the A/B test status of each campaign in the list screen.
The conditions for displaying icons are as follows
Points to keep in mind
If there is a significant difference and there are both campaigns with results above and below the baseline, it appears as "If there is a significant difference and there is one or more patterns of results above the baseline".
Pick a winning pattern
If you choose a winning pattern, the pattern will be delivered to all users in subsequent web messages.
