What is a confidence interval?

• It represents a range of the difference of the targeted indicator (CVR, open rate, click rate, etc.) between each pattern and baseline. Confidence interval is estimated such that it contains the difference with high probability if this A/B test repeated again with the same condition. It is calculated using the t-distribution. For details, please check CONFIDENCE INTERVAL.

• The numbers indicated are in proportion to the baseline. For example, if the baseline CVR was 10.0%, and it was estimated that pattern 1 of baseline CVR will increase around +2.0%~+3.0%, the screen will show +20.0%~+30.0%.

• « What is a significant difference?
• Will the results change if it was determined to have a significant difference? »