Repro - Mobile Analytics for growth
日本語
Sign Up Back to Dashboard
  • System Requirements
  • Dashboard Guide
  • Development Guide
  • Release Notes
  • FAQ
    • App
    • Web
    • General
      • Analytics
      • A/B test significant difference display
        • At what point is the data collected that is used for computing the significant difference?
        • The screen hasn’t changed from “There isn’t a significant difference yet”. Is there a rough indication of when it will be completed?
        • Around how many delivery counts is needed to display the results?
        • What is a significant difference?
        • What is a confidence interval?
        • Will the results change if it was determined to have a significant difference?
        • Is it possible to go back to past significant differences?
        • Is it possible to change the significance level used in the confidence interval?
        • When is automatic winner selection performed?
        • Can I enable automatic winner selection for a campaign that is already being delivered?
        • Can the control group be automatically selected as the winning variant?
      • API
      • Miscellaneous

When is automatic winner selection performed?¶

Significance is evaluated once a day using the cumulative delivery and conversion counts since the campaign started. If a variant is significantly better than the baseline, that variant is decided as the winner; if all variants are significantly worse than the baseline, the baseline is decided as the winner. If there is no significant difference or the result cannot be calculated, no decision is made and evaluation continues on the following days.

  • « Is it possible to change the significance level used in the confidence interval?
  • Can I enable automatic winner selection for a campaign that is already being delivered? »

Repro Service Site About Us Careers Terms of Service Privacy Policy Cookie Policy

© 2022 Repro Inc.