Repro - Mobile Analytics for growth
日本語
Sign Up Back to Dashboard
  • System Requirements
  • Dashboard Guide
  • Development Guide
  • Release Notes
  • FAQ
    • App
    • Web
    • General
      • Analytics
      • A/B test significant difference display
        • At what point is the data collected that is used for computing the significant difference?
        • The screen hasn’t changed from “There isn’t a significant difference yet”. Is there a rough indication of when it will be completed?
        • Around how many delivery counts is needed to display the results?
        • What is a significant difference?
        • What is a confidence interval?
        • Will the results change if it was determined to have a significant difference?
        • Is it possible to go back to past significant differences?
        • Is it possible to change the significance level used in the confidence interval?
        • When is automatic winner selection performed?
        • Can I enable automatic winner selection for a campaign that is already being delivered?
        • Can the control group be automatically selected as the winning variant?
      • API
      • Miscellaneous

Around how many delivery counts is needed to display the results?¶

Significance can be calculated when each variant has a delivery count (sample size) of 100 or more. If the delivery count of any variant is less than 100, or if the baseline result rate (the value of the target indicator) is 0, “Cannot calculate” is displayed. Note that results tend to fluctuate while the delivery count is small, so we recommend waiting a certain period before making a decision.

  • « The screen hasn’t changed from “There isn’t a significant difference yet”. Is there a rough indication of when it will be completed?
  • What is a significant difference? »

Repro Service Site About Us Careers Terms of Service Privacy Policy Cookie Policy

© 2022 Repro Inc.