Let’s take a quick look at a very important and common experimental problem: checking if the difference in success rates of two Binomial experiments is statistically significant. This can arise in A/B testing situations such as online advertising, sales, and manufacturing.
We already share a free video course on a Bayesian treatment of planning and evaluating A/B tests (including a free Shiny application). Let’s now take a look at the should be simple task of simply building a summary statistic that includes a classic frequentist significance.
Continue reading Quick Significance Calculations for A/B Tests in R
Many data scientists (and even statisticians) often suffer under one of the following misapprehensions:
- They believe a technique doesn’t work in their current situation (when in fact it does), leading to useless precautions and missed opportunities.
- They believe a technique does work in their current situation (when in fact it does not), leading to failed experiments or incorrect results.
I feel this happens less often if you are working with observable and composable tools of the proper scale. Somewhere between monolithic all in one systems, and ad-hoc one-off coding is a cognitive sweet spot where great work can be done.
Continue reading We Want to be Playing with a Moderate Number of Powerful Blocks
The Win-Vector public R packages now all have new
pkgdown documentation sites! (And, a thank-you to Hadley Wickham for developing the
Please check them out (hint:
vtreat is our favorite).
Continue reading More documentation for Win-Vector R packages
sigr is a simple
R package that conveniently formats a few statistics and their significance tests. This allows the analyst to use the correct test no matter what modeling package or procedure they use.
Continue reading sigr: Simple Significance Reporting