Statistics To English Translation – Page 3

June 26, 2011December 18, 2013John MountStatistics, Statistics To English Translation, TutorialsEstimation, Hoeffding's Inequality, Pseudorandomness, Random Sampling

What is a large enough random sample?

With the well deserved popularity of A/B testing computer scientists are finally becoming practicing statisticians. One part of experiment design that has always been particularly hard to teach is how to pick the size of your sample. The two points that are hard to communicate are that:

The required sample size is essentially independent of the total population size.
The required sample size depends strongly on the strength of the effect you are trying to measure.

These things are only hard to explain because the literature is overly technical (too many buzzwords and too many irrelevant concerns) and these misapprehensions can’t be relieved unless you spend some time addressing the legitimate underlying concerns they are standing in for. As usual explanation requires common ground (moving to shared assumptions) not mere technical bullying.

We will try to work through these assumptions and then discuss proper sample size. Continue reading What is a large enough random sample?

February 3, 2010September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English Translationcustomer value, lognormal distribution, long tail theory, McPhee's Theory of Exposure, median versus mean, power law distribution, Statistics

Living in A Lognormal World

Recently, we had a client come to us with (among other things) the following question:
Who is more valuable, Customer Type A, or Customer Type B?

This client already tracked the net profit and loss generated by every customer who used his services, and had begun to analyze his customers by group. He was especially interested in Customer Type A; his gut instinct told him that Type A customers were quite profitable compared to the others (Type B) and he wanted to back up this feeling with numbers.

He found that, on average, Type A customers generate about $92 profit per month, and Type B customers average about $115 per month (The data and figures that we are using in this discussion aren’t actual client data, of course, but a notional example). He also found that while Type A customers make up about 4% of the customer base, they generate less than 4% of the net profit per month. So Type A customers actually seem to be less profitable than Type B customers. Apparently, our client was mistaken.

Or was he? Continue reading Living in A Lognormal World

December 13, 2009September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English TranslationANOVA, F-test, significance, Statistics, t-test

Statistics to English Translation, Part 2b: Calculating Significance

In the previous installment of the Statistics to English Translation, we discussed the technical meaning of the term ”significant”. In this installment, we look at how significance is calculated. This article will be a little more technically detailed than the last one, but our primary goal is still to help you decipher statements about significance in research papers: statements like “
”.

As in the last article, we will concentrate on situations where we want to test the difference of means. You should read that previous article first, so you are familiar with the terminology that we use in this one.

A pdf version of this current article can be found here.
Continue reading Statistics to English Translation, Part 2b: Calculating Significance

December 4, 2009September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English Translationeffect size, hypothesis testing, significance, Statistics4 Comments

Statistics to English Translation, Part 2a: ’Significant’ Doesn’t Always Mean ’Important’

In this installment of our ongoing Statistics to English Translation series¹, we will look at the technical meaning of the term ”significant”. As you might expect, what it means in statistics is not exactly what it means in everyday language.

As always, a pdf version of this article is available as well. Continue reading Statistics to English Translation, Part 2a: ’Significant’ Doesn’t Always Mean ’Important’

November 3, 2009September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English TranslationAccuracy Measures, Classifiers, Diagnostic Tests, Precision and Recall, ROC Curves, Sensitivity and Specificity, Statistics4 Comments

“I don’t think that means what you think it means;” Statistics to English Translation, Part 1: Accuracy Measures

Scientists, engineers, and statisticians share similar concerns about evaluating the accuracy of their results, but they don’t always talk about it in the same language. This can lead to misunderstandings when reading across disciplines, and the problem is exacerbated when technical work is communicated to and by the popular media.

The “Statistics to English Translation” series is a new set of articles that we will be posting from time to time, as an attempt to bridge the language gaps. Our goal is to increase statistical literacy: we hope that you will find it easier to read and understand the statistical results in research papers, even if you can’t replicate the analyses. We also hope that you will be able to read popular media accounts of statistical and scientific results more critically, and to recognize common misunderstandings when they occur.

The first installment discusses some different accuracy measures that are commonly used in various research communities, and how they are related to each other. There is also a more legible PDF version of the article here.

Continue reading “I don’t think that means what you think it means;” Statistics to English Translation, Part 1: Accuracy Measures