Statistics To English Translation – Page 3

November 21, 2011February 22, 2013Nina ZumelExpository Writing, Pragmatic Machine Learning, Statistics, Statistics To English Translationcorrelation, goodness-of-fit, linear regression, R-squared, Statistics1 Comment

Correlation and R-Squared

What is R²? In the context of predictive models (usually linear regression), where y is the true outcome, and f is the model’s prediction, the definition that I see most often is:

In words, R² is a measure of how much of the variance in y is explained by the model, f.

Under “general conditions”, as Wikipedia says, R² is also the square of the correlation (correlation written as a “p” or “rho”) between the actual and predicted outcomes:

I prefer the “squared correlation” definition, as it gets more directly at what is usually my primary concern: prediction. If R² is close to one, then the model’s predictions mirror true outcome, tightly. If R² is low, then either the model does not mirror true outcome, or it only mirrors it loosely: a “cloud” that — hopefully — is oriented in the right direction. Of course, looking at the graph always helps:

The question we will address here is : how do you get from R² to correlation?

Continue reading Correlation and R-Squared

September 23, 2011September 23, 2011John MountExpository Writing, Statistics, Statistics To English Translation, TutorialsCalculus of Variations, log-likelihood, Logistic Regression, Max-Ent, Maximum Entropy, Statistics

The equivalence of logistic regression and maximum entropy models

Nina Zumel recently gave a very clear explanation of logistic regression ( The Simpler Derivation of Logistic Regression ). In particular she called out the central role of log-odds ratios and demonstrated how the “deviance” (that mysterious
quantity reported by fitting packages) is both a term in “the pseudo-R^2” (so directly measures goodness of fit) and is the quantity that is actually optimized during the fitting procedure. One great point of the writeup was how simple everything is once you start thinking in terms of derivatives (and that it isn’t so much the functional form of the sigmoid that is special but its relation to its own derivative that is special).

We adapt these presentation ideas to make explicit the well known equivalence of logistic regression and maximum entropy models. Continue reading The equivalence of logistic regression and maximum entropy models

September 14, 2011September 24, 2014Nina ZumelExpository Writing, Pragmatic Machine Learning, Statistics, Statistics To English Translation, Tutorialslikelihood, log-likelihood, Logistic Regression, newton's method, Statistics4 Comments

The Simpler Derivation of Logistic Regression

Logistic regression is one of the most popular ways to fit models for categorical data, especially for binary response data. It is the most important (and probably most used) member of a class of models called generalized linear models. Unlike linear regression, logistic regression can directly predict probabilities (values that are restricted to the (0,1) interval); furthermore, those probabilities are well-calibrated when compared to the probabilities predicted by some other classifiers, such as Naive Bayes. Logistic regression preserves the marginal probabilities of the training data. The coefficients of the model also provide some hint of the relative importance of each input variable.

While you don’t have to know how to derive logistic regression or how to implement it in order to use it, the details of its derivation give important insights into interpreting and troubleshooting the resulting models. Unfortunately, most derivations (like the ones in [Agresti, 1990] or [Hastie, et.al, 2009]) are too terse for easy comprehension. Here, we give a derivation that is less terse (and less general than Agresti’s), and we’ll take the time to point out some details and useful facts that sometimes get lost in the discussion. Continue reading The Simpler Derivation of Logistic Regression

June 26, 2011December 18, 2013John MountStatistics, Statistics To English Translation, TutorialsEstimation, Hoeffding's Inequality, Pseudorandomness, Random Sampling

What is a large enough random sample?

With the well deserved popularity of A/B testing computer scientists are finally becoming practicing statisticians. One part of experiment design that has always been particularly hard to teach is how to pick the size of your sample. The two points that are hard to communicate are that:

The required sample size is essentially independent of the total population size.
The required sample size depends strongly on the strength of the effect you are trying to measure.

These things are only hard to explain because the literature is overly technical (too many buzzwords and too many irrelevant concerns) and these misapprehensions can’t be relieved unless you spend some time addressing the legitimate underlying concerns they are standing in for. As usual explanation requires common ground (moving to shared assumptions) not mere technical bullying.

We will try to work through these assumptions and then discuss proper sample size. Continue reading What is a large enough random sample?

February 3, 2010September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English Translationcustomer value, lognormal distribution, long tail theory, McPhee's Theory of Exposure, median versus mean, power law distribution, Statistics

Living in A Lognormal World

Recently, we had a client come to us with (among other things) the following question:
Who is more valuable, Customer Type A, or Customer Type B?

This client already tracked the net profit and loss generated by every customer who used his services, and had begun to analyze his customers by group. He was especially interested in Customer Type A; his gut instinct told him that Type A customers were quite profitable compared to the others (Type B) and he wanted to back up this feeling with numbers.

He found that, on average, Type A customers generate about $92 profit per month, and Type B customers average about $115 per month (The data and figures that we are using in this discussion aren’t actual client data, of course, but a notional example). He also found that while Type A customers make up about 4% of the customer base, they generate less than 4% of the net profit per month. So Type A customers actually seem to be less profitable than Type B customers. Apparently, our client was mistaken.

Or was he? Continue reading Living in A Lognormal World

December 13, 2009September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English TranslationANOVA, F-test, significance, Statistics, t-test

Statistics to English Translation, Part 2b: Calculating Significance

In the previous installment of the Statistics to English Translation, we discussed the technical meaning of the term ”significant”. In this installment, we look at how significance is calculated. This article will be a little more technically detailed than the last one, but our primary goal is still to help you decipher statements about significance in research papers: statements like “
”.

As in the last article, we will concentrate on situations where we want to test the difference of means. You should read that previous article first, so you are familiar with the terminology that we use in this one.

A pdf version of this current article can be found here.
Continue reading Statistics to English Translation, Part 2b: Calculating Significance

December 4, 2009September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English Translationeffect size, hypothesis testing, significance, Statistics4 Comments

Statistics to English Translation, Part 2a: ’Significant’ Doesn’t Always Mean ’Important’

In this installment of our ongoing Statistics to English Translation series¹, we will look at the technical meaning of the term ”significant”. As you might expect, what it means in statistics is not exactly what it means in everyday language.

As always, a pdf version of this article is available as well. Continue reading Statistics to English Translation, Part 2a: ’Significant’ Doesn’t Always Mean ’Important’

November 3, 2009September 11, 2012Nina ZumelApplications, Expository Writing, Pragmatic Data Science, Pragmatic Machine Learning, Statistics, Statistics To English TranslationAccuracy Measures, Classifiers, Diagnostic Tests, Precision and Recall, ROC Curves, Sensitivity and Specificity, Statistics4 Comments

“I don’t think that means what you think it means;” Statistics to English Translation, Part 1: Accuracy Measures

Scientists, engineers, and statisticians share similar concerns about evaluating the accuracy of their results, but they don’t always talk about it in the same language. This can lead to misunderstandings when reading across disciplines, and the problem is exacerbated when technical work is communicated to and by the popular media.

The “Statistics to English Translation” series is a new set of articles that we will be posting from time to time, as an attempt to bridge the language gaps. Our goal is to increase statistical literacy: we hope that you will find it easier to read and understand the statistical results in research papers, even if you can’t replicate the analyses. We also hope that you will be able to read popular media accounts of statistical and scientific results more critically, and to recognize common misunderstandings when they occur.

The first installment discusses some different accuracy measures that are commonly used in various research communities, and how they are related to each other. There is also a more legible PDF version of the article here.

Continue reading “I don’t think that means what you think it means;” Statistics to English Translation, Part 1: Accuracy Measures