Win-Vector Blog

In our article How robust is logistic regression? we pointed out some basic yet deep limitations of the traditional full-step Newton-Raphson or Iteratively Reweighted Least Squares methods of solving logistic regression problems (such as in R‘s standard glm() implementation). In fact in the comments we exhibit a well posed data fitting problem that can not be fit using the traditional methods starting at the traditional (0,0) start point. And we cited an example where the traditional methods fail to compute the average from a non-zero start. The question remained: can we prove the standard methods always compute the average correctly if started at zero? It turns out they can, and the proof isn’t as messy as I anticipated. Read more…

Related posts:

1. How robust is logistic regression?
2. The equivalence of logistic regression and maximum entropy models
3. What does a generalized linear model do?
4. My Favorite Graphs
5. Learn Logistic Regression (and beyond)

We extend the ideas of from Automatic Differentiation with Scala to include the reverse accumulation. Reverse accumulation is a non-obvious improvement to automatic differentiation that can in many cases vastly speed up calculations of gradients. Read more…

Related posts:

1. Automatic Differentiation with Scala
2. Automatic Detection of Potential Deadlock
3. Brevity is a Virtue
4. Yet Another Java Linear Programming Library
5. My Favorite Graphs

This article is a worked-out exercise in applying the Scala type system to solve a small scale optimization problem. For this article we supply complete Scala source code (under a GPLv3 license) and some design discussion. Read more…

Related posts:

1. Gradients via Reverse Accumulation
2. Programmers Should Know R
3. Automatic Detection of Potential Deadlock
4. What to do when you run out of memory
5. R examine objects tutorial