We describe briefly the powerful simulation tefchnique known as
“importance sampling.” Importance sampling is a technique that lets
you use numerical simulation to explore events that, at first look,
appear too rare to be reliably approximated numerically. The correctness
of importance sampling follows almost immediately from the definition
of a change of density. Like most mathematical techniques, importance
sampling brings in its own concerns and controls that were not obvious
in the original problem. To deal with these concerns (like picking
the re-weighting to use) we will largely appeal to the ideas from
“A Tutorial on the Cross-Entropy Method” Pieter-Tjerk de Boer, Dirk P Kroese, Shie Mannor, and Reuven Y Rubinstein, Annals of Operations Research, 2005 vol. 134 (1) pp. 19-67. Read more…
We share our admiration for a set of results called “locality sensitive hashing” by demonstrating a greatly simplified example that exhibits the spirit of the techniques. Read more…
We extend the ideas of from Automatic Differentiation with Scala to include the reverse accumulation. Reverse accumulation is a non-obvious improvement to automatic differentiation that can in many cases vastly speed up calculations of gradients. Read more…
Categories: Applications, Coding, Exciting Techniques, Mathematics, Programming, Tutorials Tags: Automatic Differentiation, Conjugate Gradient, Gradient, Mathematical Bedside Reading, Optimization, Reverse Accumulation, Scala
This article is a worked-out exercise in applying the Scala type system to solve a small scale optimization problem. For this article we supply complete Scala source code (under a GPLv3 license) and some design discussion. Read more…
Categories: Applications, Coding, Computer Science, Exciting Techniques, Mathematics, Programming, Tutorials Tags: Automatic Differentiation, Conjugate Gradient, Dual Numbers, Geometric Median, Numeric Methods, Optimization, Scala, Steiner Tree
We describe the “the local to global principle.” It is a principle used to break algorithmic problem solving into two distinct phases (local criticism followed by global solution) and is an aid both in the design and in the application of algorithms. Instead of giving a formal definition of the principle we quickly define it and discuss a few examples and methods. We have produced both a stand-alone PDF (more legible) and a HTML/blog form (more skimable).
Read more…
What makes a good graph? When faced with a slew of numeric data, graphical visualization can be a more efficient way of getting a feel for the data than going through the rows of a spreadsheet. But do we know if we are getting an accurate or useful picture? How do we pick an effective visualization that neither obscures important details, or drowns us in confusing clutter? In 1968, William Cleveland published a text called The Elements of Graphing Data, inspired by Strunk and White’s classic writing handbook The Elements of Style . The Elements of Graphing Data puts forward Cleveland’s philosophy about how to produce good, clear graphs — not only for presenting one’s experimental results to peers, but also for the purposes of data analysis and exploration. Cleveland’s approach is based on a theory of graphical perception: how well the human perceptual system accomplishes certain tasks involved in reading a graph. For a given data analysis task, the goal is to align the information being presented with the perceptual tasks the viewer accomplishes the best. Read more…
Our first “exciting technique” article is about a statistical language called “R.”
R is a language for statistical analysis available from http://cran.r-project.org/ . The things you can immediately do with it are incredible. You can import a spreadsheet and immediately spot relationships, trend and anomalies. R gives you instant access to top notch visualization methods and sophisticated statistical methods.
Read more…