The Local to Global Principle

John Mount¹

Date: November 11, 2009

Abstract:

Contents

• Introduction
• The Examples
  • Web Page Link Analysis
  • Natural Language Processing
  • Machine Learning

• Some Methods
  • Local Methods
  • Globalization Methods

• Conclusion
• Bibliography
• Acknowledgement

Introduction

A common vain hope of computer scientists and algorithm designers is that a domain expert has already “boiled down” a problem to a precise, but unsolved, algorithmic core. On this point the mathematician Gian-Carlo Rota wrote:

One of the rarest mathematical talents is the talent for applied mathematics, for picking out of a maze of experimental data the two or three parameters that are relevant, and to discard all other data. This talent is rare. It is taught only at the shop level.[Rot97, ``A Mathematician's Gossip'']

We describe a useful tool for designing algorithmic applications and solutions which we call “the local to global principle.” The local to global principle is the method of deriving applications and solutions by specifying “local” (and deliberately myopic) heuristics, critiques and methods followed by using a powerful general method to “globalize” this specification into a complete solution.

There are many important problem solving prescriptions and methods of thought already systematically described and taught:

• Bacon’s “New Organon” and Mill’s principles of inductive logic.[Mil02]
• Feynman’s genius method.[Rot97, ``Ten Lessons I Wish I Had Been Taught'']
• Reductionism (top down and bottom up).
• Divide and conquer.[CLRS09]
• Forward deduction, backwards induction.
• Root Cause Analysis.
• Polya’s heuristic and conjecture and prove patterns [Pol71,Pol54a,Pol54b]
• Doron Zeilberger’s “Method of Undetermined Generalization and Specialization.” [Zei95]
• Zbigniew Michalewicz and David B. Fogel’s presentation of evolutionary algorithms.[MF00]

The local to global principle is more of an organizational pattern than “computer aided technique” as no one specific species of software or family of notation is required.

The local to global principle can be identified in a number of previous important applications, but it is not currently an identified principle.² The principle is very general, so any succinct description of it is going to be painfully vague. Instead, we explain the principle by discussing some example applications and methods. For each of our example applications we deliberately use a different globalization technique. The effective algorithmist or practitioner must in fact come to each problem already familiar with a reasonably large set of already known local and global techniques, so we conclude with some appropriate fields of study and preparation.

The local to global principle is divided into two parts: local encoding of the problem followed by a globalization step that uses the encoding. The guiding feature of local encodings is that they are usually easy to compute from the data at hand. Any extension that looks like enumeration, search or optimization is best left to the global step. The local step is essentially the translation of your problem into an abstract language that is ready for the globalization step. In contrast globalization methods are often “off the shelf” in that once you abstract and encode the particulars of your problem you can look for pre-existing useful methods or software to finish your solution. The idea of globalization is to find a best overall or global compromise between competing local criteria. The local step does not so much have to avoid conflicts but instead “price them.” There is also an important trade-off that sophisticated local techniques allow the use of simpler globalization methods and more powerful globalization methods allow the use of simpler local techniques.

The Examples

To demonstrate the breadth of the local to global principle we choose a diverse collection of example applications: web page link analysis, natural language processing and machine learning. For each example application we will set up the problem, introduce a reasonable set of local criteria and pick an appropriate globalization technique. We will favor finishing each example without describing the globalization technique in detail, as this would distract from our point and is best left to the given references. These examples are previously solved problems, our contribution is demonstrating the shared underlying principle.

Web Page Link Analysis

For our first example application we demonstrate web page link analysis in the form of the famous PageRank score.[PBMW98]

One of the many good ideas leading up to the early Google search engine was the design of a non-text based measure of importance or interestingness of web pages. A search engine that could fold “interestingness” or popularity into its notion of relevance could better sort important pages into the search user’s view. When the web got so large that there were many pages that were exact matches to any common user query popularity became a critical consideration. A link based notion of popularity exploits what is important about the web (the link structure, for example see [Kle97]) and avoids having to depend on a lot of natural language understanding technology. This technique also uses authority outside of the given page, so has some hope at being resistant (though not immune) to web-spam.

Taken all at once, the task of designing a score of page importance is a daunting task. However, by working in stages (as the local to global principle prescribes) we can quickly derive interesting scores including the famous PageRank score. We start with the idea that popularity (or the amount of web traffic a page receives) is (loosely) correlated with importance. So for our first approximation step we decide to try to estimate popularity (or web traffic) and use this estimate as our importance score. Accurately estimating web traffic is itself a hard problem and a big industry (just a few of the major companies involved in this are: Google/Urchin, Quantcast, Nielsen, comScore, Alexa, Hitwise and LookSmart). For our second approximation step we are going to try and estimate popularity from the link structure⁴ of the web (using no other measurements or historic data) and use this as our score. This link based estimate is unlikely to completely reproduce real web surfing patterns, but it is very interesting in its own right and has been proven in the market to be a useful score.

Now the problem is to try to estimate the popularity of a web page from the link structure of the web. We claim: we can generate a useful (but not necessarily accurate) estimate of web traffic from the web’s link structure alone. Consider Figure 1 where we have a universe of three web pages A,B and C that link to each other in the pattern illustrated by what is called a graph⁵

Figure 1: A set of Mutually Linked Web Pages

In Figure 1 we can consider each link to another page as evidence the other page is interesting or popular. One idea is to simulate a very simple web surfer who clicks on the links on a page uniformly at random. This is called “the random surfer model” and even a model this simple allows us to read some useful information from the link structure of the web. For instance, we could ask what fraction of their time the random surfer spends on each web page, with an eye to the idea that the pages the random surfer visits more often are the more important ones. Let denote the proportion of time the random web surfer spends on page A (and define and similarly). While we do not know any of
or we can derive some relationships between them by inspecting the link graph:

The first equation is just reading from the graph that: all visits on page-A must come from pages B and C, half of the visitors on page-B continue on to A and all of the visitors on page-C continue on to A. The second and third equations are the appropriate summaries of how traffic is routed to pages B and C. We can insist that
as we want these numbers to represent the fraction of time the random web surfer spends on each page. A more sophisticated model would add more features⁶ to get a more useful result.

It turns out we have already encoded enough local rules to completely determine
and . In this example application an algorithmist already familiar with linear algebra [Str76] would recognize these local conditions as “a system of linear equations.” Solving even web-scale systems of linear systems is considered easy with modern techniques and modern computers. For our small example example the solution is:
,
, and
. The role of the local steps was to reduce a new problem (estimating the importance or popularity of web page from the link structure) to something with its already known known techniques (like solving a linear system as illustrated in Figure 2).

Figure 2: Linear Algebra Solution: As Taught in School

So page-A is the most important page by the PageRank measure.

In this example application the local step was setting up the system of linear equalities (which are easy to derive from the web link graph) and the global step was solving the entire system for the final scores (which were not obvious). You spend most of your time encoding the problem and then use a known technique (in this case solving a linear system) to finish the solution.

Natural Language Processing

Our next example application is natural language processing [Cha96,Cha97]. Speech recognition (the alignment or transcription of recognized intelligible segments of sound to written text) is an important problem in natural language processing. An example problem is the need to find the most likely text matching a sequence of sounds such as is shown in Figure 3.

Figure 3: A Sequence of Sounds

Consider Figure 4 (which shows a bad transcription) and Figure 5 (which shows a good transcription).

Figure 4: A Bad Transcription

Figure 5: A Good Transcription

Our claim: we can (given access to training data, and this is the age of data [HNP09]) solve this problem with a local step that is a set of simple criticisms of proposed transcriptions. A good starting point is a database of previous sounds to text transcriptions. This database allows the construction of a series of tables that give the historic frequency (or probability) of all of the following:

• Prior probability of each sound
• Probability of each sound given the immediately previous sound
• Prior probability of each word
• Probability of each word given the immediately previous word
• Which combinations of word fragments are legitimate words
• Probability of each sound being assigned to each word fragment (syllables, phonemes and so on).

These tables encode a “speech model” (the rules involving sounds only), a language model (the rules involving text or words only) and the linkage between the two models. These models are deliberately simple in that they capture only local interactions (like probability of a word given the word before it) but no long range interactions (like subject predicate agreement).

Each box, nested box and arrow on our diagram represents one possible local critique. For each item in our diagram (again, the boxes and arrows) we can use our tables to assign a goodness or plausibility score. For instance bad word to word transitions (like “won”
“won”) will be rare in our historic tables so, just looking up probabilities from the tables (or, better, using the logarithms of probabilities) gives as a “plausibility score” that prefers known patterns of language. Then a score for the overall transcription can be derived by multiplying all of the local scores together. These local scores (though simple) already have encoded enough evidence to prefer the good transcription to the bad transcription without requiring any deep knowledge of speech, text or the meaning of the text. This is because the bad transcription has a series of obvious flaws such as: unlikely sound to word fragment assignments and unlikely word to word transitions.

Figure 6: Naively Extending a Partial Transcription

For example consider Figure 6 where a naive solver is in the process of considering selecting the word “one” as the third word to fill in. The only local critiques they need to consider are:

• how likely the word “one” is in general (call this )
• how likely the word “one” is to follow the word “nine” (call this
  )
• how likely the letter sequence “o” is given the sound “w” (call this
  )
• how likely the letter sequence “ne” is given the sound “n” (call this
     n ).

So the local plausibility of the fill-in word “one” is:
. We will call this the critique of “one” in position 3 and write as
where is the word known to be in position 2. Similarly we can generate all of the possible critiques ,
,
,
and the overall criticize of a sequence
:
from our pre-computed tables of probabilities. Notice for all of these critiques only the immediately previous word and the nearby sounds were used to determine the plausibility of the word we are attempting to fit in. Instead of using these critiques to directly fill in a possible solution (or using search) we will package up these critiques (in the form of the ) and pass them on to a powerful separate globalization step called Dynamic Programming [Bel57].

The globalization or finding of a best overall transcription is not trivial even though our score is simple. This is because the overall best sequence could depend on clever non-local fill-ins (like deliberately picking a less likely first word to allow a later favored transition to a fantastically good third word). Dynamic Programing does not fill in the transcription from left to right, but instead uses a table of scores derived from the left to right arrows and the . In our example Dynamic Programming consists of building a table of information as shown in Figure 7. Let represent the word position we are working looking at (so ranges from 1 to 4) and let be a variable that ranges over every word in the dictionary. Our table is indexed by and and when filled in stores what the highest “plausibility score” of a partial sequence of words where words 1 through have been filled in and the -th word is .

Figure 7: Dynamic Programming: Back Chaining in for a Solution

If we already had this magic table we could find a best possible sequence by “back chaining.” We start by finding a fourth word ( ) such that is maximal (in this case “one”). We then find a best third word ( ) by enumerating all words and picking such that
. We continue back until we had found words and to get a complete best sequence. Notice that we work from right to left (backwards) and except for the starting step we pick each word to match the calculation we are trying to un-roll, not to be the maximal entry in the column. For instance we pick
even though it does not have a the highest score, but because
is the maximal complete chain.

Of course, we don’t start with the table already filled in- so we need a procedure to build it. This procedure is the heart of the Dynamic Programming method (for more examples see: “Introduction to Algorithms” [CLRS09]). Notice that can be filled in for all just by plugging in words and computing the critiques (i.e.
). Once all the are filled in we can fill in the the with the general (and slightly trickier) formula:

as we illustrate for in Figure 8.

Figure 8: Dynamic Programming: Building the Table

The magic of the Dynamic Programing technique is: by being careful to not store too much in the table we avoid an explosion in record keeping that would render the method inefficient. Dynamic Programming exploits the small dependence structure encoded in (each box in our diagram depending on only a few arrows) and as we have shown can find “clever” solutions (such as taking a sub-optimal first word to get better transitions into preferred later words). For those who want more detail on solving this problem we recommend [Cha96] (as our goal is not to fully explain Dynamic Programming, but to demonstrate how it could be applied to the transcription problem as a pre-packaged globalizer).

In this example the local step was the graph link based critiques and the globalization step was Dynamic Programming. The separation of concerns from the local scoring to the globalizing step is a strength of the local to global principle.

Machine Learning

Our final example application is machine learning. Machine learning is loosely defined as computer programs that adapt or learn from data. Thomas Mitchell helps distinguish this activity as a specialty of artificial intelligence that concentrates on “well-posed learning problems.” [Mit97] Trevor Hastie, Robert Tibshirani, Jerome Friedman emphasize the relation to statistics (versus more traditional symbolic AI) [TH09]. A simple demonstration can be found in [Mou09b].

Machine learning is perhaps the strongest example of the local to global principle and is inspired by the work of Kristin P. Bennett and Emilio Parrado-Hernandez [BPH06]. In hindsight many machine learning algorithms (each of which has had a turn at being “the most exciting breakthrough ever” for a while) can be seen as the pairing of a performance criterion (which we call a local criterion as it applies to one specific set of parameter values at a time) and an optimization method (what we have been calling the globalization step). The work of Bennett and Parrado-Hernandez calls this distinction out and shows how it is not productive to present machine learning systems as unique named monolithic units, but instead to consider how to break them into an objective function and an optimizer. This allows both choice of better optimizers (such as replacing the inferior method of gradient descent method wherever it occurs) and for explicit control of important concepts such as hypothesis regularization and control of over-fitting (which some algorithms claim to achieve by deliberately using early exit from a an inferior optimizer).

At a “30,000 feet level” we can build a table of common machine learning techniques and name what is commonly used to implement their local and global steps. When a machine learning algorithm is defined by what conditions are meant to be true at the optimum we are no longer bound by details of the original implementation and can examine fix and improve the components.⁷ Table 1 is a crude summary of a wide selection for machine learning algorithms that may be more likely to offend everybody than just offend somebody. But this is also the point: it is the algorithmist’s job to think fluidly (beyond given names and provenances) and to invent scaffolding to convert partial analogies into practical correspondences.

This table is a necessarily crude summary. For example: notice that several known techniques can not even be distinguished from each other by the local and global columns of the table.

There are a few points we would like to make. Back propagation was considered unique to Neural Nets for quite a while because it was so entwined with the technique it was not recognized as the simple application of Automatic Differentiation [RC96] that it is. Support Vector Machines (SVM) are remarkable for their uniform very good choice of component methods (maximum L1 margin objective regularization, Kernel Methods [STC04] and sophisticated optimization methods [Joa06]). Many of the machine learning methods that SVM outperforms become again competitive when they adopt some of SVM’s technologies (especially using kernel methods to produce synthetic features).

Beyond these points we invoke a “globalizers are pre-packaged” principle and leave the discussion of machine learning and optimization to our reference: [BPH06]. In this example the local step is a per-example score or penalty and the globalization step is optimization.

Some Methods

The application of the local to global principle is similar to the Feynman “genius method.” Feynman’s method is to always have in mind a list of problems and a list of solution methods. The genius step is: anytime you see a new problem or a new solution method to immediately try it against every item from the complementary list. [Rot97, ``Ten Lessons I Wish I Had Been Taught''] This deliberate retention and activity greatly increases your problem solving ability. The power of the local to global principle is itself proportional to the number of local methods times the number of globalization strategies. Of course, to even start: the practitioner must already have available a number of candidate local and globalization methods. We list some methods and some guidance on variation and invention.

Local Methods

Good sources of ideas and analogies for local methods include:

• Introduce a Graph Structure

  A graph structure is a network of nodes connected by edges. Use of graphs was demonstrated both in the natural language processing and web page link analysis examples. We can dress up how we solved these problems and say we used a “Hidden Markov Model”, but the real power was we encoded our problem in a simple graph. Some problems (especially those from logic or those involving time) are essentially solved once they are translated out of their original form and into graph notation (for an example see: [Mou00]).

• Appeal to Physical Conservation Laws

  A good example physical law is Kirchhoff’s law or conservation of flow. All of the web page link analysis’s equations were derived by saying that the attention of at node is essentially the sum of attentions from other nodes (more sophisticated versions of the analysis actually do create and destroy flow, but they do it in a principled way).

• Encode the Problem into an Objective Function

  This method is essentially your declaration that you intend to use an optimizer for the globalization step. In operations research this specific technique has long been the practice (with no disrespect: a very productive part of operations research has been translating different problems into linear programs so the simplex method can be applied, for an example see [Mou09a]).

• Gradient Like Computations

  Includes Gradients, Secants, Lagrangians and other ideas from calculus. Gradients can drive optimizer based globalizers and techniques like Lagrangians are often powerful enough use mere inspection as the globalization step.

• Violation Driven Updates

  This method is particularly effective when your problem is not amenable to continuous optimization. A good example is the Lin-Kernighan heuristic for solving the traveling salesman problem.[LK73] This heuristic looks at subsets of the problem and suggests improving “surgeries” (until no more such improvements are possible).

• Introduction of Symbols

  Often, as with the web page link analysis example, you can not specify specific values for the unknowns, but you can specify relationships. You often can then solve for the symbols or introduce additional conditions and use an optimizer to complete the solution (see for example the maximum entropy method as described in [Ski88]).

• Over Specification

  If we anticipate using a global step like search, enumeration, summation or integration then over specification is a good local idea.

  For example: consider computing the probability that a fair count flipped 10 times comes up with heads exactly 3 times. The easiest way to perform this calculation is to specify exactly which 3 coins come up heads (the local over-specification) and then sum over all choices of 3 out of 10 coins (the global step). In mathematical notation this is:

  exactly 3 heads out of 10 flips

  or just under 12%.

• Under Specification

  One of the core principles of Dynamic Programming is to forget as much as possible about partial solutions, keeping only partial solution cost and just enough information to extend partial solutions. If you anticipate using something like Dynamic Programming as your globalization step then your goal should be to under specify.

• Tables

  A key step of the natural language processing example was the use of tables of past experience to determine which sounds likely corresponded to which words, which words likely followed each other (and so on). Encoding domain knowledge or expertise as probability tables is a very effective problem solving strategy (especially if the globalization strategy is going to be search or Dynamic Programming). In natural language processing examples tables and statistics are much easier to manage than comprehensive rules or grammars.

• Set up as Ranking or Machine Learning Problem

  This tactic is especially appropriate if your solution success metric is counts, frequencies or probabilities (instead of having to always be correct or always be optimal).

Globalization Methods

The universe of possible globalization methods is very diverse (in particular globalization is not always optimization).

• Search / Enumeration

  Search can be slow, but it is always an option to consider. If your problem translates naturally into a graph structure or your solutions are naturally seen as being composed of small pieces search should be considered. One of the big advantages using the local phase to formally encode your problem’s structure and putting search off to the global phase is: you can use advanced search techniques. Once you are freed from your specific problem details it becomes much easier to consider search techniques like branch and bound, A*, game theoretic search and general speed-up techniques like hashing and caching.

• Dynamic Programming

  If your problem has a bit more structure (in that partial solutions summarize and compose easily) then you can likely replace search with Dynamic Programming. The advantage is that Dynamic Programming typically offers an incredible speed up when compared to search.

• Optimization

  If your problem is continuous (involves numbers instead of discrete or categorical decisions), can be encoded as a reasonable objective function (linear, positive definite quadratic) and has reasonable constraints (linear or convex) then you can immediately apply an optimizer as your globalization step. Typical optimization methods include: conjugate gradient, Newton methods, quasi Newton methods, linear programming and quadratic programming.

• Combinatorial Optimization

  If your problem includes a “discrete variables” (that is variables that take on one of fixed set of values instead of values from a numeric range) then you may not be able to apply standard optimization techniques. At this point you may want to use more expensive combinatorial optimization techniques like integer linear programing or constraint satisfaction.

• Fixed Point Methods / Iteration

  Fixed point methods are based on the idea: “incrementally improve until there is no incremental improvement possible.” If the problem is continuous this is similar to steepest descent. If the problem is discrete then this is similar the Lin-Kernighan heuristic.

• Linear Algebra

  The web page link analysis and optimization examples were essentially solved once we reduced them to linear algebra. If you can write your problem as a linear relationship between unknowns or as the fixed-point of a linear operator (i.e. an such that ) then you can immediately use linear algebra to solve the problem at very large scale (e.g. web scale).

• Sampling / Problem Kernels

  A very successful line of attack on large problems is to reduce to a smaller problem containing most of the essential difficulty. David Karger has produced a number of effective algorithms for graph cuts and flows using a theory of sampling [Kar98]. Rod Downey and M. Fellows have demonstrated an effective theory of “problem kernels” that finds solution by focusing on smaller sub-problems (on which we can afford to use more expensive procedures).[DF98]

• Amortized Analysis / Economic Mechanism Methods

  Daniel Sleator and Robert Tarjan’s ideas of amortized analysis [ST85] allow approximation schemes similar to problem kernels. The method is to approximately optimize by pairing a bunch of unavoidable large penalties (conditions we can’t meet) with some accounting credits (say bonuses from other conditions we are meeting very well). We then isolate these paired items and optimize the rest of the problem exactly. The technique often works by showing the approximation can not be too bad because, due to the pairing of large penalties to good credits, there can not be too many large penalties. An informal example is: if it is impossible to pick someplace where all of an office will eat for lunch, perhaps you can solve the problem by paying one person to accept a restaurant they do not like (if the removal of their objection opens up a venue that is acceptable to everybody else).

• Relaxation / Homotopic methods

  These methods involve changing hard constraints to soft penalties (so allowing the constraints to be violated, but at a slowly increasing cost). After such a relaxation the homotopic (or continuous deformation) method is to increase the cost of violation and re-solve to try and get a trajectory of better and better nearly acceptable solutions that point to a possible overall solution.

Conclusion

The purpose of this article has been to make more visible an idea we call the local to global principle. This principle is an organizing tool useful both in designing and analyzing a wide variety of applications. Essentially the whole point of this writeup is to set up enough framework to quickly write down a table of advice such as Table 2 (and for such a table to mean something).

The principle is not universal; not everything can be fit into such a table. For example the local to global decoupling is not a feature of the famous EM algorithm [DLR77], which depends on mixing predictions and corrections.

To conclude: the recipe is as follows. If you come to a problem with a large shopping bag of possible ways to build local criteria and powerful globalization procedures then you stand a very good chance of solving the problem quickly. Also, if you keep the local to global principle in mind you are more likely to identify and retain potential local tricks and globalizers when you see them and thus have a larger more nimble set of tools available to solve problems when the time comes.

Bibliography

AC06

Nir Ailon and Bernard Chazelle, Approximate nearest neighbors and the fast johnson-lindenstrauss transform, STOC (2006).

AI06

Alexandr Andoni and Piotr Indyk, Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions.

BD02

Avrim Blum and John Dunagan, Smoothed analysis of the perceptron algorithm for linear programming, SODA (2002), 11.

Bel57

Richard Bellman, Dynamic programming, Princeton University Press, 1957.

BF97

Leo Breiman and Jerome H Friedman, Predicting multivariate responses in multiple linear regression, Journal of the Royal Statistical Society, Series B (Methodological) 59 (1997), no. 1, 3-54.

BFSO84

Leo Breiman, Jerome Friedman, Charles J. Stone, and R. A. Olshen, Classification and regression trees, Chapman & Hall/CRC, January 1984.

BNJ03

David M Blei, Andrew Y Ng, and Michael I Jordan, Latent dirichlet allocation, Journal of Machine Learning Research 3 (2003), 993-1022.

BPH06

Kristin P. Bennett and Emilio Parrado-Hernandez, The interplay of optimization and machine learning research, Journal of Machine Learning Research 7 (2006), 1265-1281.

Bre00

Leo Breiman, Special invited paper. additive logistic regression: A statistical view of boosting: Discussion, Ann. Statist. 28 (2000), no. 2, 374-377.

BRS91

R Beigel, N Reingold, and D Spielman, The perceptron strikes back, Structure in Complexity Theory Conference 6 (1991), 286-291.

Cha96

Eugene Charniak, Statistical language learning, MIT Press, 1996.

Cha97

to3em, Statistial techniques for natural language parsing, AI Magazine 18 (1997), no. 4, 33-44.

CLRS09

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, Introduction to algorithms, MIT Press, 2009.

CSS02

Michael Collins, Robert E Schapire, and Yoram Singer, Logistic regression, adaboost and bregman distances, Machine Learning 48 (2002), no. 1/2/3, 30.

CV05

Rudi Cilibrasi and Paul M.B Vitanyi, Clustering by compression, IEEE Transactions on Information Theory 51 (2005), no. 4, 1523-1545.

DF98

Rod G. Downey and M. R. Fellows, Parameterized complexity, Monographs in Computer Science, Springer, November 1998.

DLR77

A P Dempster, N M Laird, and D B Rubin, Maximum likelihood from incomplete data via the em algorithm, Journal of the Royal Statistical Society, Series B (Methodological) 39 (1977), no. 1, 1-38.

Fis36

Ronald A Fisher, The use of multiple measurements in taxonomic problems, Annals of Eugenics 7 (1936), 179-188.

FS99

Yoav Freund and Robert E Schapire, A short introduction to boosting, Journal of Japanese Society for Artificial Intelligence 14 (1999), no. 5, 771-780.

GD04

Peter D Grunwald and A Philip Dawid, Game theory, maximum entropy, minimum discrepancy and robust bayesian decision theory, Ann. Statist. 32 (2004), no. 4, 1367-1433.

Gru00

PD Grunwald, Maximum entropy and the glasses you are looking through, Conference on Uncertainty in Artificial Intelligence (2000), 238-246.

HNP09

Alon Halevy, Peter Norvig, and Fernando Pereira, The unreasonable effectiveness of data, IEEE Intellegent Systems (2009).

Hus99

Dirk Husmeier, Neural networks for conditional probability estimation, Springer, 1999.

IM99

Piotr Indyk and Rajeev Motwani, Approximate nearest neighbors: Towards removing the curse of dimensionality.

Joa98

Thorsten Joachims, Making large-scale svm learning practical, Advances in Kernel Methods – Support Vector Learning (1998).

Joa06

to3em, Training linear svms in linear time, KDD (2006).

Kar98

David R Karger, Randomization in graph optimization problems: A survey, Optima: Mathematical Programming Society Newsletter 58 (1998).

KCVM04

Trausti Kristjansson, Aron Culotta, Paul Viola, and Andrew Kachites McCallum, Interactive information extraction with constrained conditional random fields, AAAI (2004).

Kle97

Jon M Kleinberg, Authoritative souces in a hyperlinked environment, ACM SIAM Symposium on Discrete Algorithms (1997).

Kom08

Paul Komarek, Logistic regression for data mining and high-dimensional classification, CMU CS Thesis (2008), 138.

KWA95

J Kivinen, Manfred K Warmuth, and P Auer, The perceptron algorithm v.s. winnowo: Linear v.s. logarithmic mistake bounds when few input variables are relevant, COLT (1995), 289-296.

Lew98

David D Lewis, Naive (bayes) at forty: The independence assumption in information retrieval, find journal (1998).

LK73

S Lin and BW Kernighan, An effective heuristic algorithm for the traveling-salesman problem, Operations Research (1973), 498-516.

Mar61

M E Maron, Automatic indexing: An experimental inquiry, RAND Technical Report (1961), 404-417.

MF00

Zbigniew Michalewicz and David B. Fogel, How to solve it: Modern heuristics, Springer, 2000.

Mil02

John Stuart Mill, A system of logic, University Press of the Pacific, 2002.

Mit97

Thomas Mitchell, Machine learning, McGraw-Hill, 1997.

MK00

M E Maron and J L Kuhns, On relevance, probabilistic indexing and information retrieval, 1960 (2000), 1-29.

Mou00

John A Mount, Automatic detection of potential deadlock, Dr. Dobbs Journal (2000).

Mou09a

John Mount, Automatic generation and testing of un-rolls for profitable technical trades, http://www.win-vector.com/blog/2007/10/paper-on-stock-trading/, 2009.

Mou09b

to3em, A demonstration of data mining, http://www.win-vector.com/blog/2009/08/a-demonstration-of-data-mining/, 2009.

PBMW98

Lawrence Page, Sergey Brin, Rajeev Motwani, and Tery Winograd, The pagerank citation ranking: Bringing order to the web, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.31.1768 (1998).

Pol54a

G. Polya, Induction and analogy in mathematics, Princeton University Press, 1954.

Pol54b

to3em, Patterns of plausible inference, Princeton University Press, 1954.

Pol71

to3em, How to solve it, Princeton University Press, November 1971.

RC96

Louis B Rall and George F Corliss, An introduction to automatic differentiation, SIAM: Computational Differentiation: Techniques, Applications and Tools (1996), 1-18.

Rot97

Gian-Carlo Rota, Indiscrete thoughts, Birkhauser, 1997.

Ski88

John Skilling, The axioms of maximum entropy, Maximum Entropy and Bayesian Methods in Science and Engineering 1 (1988), no. 173-187.

ST85

Daniel Dominic Sleator and Robert Endre Tarjan, Amortized efficiency of list update and paging rules, Communications of the ACM 28 (1985), no. 2.

STC00

Jown Shawe-Taylor and Nello Cristianini, Support vector machines, Cambridge University Press, 2000.

STC04

to3em, Kernel methods for pattern analysis, Cambridge University Press, 2004.

Str76

Gilbert Strang, Linear algebra and its applications, Academic Press, Inc., 1976.

TH09

Jerome Friedman Trevor Hastie, Robert Tibshirani, The elements of statistical learning: Data mining, inference and prediction, Springer, 2009.

TTV08

Luca Trevisan, Madhur Tulsiani, and Salil Vadhan, Regularity, boosting, and efficiently simulating every high-entropy distribution, Electronic Colloquium on Computational Complexity (2008), 18.

Zei95

Doron Zeilberger, The method of undetermined generalization and specialization illustrated with fred galvin’s amazing proof of the dinitz conjecture, http://arxiv.org/abs/math/9506215, 1995.

Acknowledgement

A thank you to readers who supplied help and comments on earlier drafts.

Footnotes

… Mount¹

email: mailto:jmount@win-vector.com web: http://www.win-vector.com/

… principle.²

The pre-existing practice that comes cloesest to the local o global principle is found in operations research where encoding a problem to be solved by an optimizer is a central technique. We claim the natural statement of the local to global principle is more general than always encoding constraints for a particular optimizer (in particular globalization is not always optimization).

… structure⁴

By “link structure” we mean which web pages link to which other web pages.

… graph⁵

Remember, a graph is diagram consisting of nodes and edges (here depicted as arrows).

… features⁶

For example the model could account for:

• surfers entering and leaving the model
• link odds that vary where they are on a page
• surfers staying on a page proportional to how much text is on the page
• matching known traffic and click behavior where we have such data.

For simplicity we will just stick with the example given example.

… components.⁷

When a system is named and defined as an exact set of procedures the system can, by definition, not be improved. This is because with any change in procedure we have a new system that no longer matches the original definition and therefore requires a new name.

Related posts:

1. Automatic Differentiation with Scala
2. A Demonstration of Data Mining
3. Should your mom use Google search?

Also in (less legible) HTML:

Betting Best-Of Series

John Mount¹

Date: May 27, 2008

Introduction

We use the United States’ Major League Baseball World Series to demonstrate some of the “arbitrage arguments”²used in mathematical finance. This problem is a classic finance puzzle question and is an interesting introduction to some exciting techniques.

“Arbitrage” is the simultaneous buying and selling of a commodity, usually in multiple markets, that returns a risk-free profit. An example would be finding a market where apples are selling for $1 and another where they are selling for $2, and then simultaneously executing a purchase order in the cheap market and a sales order in the expensive market (assuming no significant shipping risks or costs). Typically “arbitrage opportunities” are too much to hope for and to make a profit you must add value, loan money, hold inventory or take on risk. This is just the mathematical finance way of saying “there is no free lunch,” but a number of surprising facts about markets can be proven using this principle.

The Problem

Figure 1: World Series Tree (Win over Loss)

Consider a “first to win ” contest like the United States’ Major League Baseball World Series. The World Series is a “first to win four” contest (sometimes called “best of seven”) where a number of games are played between two teams and the first team to win four games is declared the series winner. Ignoring the possibility of ties this process can take from four to seven games. We can (as in Figure 1) lay out all of the possibilities in to a picture that moves from left to right and then moves up when the first team wins and down when the second team wins.

Any sequence of games is represented by a path through this diagram (starting at the left) that reaches a node with no exit. At each node we have marked in the wins for each team (Team One on top, Team Two on the bottom). The nodes where one team has won four games are where the series ends.

The “arbitrage question” is:

If you had access to a bookie who was willing to take an even-payoff bet (on either side) in each game of the World Series, can you design a schedule of bets on games that simulates an even-payoff one dollar bet on the outcome of the entire World Series?

That is: you wish to make a bet that pays you $1 if your team wins the World Series and costs you $1 if your team is defeated. You can not find anybody to take such a bet- but you have found a bookie who makes the incredibly generous offer of taking bets (at even pay-off) on each and every game in the series. Can you, without any additional risk, simulate a World Series bet by making a series of per-game bets with this bookie?

The Answer

The answer turns out to be that you can simulate a world-series bet. The reason for hope is that both types of bets (the even-payoff bets on games and an even-payoff bet on the whole series) are expressing the same underlying belief: that both teams have an exactly equal chance of winning. The teams may or may not have the equal chances of winning- but offering to take bets on both sides at equal pay-off is equivalent expressing just such a belief.

The principle that the probability you are willing to take bets at expresses your subjective probabilities is a principle goes back to Bruno de Finetti and is the most basic “arbitrage style” argument. The principle is simple but it is useful warm-up to think about. Under the assumption that you are “rational” (in the economic sense, which just means you are not giving money away without a reason) and if denotes your personal estimate of the probability of your team winning then if you are willing to bet $1 that your team wins at even payoff (meaning you collect $1 if your team wins pay $1 if your team loses) then for this bet to make economic sense you must have:

which means
.

Similarly if you are willing (for purely economic reasons) to take the other side of the bet at the same even-payoff bet on the other side (reversing the rolls of winning and losing) then it must be true that
. We then have our conclusion: from an economic point of view you should be willing to take either side of a fair-payoff bet only if your estimate of the probability of winning is .

Figure 2: World Series With Some Values Filled In

We now return to the World Series diagram. If we bet on individual games (instead of making one bet on the whole series) then at each node in the diagram we expect to have some sort of net winnings or net losses. For example at each node where our team has won four games we should be holding , so we will label these nodes with . Similarly at each node where the opposing team has won for games we expect to have lost exactly so we label those nodes with . Our task is to figure out the amount bet at each node and our net holdings at each node. If we can find a schedule of bet amounts that leads to the correct outcomes at the end of the world series and starts with an initial net holdings of $0 then we have solved the problem.

If we look at Figure 2 we see there the node corresponding to each team having won 3 games points to two nodes we know the values of (the World Series ending with either team the winner). We can use the fact that this node points only to nodes with known net holdings to figure out both the bet that must be made at this node and the net holdings this node should have at this point in World Series.

Let be the (unknown) net holdings we have at this node and be the (unknown) amount we bet then to complete the World Series bet we must have the following:

This is enough to notice that (your holdings) must be the average of the two outcomes pointed to and (your bet) must be one half of the difference of the two outcomes. So the “each team has won three games” node (near the very right end of the diagram) should have a net holding of and we should bet . Filling in this node with the net holdings ($0) now means that there are other nodes that point only to nodes with filled-in net holdings. We can, in fact, repeat this process of filling in each node with unknown net holdings with the average of the two known nodes it points to until we complete the diagram as in Figure 3.

Figure 3: World Series All Values Filled In

In the completed figure each node is filled in with the net holdings required to implement our betting schedule. We can see that a diagram like this can always be filled out to completion by looking at the diagram as having layers like an onion and noticing that we start with the right most nodes filled in (they are the nodes where the world series ends). It is obvious that we can fill out every node in the layer of nodes just inside the outer layer if we start at the right most such node and work back. Every layer can be completed one after another until we get to the inner most layer which is just the starting node. To implement the betting strategy, we keep track of where we are in the diagram and always bet one half of the difference between the net holdings of the two nodes pointed to by the node we are at.

If the first node of the diagram was marked with a value other than zero it would mean that the world-series has a net bias for the first team or the second. Since the rules are symmetric this would be a nonsense conclusion, so we can be sure that all of the even-score nodes must be valued at zero.

The filling in of blanks using values ahead of them (from the future) is the heart of the Binomial Pricing Theory for options is based on a very deep idea called Dynamic Programming. The idea is that you may not know which future you will experience- but you may know the valuation of every possible future. It is an amazing fact that even without introducing probabilities or probability estimates of which future you will experience just knowing the value of every possible future is enough to compute the value of a bet in the present time. In our example: you may not know ahead of time the final scores of the world series, but you do know value of a world series bet for each possible ending score.

What is the analogy?

From a finance or betting point of view the problem is solved- we have procedures for building the betting schedule and we have the schedule itself. From a mathematician’s point of view we have only just started- we have some procedures and relations but what are they an analogy of?

Naively one might think that they should bet around one fourth of their desired outcome in each game to simulate a best of four series. However to simulate a total World Series bet of we use an initial bet of
in our schedule. This is almost a third of our desired total bet. This gets us wondering: what is the general form of this first bet?

Let
bet denote the amount of the first bet in the simulation of a “best of ” bet. If we compute
bet (by constructing betting schedules as above) for many values of we see that
bet seems to shrink slower than In fact it seems to shrink at a rate of around
. Even more intriguing if you plot
betbet it converges (very slowly) to
. We can conjecture that for very large the initial bet is:
where is the famous ratio of the ratio of the length of the circumference of a circle to the the length of the diameter of the same circle.

Now
is much larger that (as gets large). So the scheme says to bet a fairly large amount of your budget on the first game, and that winning the first bet is worth a bit more than you would expect (it takes you more than one th of the way to victory).

Figure 4: Weighted Paths

What is going on? We can again apply an arbitrage or de Finetti style argument and say since the whole game was “fair” with expected pay-off zero then we can relate probabilities and payoffs. The net holdings at each node encode how much of an advantage you have at the node (or how much you should pay to take over from another gambler at this point). If we let denote the probability of going on to win the World Series bet after winning the first bet then we must have:

   bet

Or
bet . For the real World Series we had
bet so
. This means we can read-off from the valuation tree that the probability of winning the World Series (for perfectly equally matched teams) rise from to after you win the first game.³ This can be confirmed from Figure 3. It is easy to confirm that a portion of all paths the node where Team One has one the first game end with Team One winning the whole World Series (each path must be weighted by its probability which are
). Instead of computing the bets we could have computed the probability of going on to win the World Series at each node⁴ (and then used the above equivalence principle to read off the required bets).

We can create a new diagram where we start at the node where our team has won the first game and we label all the non-ending nodes with the number of paths that reach the node. For example the two nodes immediately after start can be reach one way each and the next three nodes (“3 games to 0”, “2 games to 1” and “1 games to 2”) can be reached 1,2 and 1 ways respectively. It is a clever trick to notice that the easiest way to count the number of paths to a node is to just add the number of ways found on the previous nodes that point to the our target node. This clever way of counting paths is to use weighted paths (inspired by something called Pascal’s Triangle). Figure 4 shows a few columns of a weighted path diagram (thought he ending nodes are re-written as the sum of the paths reaching them where every path is divided by
which is the probability of following such a path).

The entries of weighted path diagram are identified by how many columns out from the start node they are and how many steps from one side of the row they are. Both identifiers start at zero so the starting node is denoted as
the two nodes just after them are denoted
and
. The three nodes just after these are denoted
,
,
and are (as we said before) equal to 1,2 and 1 respectively. These entries are called “binomial coefficients” and the rules for computing them (for integers ) are as follows:

   bet

A lot is known about Binomial coefficients. In fact by a formal called “Stirling’s approximation” we know

as observed.

Relations

de Finetti used this style of reasoning to provide a foundation for the basic theory of probability. Probability theory has always been somewhat problematic for mathematicians in that it has “content” or “an interpretation” whereas the power of modern mathematics comes from a more axiomatic or content-free way of thinking. The issue is if you are defining the meaning or interpretation of something like probability how do you check or demonstrate that you have the correct meaning without referring to some other pre-existing interpretation? A foundational or first interpretation has trouble looking for prior definitions to show equivalence to.[6]

The arbitrage-free arguments and the binomial arguments in particular are the basis of much of mathematical finance and are the basis for a number of Nobel Prizes in Economics including the Black-Scholes-Merton Option Pricing Model[2] and the Binomial Option Pricing Model.[5]

(the ratio of the circumference of a circle to its diameter) is one of the most famous constants in mathematics. Pascal’s Triangle is one of the oldest and most studied diagrams in mathematics with roots all the way back into ancient China.[4] It is actually remarkable how much Zhu Shijie 1303 diagram: looks like our modern version of Pascal’s Triangle (though they are separated by about 350 years, source Wikipedia): . The two diagram differ only in the notation used to write numbers and both start by filling in two diagonals of ones and all other numbers are the sums of the two numbers nearest and above them.

The arguments that replace paths with counts are a particular example of a technique called “Dynamic Programming” invented by Richard Bellman for mathematical optimization and now one of the core concepts of algorithm design.[1]

The idea of using a set of unknown futures that each have a known value is the key idea in solving a number of hard problems in probability and in optimization in the face of uncertainty. One of the the most famous of these problems is the “two armed bandit” where one must decide how to split ones bets between two slot machines that are thought to pay-off at different rates.[3]

For the two armed bandit problem the concern is how long to experiment with both machines when one machine seems to be paying more. The correct solution depends on seeing that how certain you need to be on the difference in machine vales (which in turn drives how long you experiment on both machines). This is a function of how long you intend to use the information. If you intend to play for a long time you want a long initial research phase to produce a very high confidence ranking of the machines; if you do not intend to play for long you want to switch to the machine you suspect is better sooner and on less evidence. Of course “slot machines” is just a toy-problem standing in for uncertain investments, research spending or even spending on different only advertising phrases.

Conclusions

The finance “no arbitrage” principle is actually a very powerful mathematical tool. It is equivalent to but somewhat more graceful than introducing probabilities when solving some combinatorial problems. In this setting it is equivalent to de Finetti’s principle and converting between probabilities and net holdings is very easy.

Bibliography

1

BELLMAN, R.
Dynamic Programming.
Dover Publications, 2003.

2

BLACK, F., AND SCHOLES, M.
The pricing of options and corporate liabilities.
The Journal of Political Economy 81, 3 (Jun 1973), 637-654.

3

CHERNOFF, H.
Sequential design of experiments.
Ann. Math. Statist. 30, 3 (Feb 1959), 755-770.

4

COULTER, L. O.
What is mathematics? toward a global view.
17.

5

COX, J. C., ROSS, S. A., AND RUBINSTEIN, M.
Option pricing: A simplified approach.
Journal of Financial Economics (Sep 1979), 39.

6

SHAFER, G., GILLETT, P. R., AND SCHERL, R. B.
A new understanding of subjective probability and its generalization to lower and upper prevision.
Game-Theoretic Probability Project (Oct 2002), 62.

---

Footnotes

… Mount¹

http://www.win-vector.com/

… arguments”²

More pedantically we are using the principle of “no arbitrage” or “arbitrage free” argument, but the name is traditional.

… game.³

Again, this if for the unrealistic situation of perfectly matched teams. For teams that have uneven probability the series strongly amplifies the better team’s chance of winning (which is one of the series intents). Also a better could update his subjective probability based on the first outcome which also changes things.

… node⁴

This calculation is in essence summing end outcomes across all possible paths weighted by how likely each path is. There are many possible paths, but the calculation can be performed quite efficiently.

… obvious⁵

“Obvious” is actually a special term in mathematics. To illustrate what it means we repeat a story. A mathematician was giving a lecture and stated that the point just shown was obvious. A student asked if it was really obvious. The mathematician stopped the lecture and paused to think. The mathematician thought some more, and eventually walked out of the room. Forty minutes later the mathematician returned to the lecture hall and informed the student that the last point was indeed obvious.

Related posts:

1. What is the gambler’s equivalent of Amdahl’s Law?
2. Paper on stock trading
3. New Paper

]]> http://www.win-vector.com/blog/2008/05/betting-best-of-series/feed/ 0 http://www.win-vector.com/blog/2007/10/paper-on-stock-trading/?utm_source=rss&utm_medium=rss&utm_campaign=paper-on-stock-trading http://www.win-vector.com/blog/2007/10/paper-on-stock-trading/#comments Thu, 04 Oct 2007 02:03:33 +0000 John Mount http://www.win-vector.com/blog/2007/10/03/paper-on-stock-trading/

What does the Market Think?

New Paper

A Discrete Model Gauging Market Efficiency

]]> author: John Mount

I have finally written up and released a paper in PDF: Automatic Generation and Testing of Trades describing a lot of the statistics and optimization methods used when I was technical trading on a Banc of America Securities proprietary program trading desk.  It was a very exciting time.

I have also included a less legible HTML version:

Automatic Generation and Testing of Un-Rolls for Profitable Technical Trades

John Mount¹

Date: September 9, 2007

---

Introduction

In this paper we discuss some of the basic steps in developing successful technical trading strategies. The method involves identifying an inefficiency or irregularity in the market and then using rigorous statistical methods to track and exploit this single feature of the market. We show how to automatically generate and test optimal un-rolls or trades that undo (at a profit) automatically triggered technical trades. That is to say, if the first half of technical trade is specified we show how to find the other half.

Our technique is to use standard tools, such as kernel methods[8] and Markov chains[4], to model both the efficient and the inefficient portions of the US stock markets.[6]

The author traded profitably using some of these techniques while part of a program trading desk at Banc of America Securities.

Technical Trading

Technical trading is a popular universe of security-trading strategies that trade using only the so-called technical data which are price graphs, volumes, bid/ask books and other data commonly available in market feeds.²Input sources can also include external triggers based on news, RSS feeds, on-line information and corporate announcements.³These strategies are very attractive in that that are quantifiable, easy to implement and easy to back-test on historic data. A major weakness of technical trading strategies is that they ignore deeper knowledge or analysis of the companies that are behind the securities being traded. Systems of technical trading are used both by large sophisticated hedge funds and by a varying population of day-traders.

Typical technical variables include price, time, volume and moving averages. It is important to know that many of these variables are really just analogies and not essential features of the market. For example: none of the variables current price, time, velocity, acceleration or inertia are real market quantities. What is traditionally called current price is actually the price of the last trade, which is in the past and may or may not ever be seen again. The fundamental variables of state of US stock markets are bid (best purchase price and quantity currently offered), ask (best sale price and quantity currently offered), and last trade (price and quantity). Each change of these variables is called a tick and can happen at any time. More detailed views include detailed bid and ask books from multiple market participants and estimates of inventory imbalance of various market makers and specialists.

In addition to working with the proper variables a sound strategy must also have at least two important components that we call foundation and empirical correctness. Without these components there is a large danger self-delusion and an unreliable strategy.

By foundation we mean that there are a priori reasons to believe that some variation of the strategy should be profitable. By ignoring the nature of the companies underling the securities being traded technical trading starts on shaky ground. In fact it is tempting to appeal to an efficient market hypothesis and claim that no technical trading strategy should be profitable. In some sense this is true- trades made in true ignorance expose a trader to significant risk, trading costs and pointless payment of the so-called bid-ask gap. Founded technical trading strategies are based on violations of the efficient market hypothesis- identifying situations where the market is in fact not efficient and trading into these situations. If there is no reason to suspect a market inefficiency there really is no reason to perform a technical trade. Testing numerous un-founded trading strategies is more likely to discover irrelevant anomalies in past data or discover flaws in one’s statistical procedures than it is likely to discover new valuable trading rules.[3]

Possible market irregularities include (but are not limited to):

• Market Open
• External News
• Earnings Reports
• M&A news
• Unusual Volume
• Inferred state of Market Maker / Specialist state
• Detailed Bid/Ask book.

By empirical correctness we mean that strategy can be validated and proven on historic market data. A technical strategy can have as much mathematical pedigree as you like, but it does not make sense if it can not be mechanically implemented and proven on historic data. Many technical features are popular due to their familiarity or the quality of graphs they produce- but the true measure is how well strategies generate specific executable actions and the quantified outcomes of those actions.

Given an irregularity it remains to develop the trading strategy. Typically this involves an initial trade (a buy or a sell) triggered by evidence of the irregularity/inefficiency followed somewhat later by a reversal or un-rolling of the trade (selling back against an initial buy or buying back against an initial sell). If markets were perfectly efficient and instantaneous in incorporating external events this should not work- so it is important to test that there really is a repeatable market inefficacy.

Possible initial trading strategies could include:

• Selling stock into an unusual price spike (a contrarian strategy).
• Buying stock immediately on news (a superior connection strategy).
• Selling stock into a perceived specialist imbalance (a superior knowledge strategy).

It would be naive to expect that a strategy that starts on a trigger and then reverses its trade blindly (say some fixed time after the trigger) is fully efficient. We must assume that other players in the market have seen effects of the trigger we traded and that their actions introduce biases and uncertainty into the market. Modeling these effects will allow us to produce a systematic unrolling strategy that can complete any entry strategy into a complete round-trip system. This systematic unrolling strategy is the subject of this writeup.

First Model

The Efficient Market Hypothesis

The efficient market hypothesis is a useful tool, even when you are attempting to find inefficient market situations. It represents the baseline you feel you have found a useful deviation from. The efficient market hypothesis has many variants but the essential content is that the market is full of informed players so any information is already factored in to the price. For example if there is publicly available information that gives a reasonable expectation that a stock should rise in the future then informed investors would purchase the stock early to be in a position to benefit from this increase. These purchases actually cause their own price-increase (by the simple laws of supply and demand) and have the effect of reducing the value of the information- as they move the price increase back in time (from the expected future change in value to the time of the anticipatory buying). This is what is meant by the phrase “already factored in.”

Figure 1: Dell 10-13-2006 Tick Data.

There is a mathematical concept that captures the idea of already factored in: Martingales. The Martingale condition is a concept that says the expected future value is the current value. For example betting a dollar on the flip of a fair coin is a Martingale of value (the odds of winning and losing a dollar balance out). The future value may be higher or lower- but when the Martingale condition is met the average of all these value weighted by their likelihood of occurrence is equal to the current value. The already factored in example mentioned above shows how the many players in the market tend to establish a near-Martingale by trading in such a way to move the current price to be the expected value of the future price.

Figure 2: Graph of a market-like random walk.

If market prices were the sum of many individual traders each with bounded-budgets who traded independently then we could apply the central limit theorem or law of large numbers and say that the market is indeed a random walk like the famous Brownian motion from physics. In fact on first inspection the market price histories (as in Figure 1) indeed look very similar to graphs generated by such a random process (as in Figure 2).

As we have said: it is no coincidence that the market looks nearly like a Brownian motion. Informed trading effects tend to impart Martingale like tendencies (once the overall increase factor of the value of holding wealth is factored out). Also, if the variance of the market were much larger than that of a similar Brownian motion this would itself attract channel traders who would benefit by trading in and out of the excess wiggling. The point is that an efficient market is usually pretty well described by random processes that have the Martingale property (like Brownian motions or Markov chains), so these are appropriate modeling tools.

If the market process really were such a random walk than there would be little point in technical trading. The whole theory of Martingales was developed to precisely describe situations where bets based on collecting historic information can not work. This is often called the no gambling system principle and it can be actually proven for systems like Martingales, unbiased Markov chains, drift-free Brownian motion and was even used as an foundational concept to define randomness by von Mises.[7] However, traders have a large number of pervasive dependencies. Dependencies can be shared information, herd mentality or shared trading practices. There are also some traders with very large budgets, so the conditions commonly needed to apply the law of large numbers do not apply and it is not inevitable that the market is indeed a Brownian motion. In fact one can show that even though the market overall looks very much like a Brownian motion it has too many events that would be considered very rare in this model (crashes, run-ups, events correlated in time) to have plausibly been generated by such a model.

Exploiting Inefficiency

A basic rule of thumb is: without a good reason to believe contrary you are not too far off assuming the market is efficient. So we decided to model the morning market as being nearly memoryless. That is we modeled it as if future prices depend only on the most recent price and not on the detailed history of prices. We will, however, condition the model on the bias introduced by the presence initial trade trigger.

The most basic memoryless model is the Markov Chain. In this model the world has finite number of situations called states. For example we could say the stock price being near each a number of price differences from the previous day’s close is a state. We could take our states to be: , , , , , . If our strategy involved an end of day sale followed by a next-day buy-back then knowing which state we are in allows us to assign a value to buying back the stock while in that state. This would be the negative of the relative change in stock price (price decreases work for us) times the value of the stock sold the day before (minus trading costs). If we modeled round-trip trading costs as and assume our triggered trade purchased a total value of of Dell then we could map buying back in each possible state to a net dollar value of the round trip. For instance buying back in the state would represent a net-loss of . We actually want to make the states a bit more detailed by adding a notion of time. If we modeled time in 5-minute intervals and (for the sake of diagram clarity) assumed that we only move up or down one state-level the Markov that modeled the first 15 minutes of the market could be represented in a diagram as in Figure 3.

Figure 3: Markov Chain Model

Each circle represents a state and each arrow represents a transition from state to state. We would use historic market data to find for every stock in this situation the relative frequency each transition is taken. For instance we would measure in our historic data what fraction of the time a stock that is 5 minutes and in the state moves to the state at the 10 minute mark. These learned state transition probabilities can be made to depend on factors from the previous day close (% increase, volume, market-capitalization) . In the diagram we are going to assume all transitions are equally likely except for the arrows with square bases which we each take to be twice as likely as each regular arrow leaving the same state. The success of our strategy depends on finding situations where our model predicts these sort of advantageous asymmetric conditions. Without these asymmetries (greater net propensity for price decrease than for price increase) we would be in a gambling situation where no strategy could possibly have net-positive value.

The diagram also encodes another assumption of the problem- we have a deadline for buying back the stock. In this case the diagram indicates a forced buy-back at time +15 minutes if a buy-back has not been made before that time. In reality many more levels and many more time intervals are modeled. Also note we have made the top row (representing maximal loss) absorbing. This is introducing a deliberate pessimistic flaw into the model (or equivalently adds a stop-loss condition to the strategy). We do not want the maximal loss states to have a reflected barrier (like the maximal profit states do) as this would make the model overly optimistic. Instead we force the model to be pessimistic and chose enough levels so that the maximum loss bound is not often achieved and therefor does not have large effect on the model.

Figure 4: Valuing Interior States

What we want to know is the net-value of being short (having sold) the stock the evening before. This is represented by the left-most circle which does not yet have a known value. The value of this state depends both on the transition odds of the states and on the trading strategy used to buy back the stock. There is, for example, no value in reaching the price-drop states if our strategy doesn’t take advantage and buy back while in these states. So the value of the states depends both on the uncertain future behavior of the market and of the currently unspecified buy-back strategy. The neat thing about this sort of diagram and treatment is that the forced-liquidation states at the end make it possible to simultaneously find the optimal trading strategy and assign values to all of the states. For example in the next diagram we see that the value of allowing the middle state at +10 minutes to ride (i.e. waiting instead of buying the stock back at this time) is equal to the properly weighted average of the ending states it connects to, in this case:
. The value of buying-back in this states is so the optimal strategy is to take our chances in the next time interval (see Figure 4).

We can repeat this sort of argument for each state in the second to last column and determine the net-value of each state under the optimal trading strategy. States whose optimal strategy is to stop (perform the buy-back immediately) are indicated by not having any outgoing arrows (see Figure 5).

Figure 5: Propagating the Valuation

The procedure moves from right to left using known states to fill in decisions and values for unknown states. In fact the calculation is so simple and orderly we can encode the entire filling-in procedure in a spreadsheet table:

.

This is in fact the same type dynamic programming[1] method used to value options under the binomial model.

The completed diagram is shown in Figure 6.

Figure 6: Complete Valuation

For our (made up) example the net-value of round trip trade is an expected value profit.

What remains is to choose a set of conditions to base a model estimates on. We then only trade situations that have an acceptable predicted risk and reward profile.

To build the state transition models we collect all the historic trade data and then segregate it into groups of data that match each possible trigger condition we wish to use to help bias our system. There is a trade-off: the more detailed the list of trigger conditions the more powerful biases we can detect (things are less smeared together) but we have less data available for each possible combination of conditions and lower reliability in modeling. To address this we advocate using non-parametric or kernel methods here to average data that nearly fits the conditions to get estimates that are both detailed and reliable.

For example our estimate is of the form:

A usable
can be gotten from the law of conditional probability (
), so we use
. Under empirical re-sampling each training example is treated as equally likely (more common situations are accounted by the fact they yield more examples in the training set) so we can use
.

For
we can just estimate the frequency of when we are in a near how often do we see a next-state such that
is approximately .

For both of these estimates is pays to blur things a bit during the estimation procedure replacing sums of the form:

with softer forms like:

A Second Model

One thing we one might want is to use a much more detailed model of time. One way to do this is just to add more time-states to the model. This can cause problems as we now have many more transition probabilities to estimate.⁴Suppose we wanted to switch our model from being indexed by time to being indexed by tick. Bid, Ask and Trade ticks can happen at any time and any rate so even with a trading deadline, so there is uncertainty in how many more ticks there are before the trade deadline. We can work at the tick level (without introducing too many states) by introducing a new model that has cycles in the arrow diagram (see Figure 7).

Figure 7: Recurrent Model (With Cycles)

The short vertical arrows represent the odds of moving from price-state to price-state in the same time column. The left to right dotted arrows represent the odds of being the tick that moves to the next time column. We can now estimate the transition odds from a great quantity of per-tick data giving us very reliable transition odds. We would like to fill in the values of all the states of this model (like we did in the earlier diagrams)- but the fill-in procedure will not work in the presence of cycles. States we need to fill in our given state do not yet have known values because they themselves depend on the state we are trying to value.

Linear Program Treatment

The standard way to deal with unknown quantities that simultaneously depend on each other is to introduce variables and write down a set of simultaneous inequalities.

If we introduce the variables ,
,
and
to represent all of the unknown values in our last diagram we can quickly write down many relations we know to be true for them.

For example for the set of variables through we know that each state is worth at lest as much as the value of stopping in that state. This can be written as:

Each state (except deadline and stop-loss states) is also worth at least the expected value of continuing one more step, which can be written as:

		escape
		escape
		escape
		escape

This can be re-written into matrix form where we have

and our vector of unknowns is

In matrix form we say . We are assuming we have estimates for all of the entries of and – so the only unknowns are the entries of . If these were equalities (instead of inequalities) we would call this a set of simultaneous equations and we could use linear algebra to solve for the unknown values. Because they are inequalities we will have to instead solve what is known as a linear program.[5] It turns out the optimal values for
are given by solving:

s.t.

This has an admittedly strange form (the objective condition
seems very arbitrary and one would at first think the likely form is
where is the vector probabilities of getting into each -state). There is also the issue that we merely wrote down inequalities that we knew would be true for the optimal solution to the stopping problem, but we have not guaranteed that there are not more conditions we have not thought of (i.e. these conditions are necessary, but we have not yet established that they are sufficient).

We show (in the appendix) that this is in fact the right procedure for solving for all of the -values. Each of these linear programs can be quickly solved using standard software. We can also see that the same type of procedure can then be applied to the -values (which depend only on -values, which are by this point known). In fact we can substitute back (using linear programs instead of filling-in) until we know the expected value (under the model) of the entire round-trip trade.

More on the Transition Probability Estimate

We can augment our state to carry more information that just the current ask-price relative to our previous night’s sale

If we are in of our Markov model we can modify
to be:
(to do this we build an estimated transition matrix for from only the trajectory of today’s stock and then evaluate how likely the trajectory the training example from the past is under this model, much smoothing/blurring is required to make this calculation usable). Even better: we can group training data and use Bayes’ law:

This allows us to group the training examples (on a few criteria, like less than a month old or not, trading volume, volatility …) and use a group of examples to build a model to evaluate today’s moves against (aggregated data to form model to check today’s single trajectory). As is traditional in Bayes estimates we ignore the denominator as it does not vary as a function of training group.

Conclusion

We have demonstrated some of the methods of using standard statistical and optimization techniques to automatically generate and back-test un-roll trades that turn properly conditioned technical trades into profitable round-trip trades. What we have presented is the technical machinery for building the second half of a profitable trade pair where the first half is some technical signal such as price or a market external trigger.

Bibliography

1

BELLMAN, R.
Dynamic Programming.
Dover Publications, 2003.

2

BREIMAN, L.
Stopping Rule Problems.
John Wiley & sons, 1964, ch. Applied Combinatorial Mathematics.

3

IOANNIDS, J. P. A.
Why most published research findings are false.
PLOS Medicine 2, 8 (Aug 2005), 0697-0701.

4

KEMENY, J. G., AND SNELL, J. L.
Finite Markov Chains.
Springer, 1960.

5

SCHRIJVER, A.
Theory of Linear and Integer Programming.
John Wiley & sons, 1986.

6

SHARPE, W., ALEXANDER, G. J., AND BAILLY, J. W.
Investments, 6 ed.
Prentice Hall, 1998.

7

VON MISES, R.
Probability, Statistics and Truth.
Dover Publications, 1981.

8

WASSERMAN, L.
All of Nonparametric Statistics.
Springer, 2006.

APPENDIX

Why the Linear Program Solution is Correct

How do we know the linear program solves the original problem?

• Because there are a lot of formulas?
• Linear program looks kind-of right?
• Works on a few examples?

To actually prove correctness we need to derive and compare to some representations of the optimal solution. All of the inequalities we wrote must be true for the optimal solution- but we have no prior guarantee that these are the only conditions. Their could be additional conditions that we forgot to model.

Breiman[2] presented a clever argument technique that exploits the particularly nice structure of solutions of this problem. He noticed that solutions have both a lattice like structure (you can combine solutions by taking minimums) and an operator structure (applying the probability transition matrix and stopping rules to a solution yields a solution). It turns out this is too much well behaved structure for any non-trivial solution set to have and it lets us show that optimal solutions are essentially unique which in turn lets us show the linear program solution solves the actual trading problem.

Theorem 1   Assume that every state in the Markov chain has a path to a forced stopping state. Let be a maximal optimal set of stopping nodes and define the vector such that
. Let be an optimal feasible solution to the linear program:

		0

where is the identity matrix, is the matrix of transition odds of the Markov chain and is the vector of stopping values.

Then .

The theorem says if is an optimal solution for the original valuation problem (that we may or may not know how to calculate) and is an optimal feasible solution to the linear program (which is now written in a slightly different but equivalent form) then . So, as hoped, solving the linear program is equivalent to solving the original stopping problem. The extra condition of every state being able to eventual reach a forced stopping state is true in our formulation due to the trading deadline.

The proof gets a little involved but the essential ideas are as follows:

• Check an optimal stopping solution would obey the linear program inequalities (so they are necessary, still need to show they are sufficient).
• Show that the linear program solution even if it did differ from the optimal stopping solution can not be less than the optimal stopping solution in any coordinate (this is the lattice minimum step).
• Use the fact that every state has a path to a forced stopping state to show that the linear programing solution can not hide any excess value above best possible stopping value away from the rest of the system (this is the operator step).

Proof. [Proof of Theorem 1] The theory of linear programming duality says that there is a dual problem to our linear program and this dual is:
where

		0

The point of the dual is it is known that for all feasible we have
. And for optimal we have
.

Take as an optimal solution to the linear program and the dual.

One can check itself must obey all of the conditions of the linear program so duality theory tells us
.

Define a vector such that
. also obeys the primal linear program inequalities, so we know
. Now
so we have
. Each entry of is and
for all which can only mean that . This means entry by entry we have
.

Now define the vector function such that
. For the true solution we have . The linear program solution also has . Now if we suppose then there exists an such that is maximal and state points to at least one state such that
). For this particular we claimed

So either way we have for this particular :
. But we must have
because
,
for all and
for at least one . So
and we see is essentially a contraction on the segment between and . Since a contraction on a bounded interval can not have two distinct fixed points our supposition that is untenable and we know .

We are done- we have shown there is essentially only one optimal solution to the stopping problem (the only possible variation is rules that differ in what they do for states- such that
). We also should by now have some insight as to why we used a linear program like
s.t. : the linear program is solving for the minimum value at each state that does not dip below the expected value of neighboring states.

---

Footnotes

… Mount¹

http://www.mzlabs.com/

… feeds.²

To emphasize; by technical trades we mean trades based on market data (as opposed to fundamental analysis) we do not include popular culture uses of the term such as candlesticks, Eliot waves and so on.

… announcements.³

We are assuming that these triggers can be made automatic by using a labeled information service or natural language processing techniques.

… estimate.⁴

The explosion of states can be managed by adding some regularity conditions on how transition probability estimates are allowed to change over time. This serves to reduce the complexity or rank of the estimation problem and improves the generalization ability of the model.

Related posts:

1. What does the Market Think?
2. New Paper
3. A Discrete Model Gauging Market Efficiency

]]> http://www.win-vector.com/blog/2007/10/paper-on-stock-trading/feed/ 0