Both model based science and statistics are wonderful fields, but it is important to not receive the results of one when you have paid for the other.

We will pointedly discuss one of the differences.First let us define our terms.

I will take “model based science” to essentially mean Popperian Falsifiability (an alternative to positivism). This is roughly: you construct a statement or model and the model is said to only have empirical content if it is in theory possible to “falsify the model.” That is the model must form predictions that are specific enough to potentially be disproved. If you see a single instance of the model being wrong, you say the model is wrong (or at best incomplete). And you are done. Frankly, for all the philosophical sturm und drang this is closest to what is meant by science.

I will take statistical modeling to roughly mean Fisherian Null Hypothesis rejection. This is only one branch of statistics (in addition to Fisher’s methods we also have frequentist and Bayesian methods, in particular see: Controversies in the foundations of statistics, Bradley Efron, Amer. Math. Mon. 85, 231-246, 1978) but it is closest to what is actually performed in statistical studies.

You can see the two methods sound very similar- they both emphasize rejection of a hypothesis. But this is deceptive. In the case of Popperian falsifiability you are essentially holding on to a hypothesis that you believe, but are very willing to give it up (one wrong prediction and it is out). In the case of Fisherian rejection you don’t believe the null hypothesis, but you are holding back rejection until you collect enough data to get rid of it.

Let us go over that again.

In the falsifiable or model based science regime: a theory or model would be a proscriptive set of guidelines or laws that allows you to build things (like tall skyscrapers). If ever one of your skyscrapers unexpectedly falls, you know your theory is wrong and you revise. Rejection is quick. But essentially you honestly believed the theory while you were using it. You were on its side and to counter this bias you agree to reject the theory on first failure.

In the statistical regime you never believed the null hypothesis. It is a stand-in you are trying to find a lot of evidence against to embarrass out of existence. Because you know you are against the null hypothesis you do two things try and mitigate your bias against the null hypothesis: you operationally presume it is true during reasoning and you don’t reject it until there is a lot of evidence against it.

To sum up in model based science you believe the model and are confident it can’t be toppled easily (so you don’t defend it as it you are confident it will survive) in statistics you doubt the null hypothesis and you give it every chance to survive (because you are sure that it will not survive).

Now that I have stated my premises let us move on the field I intended to criticize: paranormal powers.

To be deliberately rude: if you are investigating something that does not have a proposed mechanism that you are willing to test and reject you are not doing model based science. And by definition the paranormal is outside of current scientific explanation. It was too much to hope that we were doing model based science in this case (the appearance is deliberately that of science instead of statistics, but our science friends won’t help us call this out as they are often profiting from the same confusion). So you are doing statistics (and there is nothing wrong with that). But if you are doing statistics what is your null hypothesis?

• Null Hypothesis Candidate 1: ESP does not exist.

  This is a plausible hypothesis and sound “nully” (doesn’t claim much). But you would only be able to use this null hypothesis to try to prove the existence of ESP.

  But it is the exact wrong hypotheses to disprove ESP.
  “The null hypothesis can never be proven” (see Null Hypothesis and
  Statsmanship). Fisherian testing is unfortunately a one-sided design; it can only reject null hypothesis (not fully settle questions).

• Null Hypothesis Candidate 2: ESP does exist.

This is the null hypothesis you need to work with to reject ESP.

But here is the trap. You must operationally work with the hypothesis (even if you don’t like it) during the rejection attempt. Since you are forced to “operationally accept” the null hypothesis for the duration of the study you have absolutely no defense against critiques like:

This latter review didn’t find any problems in our methodology or writeup itself, but suggested that, since the three of us (Richard Wiseman, Chris French and I) are all skeptical of ESP, we might have unconsciously influenced the results using our own psychic powers.’

The paranormal is just one big game of Mornington Crescent. So if you failed to claim that there is no such thing as psychic dampening powers before your opponent accuses you of using such powers: you lose. The game is all about timing, not reality. If you don’t like this kind of situation, don’t get into this kind of situation.

This is why you shouldn’t use statistics to study bullshit. Statistical testing methods are deliberately designed to be weak. Unfortunately they are easy to work around if given enough rope.

None of this would matter if it didn’t also hold for a lot of what is called mainstream science. Everyone wants the adulation of having imp ortant scientific results; but they seem to only to want to pay to commission statistics.

Take big money pharmaceuticals as an example. Non-working drugs can deliver equivocal results forever (as long as you keep weakening the proposed claims after each study) and always being “on the verge” of a significant result can fund an endless number of studies and careers.

It now past time to define what I meant by “magic.” Magic, for this article, is any hypothesis that is not sufficiently specific and bounded. You can design statistical studies to test many things, but only if you can specifically describe the limits of what you are attempting to study prior to the experimental work. There are two main classes of magic hypothesis the powerful and the weak. Powerful magic hypothesis are unfalsifiable because they have no pre-defined limit on what they can bring in to defend theirselves post experiment. Weak magic hypothesis are unfalsifiable for the simple reason they can be revised after any experiment to claim the effect is present but just slightly more subtle than the resolving power of the last experiment.

You must be very clear about when you are doing science and about when you are doing statistics. The unfortunate truth is: it is very difficult to successfully dispute junk science using tools as deliberately delicate as statistical hypothesis testing. Without a sufficiently critical mindset you get deliberately bad statistics, cargo cult science and dishonest math. A good essay on this researchers wanting to claim the benefits of the trappings of mathematics (but not willing to meet the very strict pre-conditions required) is “The Pernicious Influence of Mathematics on Science” Jack Schwartz, 1962 (collected in “Discrete Thoughts: Essays on mathematics, science, and philosophy” Mark Kac, Gian-Carlo Rota, Jacob T. Schwartz, Birkhauser 1992).

Related posts:

1. Statistics to English Translation, Part 2a: ’Significant’ Doesn’t Always Mean ’Important’
2. Statistics to English Translation, Part 2b: Calculating Significance
3. “I don’t think that means what you think it means;” Statistics to English Translation, Part 1: Accuracy Measures

Tools that support complexity (versus eliminating or reducing complexity) introduce repeated ritualistic procedures (or more simply: bad habits). Without critical winnowing bad habits and bad tools accumulate in the workplace. The extra complexity and useless activity brought in by bending to bad tools creates additional procedures, rituals and stereotyped activities. These new activities attract new bad tools to support the additional complexity. In an uncritical environment this cycle of complexity and bad tools becomes self reinforcing and starves out useful production.

Most software development environments are desperately in need of an organized weeding. One criterion to consider for each tool is: does the tool support production (the presumed goal of your organization) or does it support complexity (the enemy of clarity)?

I will use the common tools of computer science and software engineering as my example.

Computer science started as a set of concerns coming from a tool that kept getting re-invented: the primitive computer (for example: Jacquard’s automated looms of 1801, Babbage’s uncompleted Difference Engine of 1822, Hollerith’s counting machines of the 1880s). The field of computer science was formed in the wake of these tools (and their successors) and codified in the 1930s by work of Church, Godel and Turing. The first tool of computer science was the computer and the first side effect was the founding of the field of computer science itself (a fairly expensive consequence).

The second generation of tools were simple and direct: loaders, linkers and eventually assemblers. Managing a computer was initially a hard job. These tools largely made managing a computer easier without introducing large side effects. The tasks were concrete and already implied by the invention of the computer (moving code and data into memory, letting different bits of code know about each other and producing the number-coded computer instructions from more readable text mnemonics). I believe it is this
generation of tools that set the expectation (in computer science) that tools are safe in that they tend to simplify pre-existing tasks (tasks that may or may not have been previously identified).

Then we moved to the age of software engineering. Newer tools such as: editors, interpreters, compilers and file systems took the field. Each of these tools had larger impacts outside of operating a computer. Editors changed the way we write (largely for the better). Interpreters and compilers allowed the introduction of new computer languages (which are essentially new models for processes, abstraction and thought). File systems become the dominant bad metaphor for organizing information.

Then an explosion of systems: graphical user interfaces, integrated development environments, source control systems, document management systems, wikis, bug trackers and issue trackers. And here is where we really need to start exerting some criticism and prudence. Some of these tools are dominated by their supporting rituals. Some of these tools support accumulation of complexity. Some of these tools specialize in managing concerns that did not exist prior their own introduction. I’ll try and focus on the unintended consequences of a few of these tools to try and make their insidious cost a bit more apparent.

Graphical user interfaces and integrated development environments are necessary for new users (and we all must spend a lot of time as new users, else we are not growing). The ability to push a single button to re-build an entire system is initially empowering. Until you find out you have to be there to push the button. You can’t automate the build (as only the IDE understands the build and eventually it can no longer export working scripts, Makefiles, ant files or other editable build files). This is so silly it has to be repeated: you can no longer automate a process that exist entirely within the computer. Then you have to push two buttons (“update files from source control” button and then “build” button). Then you have to push three buttons (refresh “source control state” button, then “update files from source control” button and then “build” button). As the tool traps more and more of the configuration knowledge you become more and more beholden and less clear on which steps are needed and which are mere superstition. Software engineering shops where checking out files takes over an hour, builds take a half hour and unit tests take an hour are not uncommon. And a user has to sit with the process to press “continue” throughout. As you would expect- under time pressure steps that are expensive are skipped and code is checked in untested (it being less effort to circumvent the source control change list managers and issue trackers than to jump through the near infinite number of hoops a semi-automated build system can support).

Document management systems and wikis also start very strong. Until they hit a certain critical mass where both their search function starts to fail (as too many irrelevant documents refer to critical search terms and mask the desired documents) and any desire to properly document or organize is lost to “it’s already in the wiki.” Meaning that a lucky search may find an out of date mis-informed document that purports to answer the question at hand. Eventually new engineers learn not to ask questions because they know not only will they have to find the wiki page, but they will be asked to update it.

My real invective is saved for issue trackers and bug trackers. Both of these systems start out solving a real problem: documenting required tasks in the first case and documenting tasks that fix flaws in the second case. But the goal should be to close issues and fix bugs, not track them. Issue trackers are routine abused by product managers. The existence of an issue tracker seems to relieve a product manager of: producing requirements, documenting requirements and answering any “a or b” question with any answer other than “both.” In a functioning business you can divide issues into: what we are working on now and everything else. It was once the product manager’s job to plan, prioritize and track issues and assign them to the development team in coherent linear order. Instead every idea (good or bad) is stored in a morass of systems with priorities, dependencies and a bunch of other knobs that grow to consume all free time. Bug trackers are similarly abused by development teams. When you submit bug: you want it fixed, not a requirement to produce copious documentation to enter into a system where it will be held for years (and become unreproducible as the software evolves forward). Both groups (product managers and development teams) should have tools for storing and organizing things- but these should be within the group, not used for communication.

There is a very solid reason to not favor tools that try to hide complexity. Such tools make complexity seem cheap and infact cause much more complexity. At some point the tool has encouraged so much complexity that the tool becomes indispensable and displaces useful work. This is a variation of the Jevons Paradox. If you make the unit cost of a good cheaper people tend to consume more of it. They use it as a substitute for other (more expensive) goods and usually consume so much more of it that they end up spending a larger total amount on the good. For example if 30% price decrease encourages a 50% increase in use, then: your total expenses go up by 5% (not down!). You lower the perceived unit cost on complexity you likely get more overall complexity as a result. These changes seem small, but they accumulate. Every time you make managing complexity and delaying decisions easier you get more overall complexity. Complexity is used as a cheap substitute for product research, user studies, making decisions, writing correct code and actually fixing bugs.

Of course it is all a matter of degree. When these tools are used to simplify an already existing workflow they are good. When they allow a useless ritual sequence to live and grow they are bad. There is a surprisingly effective way to measure the degree of damage tools are inflicting. Stand quietly and listen to the office sounds. If you mostly hear typing and talking: things are likely good. If you mostly hear mouse clicks: things are bad. Typing is usually part of a proactive task and mouse clicking is usually part of a reactive ritual. If you are an interested manager with a very strong stomach: attend new engineer training (if you have it) or shadow a new engineer for a day; it will be painfully obvious if your environment is empowering or teaching helplessness.

Uncritical repetition of bad habits literally trains you in being dumber. A work environment that accumulates bad tools becomes a mind dulling quagmire of required rituals. We must be vigilant in encouraging tools that support production and discouraging tools that support needless complexity. There should be no natural desire to “manage complexity,” the goal should always be to eliminate complexity. Prefer a board of Post-Its to an issue tracker. When your project is a mess the board will look like a mess, whereas the issue tracker will hide and support the mess. An earlier crisis is a cheaper crisis. Unexamined your organization’s procedures perform a random walk (bringing in new tools and habits) that is naturally biased towards ruin; you must continuously apply concious corrections to avoid this.

Distaste for tools is , of course, not new. In Plato’s “Phaedrus” Socrates recounts an Egyptian god warning that the invention of writing will create: “hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.” Or that the greater quantity of text (supportable only by the invention of writing) will cloud minds. The distinction is: whether a tool is more forceful in its primary purpose (in the case of writing: archiving information) or in its parasitic supporting rituals (in the case of writing: the production of papers, pencils, inks, book binding and building shelves).

Do not reward building shelves unless building shelves is your actual business.

Related posts:

1. Kernel Methods and Support Vector Machines de-Mystified
2. Must Have Software
3. Map Reduce: A Good Idea

• Squander A/B testing bandwidth on bugfixes:

  A/B testing is the art of testing two or more variations of a product in parallel to try and directly detect or measure an important difference between them. The idea is to lock in any positive changes and back out any negative changes. An A/B platform needs to manage a lot of measurements to get sample sizes large enough to return reliable results. A typical result of measuring a series of 10 attempts to raise revenue per customer might look like the following:
  

  
  
  
  Some of these changes are good and some are bad. However, if you don’t have the bandwidth to make the A/B tests you don’t know which are which and you end up essentially forced to take all of the changes that “sounded good” (shown as the blue curve in the next chart). If you do have the measurements you back-out the bad changes and keep accumulating the good ones (the violet curve in the next chart). The difference is dramatic.
  
  
  

  As you can see: with enough useful experiments you can cherry pick a bunch of risky ideas into accumulated improvements.

  All of this depends on a stream of good ideas and having enough bandwidth (customers segregated into different treatment and control groups) to make all of these measurements as each of these variations is applied. A good way to lower the demand for new ideas is to clog up the A/B testing infrastructure.

  One way to clog up the A/B testing infrastructure is to reserve the testing and statistics infrastructure to document the group is meeting goals. For example- collect a lot of statistics on a necessary bug fix. A policy saying the impact of all bug fixes (even those you have no choice but to implement) must be quantified can easily eat up all of your A/B bandwidth without testing any new ideas. If asked why you are doing this say it is to ensure that bug fixing is meeting its ROI targets.

• Encourage the A/B testing framework to sting itself to death:

  The hardest thing to measure statistically is a non-effect. This is because a non-effect (that is a change that does nothing) is identical to what statisticians call the “null hypothesis” (which is the hypothesis you are trying to reject). Any attempt to measure a non effect will return a result that isn’t quite zero but doesn’t quite have enough data to show there is an effect significant at the current effect size being looked at. A repeated study with more data will get the same sort of equivocal result, just for a smaller effect size. This is why when designing a study you need to first establish a lower bound on effect sizes or be willing to say something like “we see no change below x% as being clinically relevant.” Otherwise if there really is no effect you get a series of bad studies as you are tempted to mis-use larger and larger sample sizes to study smaller and smaller effects. Statistics can never “prove zero” they can only prove below a given bound.

  A famous example is attempting to test the difference between 41 shades of blue on a thin border. You know this
  can make no real difference, but the poor suckers running the test will only get equivocal results. You can then send them back to run larger tests (which will also fail to achieve statistical significance) because “at our scale even a very small effect is important.” Insist the statistician prove there is absolutely no effect (don’t let them get away with proving any effect is below a given size).

• Don’t provide a domain expert or product manager:

  One of the more useful tenants of modern software development (in particular some of the variations of “agile”) is that for useful work to be done a domain expert or product manager must be integral to the effort. Often this role is called “the customer” and it is an individual in the company (not a real customer) who has the experience, intuition and authority to declare success or failure (in addition to supplying ideas and useful intermediate goals). Ambitions research is even riskier than development, so make sure the research group does not even meet good development practices (let alone good research practices).

  For example:
  Statistics/Analytics is very good at testing and quantifying possible profitable hunches but (despite some of the broader claims attributed to data mining) has no systematic way of generating non-trivial hunches. So you can slow down an analytics effort by not supplying any intuition. Insist that it all “come from the data.”

• Insist on retrospective studies:

  Convince management that the market will not tolerate experimentation (customers will revolt, competitors will see our secret sauce, …). So any proposed change can only by analyzed by attempting a retrospective study on older data. Instead of exposing new customers to variations on proposed improvements have the analytics group sift through old data and model (guess at) impacts these changes would have had using machine learning or statistical modeling. Machine learning is particularly painful without training data and statistics depends on meaningful measurements. Retrospective studies are very important- but they can not be your only tool.

• Insist on perfectly clean studies:

  If the retrospective trap doesn’t work you are a good position to push for “perfectly clean studies.” Only one variation can be tried at a time (else variations interfere) and you can’t even end the trial on disaster (“could be a fluke, backing out the change now would give our data a censorship/stopping bias”). With enough procedures and insisting on sample sizes specified before having any hint at the effect size you are trying to measure you can completely crush analytics.

• Self service analytics:

  If the “clean study” gambit doesn’t work then you are in a good position to advocate “self service analytics.” Push control of the A/B testing infrastructure to all of the engineers. Any engineer can request a fraction of the site traffic to try a variation on. Each customer might see many different variations from the many different engineers- “but hey, with a little linear algebra the stat guys can iron that out.”

• Security:

  They can’t analyze the data if they can’t get to it. Partition that data into different areas of sensitivity, build elaborate procedures and protocols so data from different sensitivity areas can not be combined. Or just deny analysts access to all of the data. Your IT/Networking department can do this for you with complicated chains of trusted clients, VPNs and approved builds.

• Dining Philosophers:

  Make sure you don’t provision enough resources (machines, disk, memory, database nodes) for all of your analysts to work at the same time. Get them to turn on each other. This is sometimes called the datamart method.

• Catch 22 ROI:

  Don’t budget a study until you know the expected ROI of the result and don’t accept an ROI estimate that isn’t backed up by a study.

• Blue Ocean Strategy:

  Make sure your analyst has a “blue ocean opportunity.” Give them data that nobody has ever used or looked at before. Wait a while and then say “they are way
  too expensive to be running down these picayune data cleaning issues.”

• Run before you can walk:

  Insist the analysis scale to “billions of records” on the first try. Or try the early spec gambit: “this needs to go into development parallel to you doing the research.”

• “I could have done that in Excel”:

  The dual to the run before you can walk strategy. Don’t allow any easy victories (like “we found all of the currently unprofitable accounts”). Insist on exotic models and above all “prediction” (“predict which accounts will become unprofitable”).

• “Needs to be more explainable”:

  The entire analysis technique needs to fit onto a single Powerpoint slide- “for upper management.” This is the dual strategy to the “I could have done that in Excel” strategy.

• Insist on Excel:

  Insist on and enjoy the deadly dance of pivot tables, office data connections and plugin solvers.

• Death by software engineering:

  Insist not on a result or procedure but a “dashboard” with “an intuitive UI.”

• Postulate sub-populations of non-customers:

  Even with a product manager you can force failure by concentrating analytics on the wrong questions. Postulate three to five customer types that find your product lacking for different contradictory reasons (“too technical”, “not for power users”, …). Now you can squander effort on: characterizing the customer groups, estimating the size of the customer groups and estimating the improved uptake each incompatible change to your product would induce in each group. Instead of working on your product you are now working on psychology, demographics and many incompatible variations of your product.

In conclusion: if you can’t win this game against the analysts, you aren’t really trying. Or for the non tongue in cheek version: successful ambitious analytics requires a minimum amount of attention and flexibility. All of the “blockers” here are variations of valid concerns that only become blockers when either there is no attention or adaption.

Related posts:

1. Deming, Wald and Boyd: cutting through the fog of analytics
2. Increase your productivity
3. Map Reduce: A Good Idea

> l <- list(x=c(1,2,3))
> l$x
[1] 1 2 3

But suppose we wanted to automate this; that is pass in the name of the value we want in a variable. We are after all using a computer, so automating a step seems like a reasonable desire. R supplies a notation for this using the “[]” operator. But something slightly different comes out under the “[]” operator than under the “$” operator:

> varName <- 'x'
> l[varName]
$x
[1] 1 2 3

Notice that the printed outputs are slightly different (one echoes "$x" and one does not). Let's use the "class()" method to see what is actually being returned in each case.

> class(l$x)
[1] "numeric"
> class(l['x'])
[1] "list"

Completely different return types are returned (in one case a numeric vector in the other a general list, not interchangeable types).

At this point you may think it is time to turn in our "pro" label and call ourselves "newb" (Internet slang for "newbie" or "idiot"). But let's slow down for a bit. When two views of the same situation disagree (such as the difference in opinion between the authors of R and myself whether the "[]" and "$" operators should return the same type) you at most know that at least one of those views is wrong. You don't really know if one view is right or even if one view is right which one it is. I can, however, bring in some additional argument to try and show the design of R is in fact wrong. The additional argument is "The Principle of Least Astonishment." This principle roughly says that it is a mistake to introduce unnecessary differences in outcomes (which to the unprepared user are unpleasant surprises). There may be some deep (yet obscure) reasons the two operators prefer to return different results. But the fact you would have to find a way to document and explain these differences really should make one think that this situation is really a mis-design and the "explanation" is really an attempt at a work around. Or to put it more rudely: there may be an explanation, but there is no excuse.

For another example consider creating a 3 by 3 matrix:

> m <- matrix(c(1,2,3,1,1,1,0,0,1),nrow=3,ncol=3)
> m
     [,1] [,2] [,3]
[1,]    1    1    0
[2,]    2    1    0
[3,]    3    1    1

Now select the last two rows of the matrix.

> m[c(FALSE,TRUE,TRUE),]
     [,1] [,2] [,3]
[1,]    2    1    0
[2,]    3    1    1
>

Now (for the punchline) try to select just the middle row of the matrix.



> m[c(FALSE,TRUE,FALSE),]
[1] 2 1 0

Notice that once again (and without warning) the result is subtly different. I admit that it seems paranoid to worry about such small differences- but when you are debugging a system that should work these are exactly the killing mistakes you are looking for. In this case the problem is pretty bad. See what happens if you tried to ask for the dimension of each of these differing returns:

> dim(m[c(FALSE,TRUE,TRUE),])
[1] 2 3
> dim(m[c(FALSE,TRUE,FALSE),])
NULL

The first case works fine (reports 2 rows and 3 columns). The second case returns "NULL" (instead of 1 row and 3 columns). In R NULL is sometimes used as an error-value (instead of throwing an exception) and this value will poison any further conditions or calculations it is involved in. The main way to deal with the arbitrary introduction of such NULLs is the incredibly tedious uncertain defensive coding practices that we argue against in Postel’s Law: Not Sure Who To Be Angry With. Such code weakens both programs and programmers.

But what is going on in this example? Once again we use the "class()" method to inspect the subtly different results.

> class(m[c(FALSE,TRUE,TRUE),])
[1] "matrix"
> class(m[c(FALSE,TRUE,FALSE),])
[1] "numeric"

The result is disappointing. For a two-row select R returns a matrix (what we would expect). For a single-row select R does us the "favor" of converting the result into a vector. This is a disaster. A single row matrix is similar to a vector, but even R itself does not support the same set of operations and outcomes on vectors as it does on matrices (for example the failure of the "dim()" method). It is not safe to further calculate with these results (without by-hand converting the result back to a single row matrix which R can in fact represent). In my case this created crashing bugs deep in a long running analysis (and was hard to diagnose as the bug was in an "innocent operation" not in a "risky calculation").

All of this has to violate John Chambers' "Prime Directive" for data: "an obligation on all creators of software to program in such a way that the computations can be understood and trusted." Chambers' opinion being relevant as he is the author of the S language (of which R is an open source re-implementation). We continue to recommend R, but we also recommend being exceptionally careful when using it (which unfortunately adds time to projects).

Related posts:

1. R examine objects tutorial
2. CRU graph yet again (with R)
3. Survive R

Postel’s Law (also called The Robustness Principle) comes from RFC 761 “Transmission Control Protocol” in 1980 and is: “Be conservative in what you do; be liberal in what you accept from others.” (Side note: RFC is the now ironic acronym used to describe Internet standards- the letters stood for “request for comments”).

This idea probably worked best where it started- in the TCP/IP world which is where a lot of hairy details of how computers network are handled. When your goal is the basic establishment of transient communications- success is measured by getting information through and not unnecessarily triggering a failure. It may be okay to tolerate mistakes here- because they don’t live long anyway.

Unfortunately, the law works less well other places where it is applied. The law is a downright hazard in dealing with archiving meaningful data (instead of managing transient signaling protocols). Sometimes the cost of obeying the law far outweighs the potential benefit.

A common arena for Postel’s law is now HTML, the markup language used to represent the content we view in web-browsers. In this arena Postel’s law has had two consequences: one good, one bad.

The good: almost anyone can create a working web-page or even a web-site because modern browsers have been designed to paper around almost every common HTML mistake. It has been pointed out that this ease of creation and “worse is better” (a deep principal due to Richard P. Gabriel see wikipedia: worse is better) has been one of the reasons that HTML out-competed and killed many other ideas. Philip Greenspun’s famous story of a 10-year-old building web site to get his mother medical attention happened in the sloppy world of HTML and could not have happened in the straight jacket of RDF (Resource Description Framework: the darling of the semantic web). I would not wish having to actually read or adhere to the incredibly long and irrelevant standards from w3.org (where the ratio of value to pedantry goes to zero) on an enemy. The web is only interesting due to its content and much of its content was only possible due to low barrier of entry.

The bad: to read HTML you almost have to re-create the entire history of web-browsers. This is a history of many hostile competitors (Microsoft, Netscape, Opera, WebKit, Mozzila, Google) and billions of dollars. Reproducing a significant fraction of this history is a significant (and useless) expense. For the most part I use a permissive parsing library like TagSoup or HTMLTidy but even these miss some things that browsers accept and are far more complicated than the task truly justifies.

Even worse is the cases of XML and RDF. These are often used for archival storage of semantic data. That is you may need to read and understand (not just display) data in XML for a long time. To be liberal in what you accept you have to again master a long set of useless complications (DTDs, namespaces incredibly inept character encoding and escapes) and still get burned by improperly encoded XML (that “used to work” because the bugs in the emitted XML matched the bugs in a library that is now out of date).

It is clear in the case of HTML and XML that Postel’s law’s cost is too high for what it delivers. Or at least half of the law is too expensive: no amount of being generous in what we accept makes up for the original data not have been impounded correctly (not being “conservative in what they do” and not having checked that at the time it was created). Some of this is that the producers of the data have no way of telling they are not being “conservative in what they do” because the “generous in what they accept” libraries they use to debug don’t tell them they are emitting bad data. And lets be honest- most systems are not designed for correctness, they are instead debugged until they seem to work. I would say that in fact HTML is not an example of the power of Postel’s law but of the pernicious influence of “worse is better.” Computer science has not risen to the level of “software engineering” we still are a horrible “fit to finish” industry.

Frankly for many things we need a simpler “fail early” discipline. Tools need to be better and standards need to be simpler so that if you write something that is wrong it is easy to see why it is wrong and easy to fix it. Postel’s law has helped hide the negative impacts of complicated standards, we need to push the cost of complications back on to standards committees. The need to be “generous in what you accept” overly favors large, rich entrenched players who have had the time and resources in incrementally invest in papering around every common mistake.

However, I am not sure if we can throw out half of Postel’s law or even if we want to. When Postel’s law fails it is not clear who to be mad at.

Sun, to kick somebody who is already down, was famous for making elaborate frameworks that correctly and brutally implement many details of RFCs. Sun’s Java includes huge frameworks for XML, UTF8 and email that scrupulously implement page after page of useless standard documentation but fail in the wild due to not being “generous in what they accept.” For example Sun’s GlassFish (which got listed named as one of four or five important assets during Sun’s various acquisition talks much like the fact the car has cup-holder somehow always gets mentioned in spec sheets) is an “open source production-quality enterprise software application server.” A supposedly major component of the GlassFish is its email component which is a huge unwieldy framework that implements many of the email related RFCs and protocols including IMAP. Unfortunately for all its hugeness it can not reliably read email folder names from one of the biggest IMAP servers: Google Mail. Google Mail includes “against standard” characters in the protocol and crashes the GlassFish software.

And here is where Postel’s law fails us: under Postel’s law both sides are at fault (Sun for failing to be generous in what they accepted, Google for failing to be conservative in what they did). We can’t assign only one villain. We have no proscription of who to ask for a fix. Postel’s law seems useful in that if either Google or Sun had followed it the two systems would work. But the law doesn’t pick one side to assign blame and help us to efficiently diagnose and fix the problem. It becomes difficult to find the critical bugs when they are masked by a see of “acceptable” bugs. Take a contrary example: the simpler law “implement the standard or fix the standard” would clearly assign blame to GMail.

Similar pain is encountered in Java’s handling of character encodings like UTF8. It is hard to move up the stack of artificial intelligence (from words, to concepts, to ideas, to reasoning to consciousness) when you can’t even reliably transcribe characters. When faced with bad character sequences (a common occurrence on the web) there is no practical way to get Java “mostly parse it,” Java libraries and frameworks authors seem to extract a perverse joy in throwing a program-killing exception (it does not matter if you catch it the library has already stopped doing what you wanted) because they are concerned that a diacritical mark was not properly encoded (web browsers, on the other hand, lose the mark or show some sort of damage near the mistake and blunder on). And here is were the frustration sets in, how can you make applications that are generous in what they accept when the libraries and frameworks are overly proud and picky? This, at first, seems like an argument for Postel’s law- if everybody else (especially the library authors) were generous in what they accepted your life could be easy. That is certainly one possibility- but I argue it often becomes a matter of semantics to assign blame where there is no pre-existing specification or performance agreement. In the end you will waste more time dealing with errors that should never have made it to you than the time you save emitting the odd error of your own.

The unit testing people have a somewhat better idea: fail early, fail at the factory where it is cheap to fix. Don’t litter all of your code with indecisive statements like:

  Set matches = computeMatches();
  if( matches!=null ) {
     for(String match: matches) {
         ...
     }
  }

Instead: write a unit test to document you expectation that the empty set is expressed in single consistent way:

   Set matches = computeMatches();
   assertNotNull(matches);

And from then on write more confident code:

  for(String match: computeMatches()) {
         ...
  }

This may seem overly optimistic and overly strict- but I have a point. One of the few good principles in computer science (and perhaps one of computer science’s contributions to knowledge, computers are a huge contribution to society- but they were made by engineers) is composition. A plan for getting from A to B followed by (or composed with) a plan for getting from B to C is a plan for getting from A to C. Well a correct plan for getting from A to B when composed with a correct plan for getting from B to C, if each of the plans “is mostly right if the piece after is so nice to fix up a few mistakes” you really don’t know what you have. You may have nothing.

That is my complaint- you can’t put an a priori bound on how expensive attempting to allow both sides of Postel’s law will be. You would like others to paper over your mistakes, but it is becoming too expensive to paper over the mistakes of others. In the end Postel’s law is of little help when cleaning up the inevitable mess.

Related posts:

1. R annoyances

Q: What is the difference between a banker and a trader?
A: A banker will try and tell you a 10% loss followed by a 10% gain is breaking even.

This is a bit less arcane than some of the issues we usually discuss on the Win-Vector Blog, but it is a fun one. And it does take some effort to disabuse yourself of the banker’s fallacy.

It turns out that a lot of our instincts about something as simple as ratios is not quite right. Likely this is because the innate skills of counting leads to a deep understanding of addition and not of multiplication. Take for example the opening joke: a 10% loss followed by a 10% gain sounds like it should be exactly breaking even. But in fact it is exactly a 1% loss.

To compute the loss and gain on $100 we would say after the 10% loss we have $100*(1-0.1) = $90. A 10% gain on this remaining portion would be written as $90*(1 + 0.1) = $99 which, as predicted, is missing a dollar. An incorrect explanation would be something along the lines “well the loss was first, so it applied to a larger number than the gain.” But relative losses and gains work by multiplying and therefore is insensitive to order. It is a fact that a 10% loss followed by a 10% gain is exactly the same as a 10% gain followed by a 10% loss (which eliminates the attempted explanation). The correct explanation is the flaw was far earlier than you would think: you should not believe that the opposite of 10% loss is a 10% gain. To undo the effect of a 10% loss you need just over an 11% gain (a 11.1111111% gain).

For a more dramatic example consider the Dow Jones Industrial Average. It was at $12827 on January 7th 2008, by March 5th of 2009 it had fallen to $6594 or a 48% loss. By January 4th 2010 it had experienced a 60% gain relative to March 5th 2009- but that only got us to $10583, still a 17% loss relative to January 7th 2008. The opposite of 48% loss is in fact 192% gain (which obviously has not happened).

Bankers typically quote interest rates as if they were additive. Things like points and fees are all added. This is almost correct for small interest rates. This nearly right (but actually wrong) language is why we have a bestiary of confusing terms describing interest: simple interest, compound interest and yield. The bankers need some way to signal which numbers will actually be used for computing your mortgage payments versus which numbers will be used for advertising (and in the US they tended not to tell you many of the more important numbers until they were required to by law).

Traders, on the other hand, are very comfortable with multiplying relative losses and relative gains. The main trick of achieving such mastery is to convert multiplication into addition. The way to do this is the log() function (or the logarithm).

The log() function is simple function that has the property that log(a*b) = log(a) + log(b). For connivence lets pick our notation so that log(10) = 1. From this we can deduce that it must be the case that:

log() can not be used on zero or negative numbers (at least not if you expect a real number as a result). For other values we use our calculator.

A trader uses logarithms to think additively about relative changes (also called “returns”). Breaking even is represented as 0 (our friend log(1)), relative increases are represented as positive numbers and relative decreases are represented as negative numbers. For example a 10% loss is represented additively using logarithms as log(1- 0.1) = -0.0458. Now in this logarithm notation the additive opposite of a -0.0458 is in fact (as you would hope) +0.0458. You can even double check: log(1 + 0.1111111) = 0.0458. In this notation the mathematics and the language work together- the opposite of a loss is a gain with the same magnitude (and positive sign).

Returning to our initial example: a 10% loss is represented as -0.0458 and a 10% gain is represented as log(1 + 0.1) = 0.0414, so if we add them (how we combing operations in the logarithmic notation) we get -0.0044. Notice this is not zero, and is in fact equal to log(0 – 0.01) or a net-loss of 1%.

The point is that even trivial math becomes difficult if you are forced, by language or convention, to work from false premises.

Related posts:

1. Thievery considered harmful
2. Why you can not to use statistics to dispute magic

If the re-constructed technique is close to what was originally done then so many bad moves were taken that you can’t learn much of anything from the original “result.” This points out some of the pratfalls of not performing hold-out tests, not examining the modeling diagnostics and not remembering that linear regression models fail to low-variance models (i.e. when they fail they do a good job predicting the mean and vastly under-estimate variance).

Our article not an article on global warming, but an article on analysis technique. Human driven global warming is either happening or not happening independent of any bad analysis. Finding the physical truth is a bigger harder job than eliminating some bad reports (the opposite of a bad report is not necessarily the truth). Bad analyses can have many different sources (mistakes, trying to jump ahead of your colleagues on something you believe is true, trying to fake something you believe is false or be figments of overly harsh critics) and we have not heard enough to make any accusations.

First: load the data (I re-formatted it at bit so R can read it: jonesmannrogfig2c.txt, data1400.dat_.txt ) , perform the principle components reduction and fit a first
model.

> library(lattice)
> d1400 <- read.table('data1400.dat.txt',sep='\t',header=FALSE)
> d1400r <- as.matrix(d1400[,2:23])
> pcomp <- prcomp(na.omit(d1400r))
> plot(pcomp)
> vars <- data.frame(cbind(Year=d1400[,1],d1400r %*% pcomp$rotation),row.names=d1400[,1])
> jones <- read.table('jonesmannrogfig2c.txt',sep='\t',header=TRUE)
> datUnion <- merge(vars,jones,all=TRUE)
> datUnion$avgTemp <- with(datUnion,(NH+CET+Central.Europe+Fennoscandia)/4.0)
> model <- lm(avgTemp ~ PC1 + PC2 + PC3 + PC4 + PC5 ,dat=datUnion)
> summary(model)

Call:
lm(formula = avgTemp ~ PC1 + PC2 + PC3 + PC4 + PC5, data = datUnion)

Residuals:
       Min         1Q     Median         3Q        Max
-0.8811679 -0.2658117  0.0008174  0.2933058  1.0450044 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept)  0.0065252  0.5750696   0.011   0.9910
PC1         -0.0001683  0.0003912  -0.430   0.6679
PC2         -0.0003678  0.0010114  -0.364   0.7168
PC3          0.0003177  0.0014821   0.214   0.8307
PC4          0.0044084  0.0019351   2.278   0.0246 *
PC5          0.0188520  0.0205137   0.919   0.3601
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.4505 on 113 degrees of freedom
  (484 observations deleted due to missingness)
Multiple R-squared: 0.05223,	Adjusted R-squared: 0.01029
F-statistic: 1.245 on 5 and 113 DF,  p-value: 0.2927

We used only 5 principle components as modeling variables, because as is typical of principle component analysis- beyond the first few components the components become vanishingly small and unsuitable to use in modeling (see graph pcomp below).

However, this gave a model with far smaller R-squared than people are reporting, so lets add in a lot of components like everybody else does (bad!).

> model <- lm(avgTemp ~ PC1 + PC2 + PC3 + PC4 + PC5 + PC6 + PC7 + PC8 + PC9 +PC10 +PC11 +PC12 + PC13 ,dat=datUnion)
> summary(model)

Call:
lm(formula = avgTemp ~ PC1 + PC2 + PC3 + PC4 + PC5 + PC6 + PC7 +
    PC8 + PC9 + PC10 + PC11 + PC12 + PC13, data = datUnion)

Residuals:
     Min       1Q   Median       3Q      Max
-0.87249 -0.25951  0.03996  0.25055  0.99039 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept)  7.431e-01  1.424e+00   0.522   0.6028
PC1         -1.796e-04  3.665e-04  -0.490   0.6253
PC2         -4.179e-04  9.759e-04  -0.428   0.6694
PC3          3.306e-05  1.430e-03   0.023   0.9816
PC4          3.416e-03  1.803e-03   1.894   0.0609 .
PC5          4.032e-02  1.978e-02   2.039   0.0440 *
PC6         -3.260e-03  2.660e-02  -0.123   0.9027
PC7         -7.134e-02  3.620e-02  -1.971   0.0514 .
PC8         -1.339e-01  7.895e-02  -1.696   0.0928 .
PC9          7.577e-02  5.734e-02   1.321   0.1892
PC10         2.700e-01  5.878e-02   4.594 1.22e-05 ***
PC11         8.562e-02  6.741e-02   1.270   0.2068
PC12        -8.057e-02  1.053e-01  -0.765   0.4461
PC13        -4.099e-02  1.064e-01  -0.385   0.7008
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.4141 on 105 degrees of freedom
  (484 observations deleted due to missingness)
Multiple R-squared: 0.2558,	Adjusted R-squared: 0.1637
F-statistic: 2.777 on 13 and 105 DF,  p-value: 0.001961

This is a degenerate model that essentially didn’t fit (thought the significance on PC10 component fools the fitter, but PC10 can’t be usable- it is essentially noise). Graphically we can see the fit is not very useful (despite having a little bit of R-squared) by looking at the graph of the fit plotted in the region of fitting. Notice how the fit variance is much smaller than the true data variance even in the region of training data, this is typical of bad regression fits.

> dRange <- datUnion[datUnion$Year>=1856 & datUnion$Year<=1980,]
> xyplot(avgTemp + prediction ~Year,dat=dRange,type='l',auto.key=TRUE)

Now the statement they wanted to make is that the present looks nothing like the past. The past is only available through the fit model so what you would hope is that the model looks like the present and then the model itself separates the past and present. Instead as you see in the graphs above and below this fails two ways: the model looks nothing like the present and the model’s past looks a lot like the model’s present.

> datUnion$prediction <- predict(model,newdata=datUnion)
> xyplot(avgTemp + prediction ~Year,dat=datUnion,type=c('p','smooth'),auto.key=TRUE)

What we could do to falsely drive the conclusion (which itself may or may not be true, it just is not supported by this technique, model or data) is create the infamous graph where we switch from modeled data in the past to actual data in the present and then act surprised that the two did not line up (which they did at no step during the fitting). I don’t have the heart to unify the colors or remove the legend, but here is the graph below:

> datUnion$dinked <- datUnion$prediction
> datUnion$dinked[!is.na(datUnion$avg)] <- NA
> xyplot(avgTemp + dinked ~Year,dat=datUnion,type=c('p','smooth'),auto.key=TRUE)

The reason the blue points look different than the others is they came from the average temperature data instead of the model (where everything else came from). Switching the series is essentially assuming the conclusion that recent past looks very different than the far past.

Essentially this methodology was so poor it could not have illustrated or contradicted recent global warming. There are plenty of warning signs that the model fitting are problematic and the conclusion illustrated in the last graph can not actually be proved or disproved from this data (the proxy variables are too weak to be useful, that is not to say there are not other better proxy variables or modeling techniques). The problems of the presentation are, of course, not essential problems in detecting global warming (which likely is occurring and likely will be a drain on future quality of life) but problems found in a single bad analysis.

Related posts:

1. R annoyances
2. The cranky guide to trying R packages
3. Survive R

I have two areas of concern.

My first concern is: what is going on with the ads on Google search? Are they safe to click on? Check out this ad (from a Google search page):

Where do you think the link would lead to if you clicked on it? First guess: “www.dmb.com/duns” which is further labeled “Official Site.” This sounds like we would directly link to the venerable company “Dun and Bradstreet.” This is not the case. The text and the green site are apparently part of the ad. Hovering the mouse or capturing the link yields the following huge and hard to decipher URL:

http://www.google.com/aclk?sa=L&ai=CwmvMvAVpSsyeDpvusAO4y5D8D5aTi3qAv83oDPXK5f0CEAEguVQoA1Dw0pOr-_____8BYMn2-IbIo6AZyAEBqgQdT9AhbH2fVXmJHnIy-TNNj_HkY7JsaGV106RyaVw&num=1&sig=AGiWqtwX5zxePZHhDkpqJBojcybMzKkFSw&

q=http://track.did-it.com/n%3Flid%3D13619863%26tid%3D3fe34bfe02723%26eng_creative%3D3345771492%26eng_keyword%3Dd-u-n-s%2520number%26eng_placement
%3D%26url%3Dhttp://smallbusiness.dnb.com/webapp/wcs/stores/servlet/SmbHome%3FstoreId%3D10001%26cm_mmc%3DGoogle-_-Adword-_-online-_-d-u-n-s%2520number%26LID%3D13619863

Can you anticipate where this goes? You can see the following three URL fragments inside the URL:

• http://www.google.com/
• http://track.did-it.com/
• http://smallbusiness.dnb.com/

Reading from left to right you can be pretty sure the URL goes to a Google server that uses the rest of the URL as an argument. We can then assume that this Google server performs some bookkeeping and redirects to http://track.did-it.com/ . We can then further assume that track.did-it.com performs some more bookkeeping and redirects to http://smallbusiness.dnb.com/. Notice the first non-Google URL does not match the advertised URL. We don’t know what is encoded into each of these pieces and we are only assuming the track.did-it.com server unpacks the URL from the argument; maybe that is a red herring and it redirects us to somewhere else entirely.

Frankly I have no idea if this really happens. I do know from experience that Google did re-direct to track.did-it.com because my anti-spyware traps caught this, blocked the page load and issued a warning. This sort of thing could be a very irritating late night call from Mom- she gets a web-safety warning (anti-phishing, anti-virus, cookie request or something else) from a site she is not on, she is not loading inclusions from (images, scripts, iframes and so on) and she is not knowingly clicking on.

Is there a finite list of partners that Google allows to re-direct? Can anybody do this? Can I produce and place ads with arbitrary redirections?

My second concern is: do you even need to click to leave the Google search page?

Check out what happens with a fresh copy of Mozilla Firefox (no plugins and nothing added to the browser) when you search for “Bing” on Google:

Let’s look at that pop-up a little closer:

Why would www.bing.com get a chance to set a cookie? Doesn’t that only happen in response to a request from my copy of Firefox? Did some script on the Google search page or some obscure “web acceleration” option in Firefox pre-fetch the content of the first Google link or first Google Ad (triggering the cookie request)? Let’s not worry too much over cause (Google is the major funding source for Firefox), but look for possible effects. Does the destination site (Bing) see traffic from the Google page even if the user never clicked on the link? Something like that would inflate the already huge stature of Google as a traffic source. How much of my site’s “web traffic” comes from phantom clicks (from abandoned searches)? Finally, how safe is it? How much more than the cookie is being loaded? What if the site linked to has malware- is this a no-click infection route?

Some of these subtle features are necessary to support an ad network. But the implementation does not seem minimal. Allowing hosts to differ from ad content and performing a pre-fetch with every search both expose searchers to additional risk.

EDIT:

Found the prefetch. A Firefox tag that Google knows to set (see What is firefox prefetching?)- and sure enough if I perform Google search using Firefox we see:

<link rel="prefetch" href="http://www.bing.com/">

(and I don’t see this tag when using Safari).
So that is the mechanism- still wondering how much risk this is to users. Also wondering how much this skews traffic statistics into Firefox’s and Google’s favor (I haven’t seen the prefetch tag on Yahoo search or Bing search).

Related posts:

1. YAYGDA (Yet Another Yahoo Google Deal Article)
2. Is Search Advertising a Market for Lemons?
3. Public Service Article: JSTOR and other Useful Research Archives

Microsoft has tried retail outlets before. So we have some knowledge of what to expect.

In particular I am thinking of the Microsoft store that was part of the new San Francisco Metreon Mall (opened in June of 1999). At the time Microsoft had no good reason to produce such a store. They were not ready for a major consumer product push and didn’t release the original Xbox until 2001 or the Zune until 2006. This era was also a gap in their consumer operating system release cycle (Windows 98 was already out, Windows ME didn’t come out until Q2 2000 and Windows XP Q3 2001). In 1999 there still were Gateway computer retail stores and Apple Stores were still two years in the future. This Microsoft Store satisfied no discernible need, promoted no major strategy and was deservedly forgotten. From their own press release the purpose of the store was to sell “more than 90 items of apparel, desk accessories, office supplies and other items with the microsoftSF logo” ( see presspass). A fairly low marketing ambition- which predictably failed.

Fast forward to 2009. A Microsoft store could work, if Microsoft were willing to make a good one. I doubt they are willing. Likely they feel hey can easily cut themselves a lucrative slice of retail pie (now that they have major consumer products like Xbox, Zune and Windows Media Center). This will fail- these are all significant products, but together they don’t make for an interesting store or cohesive shopping experience or lifestyle pitch.

Instead, imaging the following store.

• A Microsoft store where a competent sales staff sells major brand PCs (like a Sony, HP, Leveno and so on). But for a small mark-up they wipe out the manufacturer’s defective Windows install and spyware. They install a clean full version of Windows 7 (or Windows XP). A store where they consult with you and which version of Office really is best for you and then sell and install this.

Even I would go there (and I use Apple OSX and Linux almost exclusively). Microsoft is too big an ecosystem to ignore. We need to be able to buy mainstream (and that means Microsoft) PCs, operating systems and software packages at retail (and not just at box stores).

Let me illustrate my point using the last time I purchased a PC. After an hour of trying to deal with Leveno’s atrocious web-site (they really do not want to sell computers retail) I got up and drove to the Metreon Sony Store and purchased a Sony Vaio.

Here is what happened:

• The Sony Vaio could not handle the Vista it came installed with.
• The Sony Vaio came without restore media- making a clean re-install of the operating system impossible.
• The Sony Vaio came laden with all kinds of crapware (trial anti-virus software, Internet poker offers).
• I (later) spent around $400 on Office Professional just to find out I could not uninstall enough of the Sony demo version of Office to get rid of some sort of killing dependence on “Contact Manager” (some sort of enterprise wide service that a home PC would not have).
• I purchased two old copies of XP. I used one to reformat the Vaio and donated that to my father in law for web-surfing. We then used the other copy of XP to host Office Professional (which we really wanted and needed) inside a virtual machine on a hand-me-down Macbook. End of Microsoft Windows (outside of virtual machines) in our household. If Windows can do something good we now have no way of seeing it (since we now only run Windows inside a virtual machine, which is unfair to Windows).

A careful reader would say “this is unfair, it was Sony that was bad not Microsoft.” That is correct. But there really (right now) is no way for an individual consumer to directly deal with Microsoft. Eliminate some of the wall of stupid between me and Microsoft and I probably would use more of their products.

Here is what I (and probably most other consumers would pay big for).

• A Microsoft store that re-sells major brand PCs (Sony, HP so on) that have been opened, re-imaged with a clean OS from Microsoft (not the evil OEMs) and given out with install/restore media.
• Ability to buy a copy of Office Professional that will work on my PC. Here is my proposition. I drop off my PC and $400, next day I come back and take back only the PC and I have a copy of Office Professional that works and isn’t whining about a bunch of unresolved dependencies to servers and services I do not own. I also need to know which version of Office to get (and I know it may cost more than the cheapest one) without reading some impenetrable “enterprise white paper.”
• A store with a staff that lets me safely say “I don’t know Dad, maybe you should take it back in to the Microsoft Store.” I’ll pay. I’d buy training, support, warrantees, read marketing materials and try product demos

Again, a careful reader would point out that all of these activities are labor intensive and going to be very low margin. This too can be fixed. Once Microsoft installers are wasting Microsoft’s time and money (instead of wasting my time and money) the installers will be fixed. Installing Microsoft Office could be come as easy as selecting a new application in Ubuntu (my current gold standard in easy installs).

I would welcome and support a good Microsoft retail effort, I just don’t believe one to be very likely.

Related posts:

1. Must Have Software
2. Something I don’t get about business and bailouts

• Special Agent Michael G. McSwain’s charges
• Mathew Goldstein’s Reuters article
• Zero Hedge blog entry

For me this triggers some strong (and sad) personal memories.

No matter what inappropriate “Robin Hood” intellectual property fantasies you have this (if true) is just wrong. I have never been a huge fan of the ACM Code of Ethics (which does cover this situation, but fails to seriously address much beyond having responsibilities to your employer) but this sort of incident reminds me why computer science needs some approximation of a shared set of ethics that we can try to refer to.

A particularly sad part of the story that attracted my attention was the reliance on “bash history” to try and establish what happened. Attempting to “prove” something using a “bash history” is something I have painful experience with. The “bash history” system is incredibly inadequate even for what it was designed for (caching recent commands). Simply having two shells open can cause non-deterministic overwriting, deletion, clobbering and time disorderings in the history file. Furthermore bash history has no dates, times, directories or any other contextual hints written into it. Finally bash history has no hashes, signatures, nonces, sequence numbers or any other device that helps establish authenticity.

Now for my story. We (by chance) caught somebody walking off with our group’s entire source tree. In the end all we had to go on was the bash history. To hostile eyes bash history is nowhere near what you would call “forensic grade evidence.” Unfortunately for us the theft was intramural, the thief was merely taking the code to another group in the same company to later mine and represent as their own work. At this point even language worked against us- every time we accidentally said something like “our code” (as in the code we produced, not the code we own) we were perceived as being anti-company. Evil prevailed (the thief was promoted) and I looked stupid for working so hard to try to interpret such low-quality evidence. But we live in an objective world- just because you can’t prove something doesn’t mean there isn’t some buried ugly truth.

So what was stolen? Not the code, that moved from one pocket of the corporation that owned it to another pocket of the same corporation. What was stolen was reputation. The thief presumably appeared to out-produce both his old colleagues and his new ones (who don’t have a few absconded person-years of development to draw from). So an apology to anyone who was asked why they could not code as fast as our escaped “genius,” it was certainly not our intent to so equip him. And a larger apology to the rest of the team, sorry we could not prove the misappropriation of your work.

Of course Shakespeare said it much better (from Othello):

Good name in man and woman, dear my lord,
Is the immediate jewel of their souls:
Who steals my purse steals trash; ’t is something, nothing;
’T was mine, ’t is his, and has been slave to thousands;
But he that filches from me my good name
Robs me of that which not enriches him
And makes me poor indeed.

Related posts:

1. Relative returns: a banker versus trader paradox