FeaturedPosted on Categories Administrativia, StatisticsTags , , , , Leave a comment on Going to Strata / Hadoop World 2017 San Jose?

Going to Strata / Hadoop World 2017 San Jose?

Are you attending or considering attending Strata / Hadoop World 2017 San Jose? Are you interested in learning to use R to work with Spark and h2o? Then please consider signing up for my 3 1/2 hour workshop soon. We are about half full now, but I really want to fill the room, while making sure that people who really want to go get in.

Win-Vector LLC is partnering with RStudio to produce and present some awesome material that will allow you to perform data science at scale using R to control Spark and even h2o.

The links to the event are below. To make sure you get to participate please sign up soon!

  • Modeling big data with R, sparklyr, and Apache Spark (by RStudio and Win-Vector LLC)

    03/14/2017 1:30pm – 5:00pm PDT (210 minutes)

    Strata & Hadoop World West, San Jose Convention Center, CA; Room: LL21 C/D

    link

    Win-Vector LLC’s John Mount will teach how to use R to control big data analytics and modeling. In depth training to prepare you to use R, Spark, sparklyr, h2o, and rsparkling.

    This is going to be hands-on exercises with R, sparklyr, and h2o using RStudio Server Pro (generously provided by RStudio!).

    Sponsored by RStudio and
    Win-Vector LLC.

  • Office Hour with John Mount (Win-Vector LLC)

    03/15/2017 2:40pm – 3:20pm PDT (40 minutes)

    Strata & Hadoop World West, San Jose Convention Center, CA; Room: Table B

    link

    Come and ask me questions about data science, machine learning, R, statistics, or whatever you like.

Posted on Categories Opinion, Programming, TutorialsTags , , , 1 Comment on Iteration and closures in R

Iteration and closures in R

I recently read an interesting thread on unexpected behavior in R when creating a list of functions in a loop or iteration. The issue is solved, but I am going to take the liberty to try and re-state and slow down the discussion of the problem (and fix) for clarity.

The issue is: are references or values captured during iteration?

Many users expect values to be captured. Most programming language implementations capture variables or references (leading to strange aliasing issues). It is confusing (especially in R, which pushes so far in the direction of value oriented semantics) and best demonstrated with concrete examples.


NewImage

Please read on for a some of the history and future of this issue. Continue reading Iteration and closures in R

Posted on Categories data science, Practical Data Science, Pragmatic Data Science, Programming, Statistics, TutorialsTags , , , , 15 Comments on The Zero Bug

The Zero Bug

I am going to write about an insidious statistical, data analysis, and presentation fallacy I call “the zero bug” and the habits you need to cultivate to avoid it.


The zero bug

The zero bug

Here is the zero bug in a nutshell: common data aggregation tools often can not “count to zero” from examples, and this causes problems. Please read on for what this means, the consequences, and how to avoid the problem. Continue reading The Zero Bug

Posted on Categories Administrativia, Programming, StatisticsTags , , , , 5 Comments on Announcing the wrapr packge for R

Announcing the wrapr packge for R

Recently Dirk Eddelbuettel pointed out that our R function debugging wrappers would be more convenient if they were available in a low-dependency micro package dedicated to little else. Dirk is a very smart person, and like most R users we are deeply in his debt; so we (Nina Zumel and myself) listened and immediately moved the wrappers into a new micro-package: wrapr.


WrapperImage: Friedensreich Hundertwasser
Continue reading Announcing the wrapr packge for R

Posted on Categories Administrativia, StatisticsTags , , , , , , 1 Comment on My recent BARUG talk: Parametric Programming in R with replyr

My recent BARUG talk: Parametric Programming in R with replyr

I want to share an edited screencast of my rehearsal for my recent San Francisco Bay Area R Users Group talk:



Posted on Categories Programming, Statistics, TutorialsTags , , , , , 1 Comment on Evolving R Tools and Practices

Evolving R Tools and Practices

One of the distinctive features of the R platform is how explicit and user controllable everything is. This allows the style of use of R to evolve fairly rapidly. I will discuss this and end with some new notations, methods, and tools I am nominating for inclusion into your view of the evolving “current best practice style” of working with R. Continue reading Evolving R Tools and Practices

Posted on Categories Exciting Techniques, Statistics, TutorialsTags , , , , , , , 2 Comments on Using the Bizarro Pipe to Debug magrittr Pipelines in R

Using the Bizarro Pipe to Debug magrittr Pipelines in R

I have just finished and released a free new R video lecture demonstrating how to use the “Bizarro pipe” to debug magrittr pipelines. I think R dplyr users will really enjoy it.



Please read on for the link to the video lecture. Continue reading Using the Bizarro Pipe to Debug magrittr Pipelines in R

Posted on Categories Administrativia, StatisticsTags , , , , , , , Leave a comment on Upcoming Win-Vector LLC public speaking engagements

Upcoming Win-Vector LLC public speaking engagements

I am happy to announce a couple of exciting upcoming Win-Vector LLC public speaking engagements.

Hope to see you there!

Posted on Categories Computers, Opinion, Public Service Article, Statistics, TutorialsTags , , , , 12 Comments on Upgrading to macOS Sierra (nee OSX) for R users

Upgrading to macOS Sierra (nee OSX) for R users

A good fraction of R users use Apple computers. Apple machines historically have sat at a sweet spot of convenience, power, and utility:

  • Convenience: Apple machines are available at retail stores, come with purchasable support, and can run a lot of common commercial software.
  • Power: R packages such as parallel and Rcpp work better on top of a Posix environment.
  • Utility: OSX was good at interoperating with the Linux your big data systems are likely running on, and some R packages expect a native operating system supporting a Posix environment (which historically has not been a Microsoft Windows, strength despite claims to the contrary).

Frankly the trade-off is changing:

  • Apple is neglecting its computer hardware and operating system in favor of phones and watches. And (for claimed license prejudice reasons) the lauded OSX/macOS “Unix userland” is woefully out of date (try “bash --version” in an Apple Terminal; it is about 10 years out of date!).
  • Microsoft Windows Unix support is improving (Windows 10 bash is interesting, though R really can’t take advantage of that yet).
  • Linux hardware support is improving (though not fully there for laptops, modern trackpads, touch screens, or even some wireless networking).

Our current R platform remains Apple macOS. But our next purchase is likely a Linux laptop with the addition of a legal copy of Windows inside a virtual machine (for commercial software not available on Linux). It has been a while since Apple last “sparked joy” around here, and if Linux works out we may have a few Apple machines sitting on the curb with paper bags over their heads (Marie Kondo’s advice for humanely disposing of excess inanimate objects that “see”, such as unloved stuffed animals with eyes and laptops with cameras).

IMG 0726

That being said: how does one update an existing Apple machine to macOS Sierra and then restore enough functionality to resume working? Please read on for my notes on the process. Continue reading Upgrading to macOS Sierra (nee OSX) for R users

Posted on Categories math programming, Pragmatic Machine Learning, Statistics, TutorialsTags , , , , , 3 Comments on Why do Decision Trees Work?

Why do Decision Trees Work?

In this article we will discuss the machine learning method called “decision trees”, moving quickly over the usual “how decision trees work” and spending time on “why decision trees work.” We will write from a computational learning theory perspective, and hope this helps make both decision trees and computational learning theory more comprehensible. The goal of this article is to set up terminology so we can state in one or two sentences why decision trees tend to work well in practice.

Continue reading Why do Decision Trees Work?