Administrativia – Win-Vector Blog

September 17, 2018September 18, 2018John MountAdministrativia, Exciting Techniques, Programmingdot arrow, dot pipe, R, wrapr4 Comments

Dot-Pipe Paper Accepted by the R Journal!!!

We are thrilled to announce our (my and Nina Zumel’s) paper on the dot-pipe has been accepted by the R-Journal!

Continue reading Dot-Pipe Paper Accepted by the R Journal!!!

August 19, 2018August 19, 2018John MountAdministrativia, Practical Data Science, StatisticsPractical Data Science, Practical Data Science with R, R1 Comment

More Practical Data Science with R Book News

Some more Practical Data Science with R news.

Practical Data Science with R is the book we wish we had when we started in data science. Practical Data Science with R, Second Edition is the revision of that book with the packages we wish had been available at that time (in particular vtreat, cdata, and wrapr). A second edition also lets us also correct some omissions, such as not demonstrating data.table.

For your part: please help us get the word out about this book. Practical Data Science with R, Second Edition, R in Action, Second Edition, and Think Like a Data Scientist are Manning’s August 20th 2018 “Deal of the Day” (use code dotd082018au at https://www.manning.com/dotd).

For our part we are busy revising chapters and setting up a new Github repository for examples and code and other reader resources.

August 15, 2018September 12, 2018Nina ZumelAdministrativia, data science, Practical Data Science, Pragmatic Data Science, StatisticsPractical Data Science with R, R4 Comments

Announcing Practical Data Science with R, 2nd Edition

We are pleased and excited to announce that we are working on a second edition of Practical Data Science with R!

Continue reading Announcing Practical Data Science with R, 2nd Edition

July 11, 2018July 11, 2018John MountAdministrativia, data science, Opinion, Practical Data Science, StatisticsR, R and big data, rqdatatable, rquery, speaking

John Mount speaking on rquery and rqdatatable

rquery and rqdatatable are new R packages for data wrangling; either at scale (in databases, or big data systems such as Apache Spark), or in-memory. The packages speed up both execution (through optimizations) and development (though a good mental model and up-front error checking) for data wrangling tasks.

Win-Vector LLC‘s John Mount will be speaking on the rquery and rqdatatable packages at the The East Bay R Language Beginners Group Tuesday, August 7, 2018 (Oakland, CA).

Continue reading John Mount speaking on rquery and rqdatatable

June 20, 2018June 20, 2018John MountAdministrativia, data science, StatisticsR, spark, vtreat, y-aware methods

Big News: vtreat 1.2.0 is Available on CRAN, and it is now Big Data Capable

We here at Win-Vector LLC have some really big news we would please like the R-community’s help sharing.

vtreat version 1.2.0 is now available on CRAN, and this version of vtreat can now implement its data cleaning and preparation steps on databases and big data systems such as Apache Spark.

vtreat is a very complete and rigorous tool for preparing messy real world data for supervised machine-learning tasks. It implements a technique we call “safe y-aware processing” using cross-validation or stacking techniques. It is very easy to use: you show it some data and it designs a data transform for you.

Thanks to the rquery package, this data preparation transform can now be directly applied to databases, or big data systems such as PostgreSQL, Amazon RedShift, Apache Spark, or Google BigQuery. Or, thanks to the data.table and rqdatatable packages, even fast large in-memory transforms are possible.

We have some basic examples of the new vtreat capabilities here and here.

June 13, 2018John MountAdministrativia, Coding, ProgrammingR, wrapr1 Comment

wrapr 1.5.0 available on CRAN

The R package wrapr 1.5.0 is now available on CRAN.

wrapr includes a lot of tools for writing better R code:

let() (let block)
%.>% (dot arrow pipe)
build_frame() / draw_frame() ( data.frame builders and formatters )
qc() (quoting concatenate)
:= (named map builder)
%?% (coalesce) NEW!
%.|% (reduce/expand args) NEW!
uniques() (safe unique() replacement) NEW!
partition_tables() / execute_parallel() NEW!
DebugFnW() (function debug wrappers)
λ() (anonymous function builder)

I’ll be writing articles on a number of the new capabilities. For now I just leave you with the nifty operator coalesce notation.

Continue reading wrapr 1.5.0 available on CRAN

May 10, 2018John MountAdministrativia, Opinion, Programming, StatisticsBig Data, data science, R, R and big data, rquery, SQL2 Comments

rquery: SQL from R

My BARUG rquery talk went very well, thank you very much to the attendees for being an attentive and generous audience.

(John teaching rquery at BARUG, photo credit: Timothy Liu)

I am now looking for invitations to give a streamlined version of this talk privately to groups using R who want to work with SQL (with databases such as PostgreSQL or big data systems such as Apache Spark). rquery has a number of features that greatly improve team productivity in this environment (strong separation of concerns, strong error checking, high usability, specific debugging features, and high performance queries).

If your group is in the San Francisco Bay Area and using R to work with a SQL accessible data source, please reach out to me at jmount@win-vector.com, I would be honored to show your team how to speed up their project and lower development costs with rquery. If you are a big data vendor and some of your clients use R, I am especially interested in getting in touch: our system can help R users start working with your installation.