dplyrusers who had such a need, and wanted such extensions.
dplyrusers who did not have such a need ("we always know the column names").
dplyrusers who found the then-current fairly complex "underscore" and
lazyevalsystem sufficient for the task.
Needing name substitution is a problem an advanced full-time
R user can solve on their own. However a part-time
R would greatly benefit from a simple, reliable, readable, documented, and comprehensible packaged solution. Continue reading Let’s Have Some Sympathy For The Part-time R User
To illustrate this we will work an example.
I have been writing a lot (too much) on the
tidyeval lately. The reason is: major changes were recently announced. If you are going to use
dplyr well and correctly going forward you may need to understand some of the new issues (if you don’t use
dplyr you can safely skip all of this). I am trying to work out (publicly) how to best incorporate the new methods into:
- real world analyses,
- reusable packages,
- and teaching materials.
I think some of the apparent discomfort on my part comes from my feeling that
dplyr never really gave standard evaluation (SE) a fair chance. In my opinion:
dplyr is based strongly on non-standard evaluation (NSE, originally through
lazyeval and now through
tidyeval) more by the taste and choice than by actual analyst benefit or need.
dplyr isn’t my package, so it isn’t my choice to make; but I can still have an informed opinion, which I will discuss below.
I think a critical aspect of such an extension would be to treat such a notation as syntactic sugar and not insist such a pipe match magrittr semantics, or worse yet give a platform for authors to insert their own preferred ad-hoc semantics. Continue reading In praise of syntactic sugar
replyr stands for REmote PLYing of big data for R.
replyr allows users to work with
Spark or database data similar to how they work with local
data.frames. Some key capability gaps remedied by
- Summarizing data:
- Combining tables:
- Binding tables by row:
- Using the split/apply/combine pattern (
- Pivot/anti-pivot (
- Handle tracking.
- A join controller.
You may have already learned to decompose your local data processing into steps including the above, so retaining such capabilities makes working with
sparklyr much easier. Some of the above capabilities will likely come to the
tidyverse, but the above implementations are build purely on top of
dplyr and are the ones already being vetted and debugged at production scale (I think these will be ironed out and reliable sooner).
dplyr issue 2916.
The following appears to work.
suppressPackageStartupMessages(library("dplyr")) COL <- "homeworld" starwars %>% group_by(.data[[COL]]) %>% head(n=1)
## # A tibble: 1 x 14 ## # Groups: COL  ## name height mass hair_color skin_color eye_color birth_year ## <chr> <int> <dbl> <chr> <chr> <chr> <dbl> ## 1 Luke Skywalker 172 77 blond fair blue 19 ## # ... with 7 more variables: gender <chr>, homeworld <chr>, species <chr>, ## # films <list>, vehicles <list>, starships <list>, COL <chr>
Though notice it reports the grouping is by "
COL", not by "
homeworld". Also the data set now has
14 columns, not the original
13 from the
starwars data set.
In this article we will discuss composing standard-evaluation interfaces (SE: parametric, referentially transparent, or “looks only at values”) and composing non-standard-evaluation interfaces (NSE) in
To use it you must know some of its structure and notation. Here are some details paraphrased from the major
rlang client, the package dplyr:
vignette('programming', package = 'dplyr')).
:=" is needed to make left-hand-side re-mapping possible (adding yet another "more than one assignment type operator running around" notation issue).
!!" substitution requires parenthesis to safely bind (so the notation is actually "
(!! )", not "
- Left-hand-sides of expressions are names or strings, while right-hand-sides are
Here is an absolutely horrible way to confuse yourself and get an inflated reported
R-squared on a simple linear regression model in
We have written about this before, but we found a new twist on the problem (interactions with categorical variable encoding) which we would like to call out here. Continue reading An easy way to accidentally inflate reported R-squared in linear regression models