Many data scientists (and even statisticians) often suffer under one of the following misapprehensions:
- They believe a technique doesn’t work in their current situation (when in fact it does), leading to useless precautions and missed opportunities.
- They believe a technique does work in their current situation (when in fact it does not), leading to failed experiments or incorrect results.
I feel this happens less often if you are working with observable and composable tools of the proper scale. Somewhere between monolithic all in one systems, and ad-hoc one-off coding is a cognitive sweet spot where great work can be done.