Here is very interesting presentation from Andy Pulkstenis from State farm on Analytics Pitfalls to avoid.
Andy is CapitalOne Alumni. Over the years I have come to have deep respect for CapitalOne Alums. They believe in meritocracy of ideas and harness collective wisdom where status quo is challenged and money is re-imagined.
Here is the essence of the talk that I could gather.
1. Data Science and Machine Learning is not new. There are roots that go all the way to 1952. However, open source, cloud computing, executive attention is taking this to new levels.
2. Thinking that big data sets and black box algorithms can solve all the problems is naive. Very similar to what we have been hearing for decades about technology automation, if not used correctly they can have compounding impact and make some of the problems worse. Some of the common flaws are overfitting of models or lack of proper validation, correlation vs causation, blind spots in data, Simpson's paradox, etc.
3. Finally some of the pitfalls associated with how organizations are structured to drive the analytics journey.