The Pitfalls of Data Dredging: Why It's Essential to Perform Proper EDA

In today's era of big data, it's not uncommon to come across studies that seem to find significance in almost anything they care to look for. This phenomenon is often referreds as 'data dredging,' a term used somewhat pejoratively by statisticians and methodologists who recognize its potential pitfalls.

Data dredding occurs when researchers throw vast amounts of data at statistical analyses, hoping against hope that something will emerge that appears interesting or publishable. Unfortunately, the more variables (also known as predictor variables) you fish for with your net cast into a large sea of outcomes (or dependent variable), the greater becomes likelihood that eventually, you'll catch something – even if it's just by chance alone!

This issue is especially prevalent in fields such as epidemiology where researchers often have access to massive datasets containing information on numerous risk factors and health outcomes. It's not uncommon for these studies to find associations between seemingly unrelated variables simply because of the sheer volume being examined.

One classic example is when a researcher looks at millions of people over several years, they might discover that those who eat breakfast every morning are 10% less likely to develop heart disease compared to their counterparts who skip brekkie altogether! Sounds impressive right? Except wait... what if we also looked at how many times per week these same individuals did jumping jacks before bed and found out doing exactly three sets of burpees each night resulted in an astonishing reduction rate of 50% fewer colds throughout winter months?!

Of course, correlation does not necessarily imply causation here; there could be plenty other factors at play besides just the breakfast-eating or exercise habits themselves (e.g., perhaps only healthier people tend towards these lifestyle choices). However, because researchers have so many variables to choose from and can easily throw statistical tests like spaghetti against walls hoping something sticks – guess what? Sometimes it does!

This is why proper Exploratory Data Analysis (EDA) before diving headfirst into any sort of formal hypothesis testing or regression analysis is absolutely crucial. EDA involves examining your data graphically, checking for outliers and influential points, transforming variables if necessary to meet assumptions of normality and equal variance – essentially getting a feel for what you're working with.

It's only once we've done our due diligence in this regard that we can have confidence in any conclusions drawn from subsequent analyses. Anything less risks leading us down the path of data dredging, where spurious associations masquerading as real relationships do more harm than good to both science and society at large.

In conclusion: next time you come across a study claiming some incredible finding based on massive datasets with gazillions of variables being thrown around willy-nilly without so much as an eyebrow raised towards issues like multiple testing correction or confounding variable control – take it all with a healthy dose of skepticism! Data dredging may seem sexy at first glance but trust me folks - in reality, there's usually nothing more boring than chasing down another dead-end hypothesis based on methodological sloppiness rather than genuine scientific curiosity.

Data dredging eda