Exploratory data analysis (EDA) is a crucial step in the data analysis process, and while there aren’t necessarily “mistakes” in the strictest sense, there are some common pitfalls or oversights that people can make during EDA in Python. Here are a few mistakes along with corresponding code snippets using the famous iris dataset:
1. Not checking the data types of columns:
It’s important to understand the structure of your dataset and ensure that the data types are correctly interpreted, how many non-null values are there and what is the spread of the data. In the example code below, different pandas methods are used to help in this.
2. Not checking for missing values:
A common mistake occurs when individuals fail to detect and handle missing values appropriately, this can introduce bias or affect the results of the analysis. Here is how you can identify missing values:
3. Not choosing appropriate visuals:
Visualization is a powerful and essential tool in EDA for gaining insights and understanding the distribution of the variables. Not choosing appropriate visualizations for your data can make it difficult for you to interpret the data or give misleading interpretations. The code below shows an example of a correct and incorrect visualization for the iris dataset’s feature sepal_length:
4. Not checking the data for outliers:
Outliers can significantly impact the statistical analysis and modeling and sometimes, individuals can forget to identify outliers when exploring the data. The example code below uses a boxplot() which is a popular visualization used to detect outliers present in a feature.
5. Not exploring feature-to-feature relationships:
Another important step of EDA is to explore the relationships between different features present in the dataset. Overlooking feature interactions can lead to missed insights and inaccurate conclusions. The code below shows the relationship between the sepal_length and petal_width of the flowers using the scatterplot() function.