Exploratory Data Analysis (EDA)

• Develop an understanding of your data.
• EDA is process where we ask questions by exploring data
• Its fundamentally a creative process

Definitions

• Variable: A quantity, quality or a property that you can measure
• Value: State of variable when you measure it.
• Observation: A set of measurements, several values each associated with a different variable
• Tabular Data: A set of values, each associated with a variable and an observation. Tabular data is tidy if
• Each value is place in its own cell
• Each variable in its own column
• Each observation in its own row
• Variation:
• The tendency of the values of a variable to change from measurement to measurement
• Each measurement will include small amount of error that varies from measurement to measurement
• The best way to understand the pattern f variation is to visualize the distribution of variable’s values
• Types of Variables
• Categorical: If it takes one of small set of values (factors or character vectors)
• Continuous: If it can take any of the infinite set of ordered values (numbers or datetimes)
• Other values:
• Unusual Values:
  • Outliers are observations that are unusual (data points that doesn’t seem to fit the patter)
  • Two Options same as missing values
  • Drop entire row with the unusual values
  • Replace the unusual value with imputation (mean, median, KNN)
• Missing Values: Two Options
  • Drop entire row with the missing values
  • Replace the missing value with imputation (mean, median, KNN)
• COVARIATION
• Variation describes the behaviour within a variable, whereas CoVariation is the tendency of values of two or more variables to vary together in a related way
• Best way to spot covariation is to visualize the relationship between two or more variables
• How depends on type of variables involved
  • Categorical vs Continuous (Frequency/density/box plot)
  • Categorical vs Categorical
  • Continuous vs Continuous (scatter plot)

Hands On EDA

• Load the tidyverse and ggplot2 libraries
• We have a dataset from ggplot2 called as diamonds
• Try to observe the diamonds data and find the basic information and what the varaibles are by reading documentation ?diamonds
  
  
  
  
  
  
  
  
  
  
  
• Exercise: Try finding patterns b/w color and cut and color and clarity