Descriptive Statistics

Descriptive Statistics

Before you build models or run tests, you need to understand your data at a basic level. Descriptive statistics summarize your data into a few key numbers. Think of it as the vital signs of your dataset — quick, essential diagnostics.

Measures of Central Tendency

These tell you where the "center" of your data is. Mean, median, and mode each answer this question differently — and the right one depends on your data.

import pandas as pd
import numpy as np

salaries = [30000, 35000, 40000, 42000, 150000]

print(f"Mean: {np.mean(salaries)}")
print(f"Median: {np.median(salaries)}")

from scipy import stats
print(f"Mode: {stats.mode([1, 2, 2, 3, 3], keepdims=False).mode}")
    

The mean is sensitive to outliers — notice how that $150K salary skews the average. The median is more robust. Always check both.

Measures of Spread

Knowing the center isn't enough — you need to know how spread out the data is. Standard deviation and variance measure this.

import numpy as np

data = [10, 12, 14, 15, 18, 22, 25]

print(f"Range: {max(data) - min(data)}")
print(f"Variance: {np.var(data, ddof=1):.2f}")
print(f"Std Dev: {np.std(data, ddof=1):.2f}")
    

The ddof=1 parameter uses Bessel's correction for sample variance. Without it, you get population variance. For most data science work, use ddof=1.

Percentiles and Quartiles

Percentiles tell you the value below which a certain percentage of data falls. Quartiles split your data into four equal parts.

import numpy as np

scores = np.random.normal(75, 10, 1000)

print(f"25th percentile: {np.percentile(scores, 25):.1f}")
print(f"50th percentile (median): {np.percentile(scores, 50):.1f}")
print(f"75th percentile: {np.percentile(scores, 75):.1f}")

q1 = np.percentile(scores, 25)
q3 = np.percentile(scores, 75)
iqr = q3 - q1
print(f"IQR: {iqr:.1f}")
    

The Interquartile Range (IQR) is the middle 50% of your data. It's used to detect outliers — values beyond 1.5 * IQR from Q1 or Q3 are considered unusual.

Key Takeaways

• Mean is sensitive to outliers — always check the median too
• Standard deviation measures how spread out data is from the mean
• IQR identifies the middle 50% of your data
• Use ddof=1 for sample variance and standard deviation