Logistic Regression

Classification using the sigmoid curve.

Classification with Regression

Don't let the name fool you — Logistic Regression is a classification algorithm, not a regression one. It's used when the output is a category (like yes/no, spam/not spam, cat/dog) rather than a continuous number. It's one of the most widely used algorithms for binary classification.

The Sigmoid Function

Logistic Regression uses the sigmoid function to squash any input into a value between 0 and 1. This output can be interpreted as a probability.


  Sigmoid Function: σ(z) = 1 / (1 + e^(-z))

  Output
  1.0 ┤                          ●●●●●●●●●
      │                       ●●●
  0.8 ┤                     ●●
      │                   ●●
  0.6 ┤                 ●●
      │               ●●
  0.5 ┤─ ─ ─ ─ ─ ─●●─ ─ ─ ─ ─ Decision boundary
      │           ●●
  0.4 ┤         ●●
      │       ●●
  0.2 ┤     ●●
      │  ●●●
  0.0 ┤●●●●●●●●●
      └────────────────────────────────── Input (z)
        -6    -4    -2     0     2     4     6

  Output ≥ 0.5 → Class 1 (e.g., "spam")
  Output < 0.5 → Class 0 (e.g., "not spam")

How It Works

Take the weighted sum of inputs (like linear regression)
Pass it through the sigmoid function
If the output ≥ 0.5, predict class 1; otherwise, predict class 0
Use cross-entropy loss to measure error
Update weights using gradient descent to minimize loss

Logistic Regression vs Linear Regression


  Linear Regression:          Logistic Regression:
  ┌─────────────────┐        ┌─────────────────┐
  │ Input ──► Output │        │ Input ──► 0 or 1 │
  │ (continuous)    │        │ (probability)   │
  │                 │        │                 │
  │ Best for:       │        │ Best for:       │
  │ Predicting price│        │ Predicting class│
  │ Predicting temp │        │ Spam detection  │
  │ Forecasting     │        │ Disease diagnosis│
  └─────────────────┘        └─────────────────┘

Multi-Class Extension

While vanilla logistic regression is binary, it can be extended to multi-class problems using One-vs-Rest (OvR) or Softmax regression (also called Multinomial Logistic Regression). Softmax outputs a probability distribution across all classes.

Why It's Still Popular

Logistic Regression is fast, interpretable, and gives you probabilities, not just hard classifications. It's often the first algorithm to try for any classification problem and serves as a baseline for comparing more complex models.

🧪 Quick Quiz

What activation function is commonly used in Logistic Regression?

← Previous Linear Regression

Next → Decision Trees & Random Forests