Introduction to Machine Learning: Understanding Supervised Learning

2026 GitHub Download Slides

Python NumPy scikit-learn

This tutorial introduces machine learning from first principles, for people who are completely new to it. We'll build intuition with examples and pictures first, then add the math. By the end you'll understand what machine learning is, how data becomes geometry, the difference between classification and regression, and how linear regression learns and gets evaluated.

Linear regression is our focus because it's the foundation. The ideas you learn here (turn data into vectors, measure errors, minimize them) show up in every model you'll meet later, including neural networks.

What Is Machine Learning?

Imagine teaching a computer to recognize your handwriting, recommend the perfect playlist, or catch credit card fraud, without writing a single "if-then" rule. That's machine learning.

At its heart, machine learning is about finding patterns in data. Instead of programming every decision step by step, we give the computer examples and it learns the rules itself.

Think about Spotify. Rather than hard-coding "if someone likes rock, suggest more rock," Spotify looks at millions of listening habits and discovers subtle patterns, including ones no engineer would think to write down.

Turning the Real World into Numbers

Here's the clever part. Computers understand numbers, not songs, images, or text. So we describe each real-world thing with a list of numbers called features. That list is a vector. Think of it as a digital fingerprint.

Two songs described by three features:

Song A: [energy: 8, danceability: 6, acoustic: 2]
Song B: [energy: 3, danceability: 4, acoustic: 9]

Each song is now a point in 3D space. Songs that sound alike have similar vectors, so they sit close together on this invisible map.

Let's do the same with movies, scoring five features from 1 to 10:

Movie	Action	Comedy	Drama	Child-friendly	Romance	Vector
Avengers: Endgame	9	6	7	7	3	[9, 6, 7, 7, 3]
John Wick	10	2	5	2	1	[10, 2, 5, 2, 1]
The Lion King	4	7	6	10	4	[4, 7, 6, 10, 4]
The Notebook	1	3	9	6	10	[1, 3, 9, 6, 10]
Deadpool	9	9	4	1	5	[9, 9, 4, 1, 5]

You can already see patterns. John Wick and Avengers are both high-action, but one is serious and the other funnier. The Lion King and The Notebook are both dramatic, but one is for kids and the other for adults. Deadpool is a rare mix: lots of action, lots of comedy, and not at all child-friendly.

Two Kinds of Questions

Once data is in vector form, machine learning answers two main kinds of question.

Classification: "Which group does this belong to?"

Say we want to sort movies into Family Friendly or Not Family Friendly. We give the computer training examples that have both features and the correct label:

The Lion King [4, 7, 6, 10] → Family Friendly
John Wick [10, 2, 5, 2] → Not Family Friendly

Picture every movie as a dot in a four-dimensional space (action, comedy, drama, child-friendliness). The algorithm asks: where do I draw a boundary so the Family Friendly dots are on one side and the rest are on the other? It's like dividing the space into neighborhoods. When a new movie arrives, the computer checks which side of the boundary it lands on.

The answer is always one of the labels. "Will this movie win an Oscar?" (yes/no) and "Is this an action movie or a comedy?" are classification questions.

Regression: "What's the exact number?"

Now say we want to predict how much a movie will make at the box office. We give the computer each movie's features plus the actual earnings of past movies.

Box office revenue is not one of the feature dimensions. It's the target, the one number we want to predict. Regression learns a mapping that takes a movie's coordinates and tells you where it lands on the revenue scale. Think of it as a mathematical GPS: "if you're at action 9, comedy 6, drama 7, romance 3, you'll end up at this revenue."

"What will this movie's IMDb rating be?" and "How long is this movie?" are regression questions.

Both are supervised learning

Classification and regression are both supervised learning, because we train on examples that include the right answer, like practicing with a teacher who shows you the answers. The model learns from those labeled examples, then predicts answers for new data that has no labels.

A simple way to remember it: classification is sorting your movie library into genre folders; regression is predicting the exact star rating you'd give each movie.

Linear Regression: Finding Relationships with Lines

Linear regression is one of the oldest and most useful models in machine learning, still used everywhere from business forecasting to science. It's the "addition" you learn before "multiplication": once it makes sense, everything else starts to make sense too.

Start with one feature

Suppose we notice that movies with more action tend to earn more. We record a few:

Action level	Box office
3	$400M
6	$700M
8	$1.2B
9	$2.8B

Linear regression draws the best straight line through those points. The line might come out as:

$$\text{Revenue} = -\$500\text{M} + \$300\text{M} \times \text{Action}$$

$300M is the slope. Each extra point of action adds about $300M in revenue.
−$500M is the intercept, where the line starts. Don't worry about negative revenue. It's just the number that puts the line in the right place.

Now predict. A new superhero movie has action level 7:

$$-500\text{M} + 300\text{M} \times 7 = \$1.6\text{B}$$

That's the same $y = mx + b$ you learned in school. Linear regression gives you three things:

Understanding: action level really is related to revenue.
Prediction: you can estimate earnings for a movie that hasn't come out yet.
Strength: you can measure whether action is a strong predictor or a weak one.

The same idea predicts house prices from square footage, sales from ad spend, and grades from study hours.

More features: a mapping between two spaces

Real predictions use many features. With all five movie features:

$$\text{Revenue} = m_1(\text{Action}) + m_2(\text{Comedy}) + m_3(\text{Drama}) + m_4(\text{Child}) + m_5(\text{Romance}) + b$$

We can't picture five dimensions, but the math works the same in any number of dimensions. It helps to think of two separate spaces:

Feature space, where each movie lives as a point like [9, 6, 7, 7, 3].
Output space, where the prediction lives, a single number like $152M.

Revenue isn't inside the feature space. It's what we predict from it. Linear regression learns the mapping from one space to the other.

That mapping is just the set of weights $m = [m_1, \ldots, m_5]$. Suppose training on past movies taught us the weights [8, 3, 2, 5, 1] (in $M per feature point) and a base amount of $10M. To predict Avengers, multiply each feature by its weight and add them up (that's called a dot product), then add the base:

$$[9, 6, 7, 7, 3] \cdot [8, 3, 2, 5, 1] = 72 + 18 + 14 + 35 + 3 = 142$$

$$142 + 10 = \$152\text{M}$$

(These are toy numbers to show the mechanics, not real box office figures.)

import numpy as np

avengers = np.array([9, 6, 7, 7, 3])   # action, comedy, drama, child-friendly, romance
weights  = np.array([8, 3, 2, 5, 1])   # learned: $M per feature point
bias     = 10                          # learned base amount, $M

predicted_revenue = avengers @ weights + bias
print(predicted_revenue)  # 152  ->  $152M

Each weight tells you how much that feature contributes. A weight of 8 on action means each extra action point adds $8M to the prediction. Nobody set these weights by hand. They come from patterns in the data: if action movies consistently earned more, training automatically gives action a bigger weight.

A helpful way to picture the weights is as a lens. Feature space is what you're looking at, output space is the image, and the weights are the lens that bends one into the other. Training is the search for the right lens, the one that focuses feature vectors into accurate predictions.

The same idea scales up enormously. Image pixels (784 numbers) to digit probabilities (10 numbers) is just ten of these mappings learned at once. Neural networks are more elaborate versions of the same thing.

Which Line Is Best?

You could draw infinitely many lines through the data. How do we pick the best one? We need a way to measure how bad a line is.

Measuring error

Let's switch to a simpler example: an ice cream stand predicting cones sold from the temperature. Every prediction can be off. The gap is the error (also called the residual):

$$\text{error} = \text{actual} - \text{predicted}$$

If 50 cones were actually sold and we predicted 45, the error is +5.

Why not just add up the errors?

One day the error is +5, another day it's −5. Add them and you get 0, which looks perfect even though the model was wrong twice. Errors shouldn't cancel out.

Square them first

Instead, square each error and then add them up. This is the Sum of Squared Errors (SSE). +5 becomes 25, −5 also becomes 25, and the total is 50, an honest measure of how far off we were.

Squaring does three useful things:

No cancellation. Positive and negative errors can't hide each other.
Big mistakes count more. An error of 10 becomes 100, four times worse than an error of 5 (25).
The math stays smooth, which makes it easy for a computer to minimize using calculus.

The error landscape

Every possible line (each combination of slope and intercept) has its own SSE. Imagine a 3D landscape: slope on one axis, intercept on another, and SSE as the height. For linear regression this landscape is shaped like a bowl. The bottom of the bowl is the line with the smallest error, the best fit.

Finding that bottom is called optimization, and every machine learning algorithm does some version of it: measure how wrong you are, then make it less wrong.

How the Computer Finds It: Gradient Descent

How does the computer actually get to the bottom of the bowl? It can't try every line; there are infinitely many. Instead it uses gradient descent.

Imagine being dropped blindfolded on a hillside and trying to reach the bottom of the valley. You can't see, but you can feel which way the ground slopes under your feet. So you take a small step downhill, feel again, and repeat.

Start with a guess. For example, slope 0.5 and intercept 0. Calculate the SSE, say 4.2. You're somewhere on the side of the bowl.
Feel the slope. The derivative (gradient) tells you how steep the error landscape is at your position and which direction is downhill, for both the slope and the intercept.
Take a small step downhill. Nudge the intercept up to 0.3 and the SSE drops to 3.7. Nudge the slope up to 0.6 and it drops to 3.1.
Repeat until the ground is flat, meaning the derivatives are close to zero. You're at the bottom, and any further change would make predictions worse. This is called convergence.

The size of each step is the learning rate. Why keep steps small? If they're too big, you overshoot the bottom, land on the other side of the bowl, and bounce around. Steep ground means you're far from the bottom; gentle ground means you're close.

import numpy as np

def fit_line_gd(x, y, lr=0.01, epochs=1000):
    m, b = 0.5, 0.0                      # initial guess
    n = len(x)
    for _ in range(epochs):
        y_hat = m * x + b
        error = y_hat - y
        # gradients of the mean squared error w.r.t. m and b
        dm = (2 / n) * np.sum(error * x)
        db = (2 / n) * np.sum(error)
        # step downhill
        m -= lr * dm
        b -= lr * db
    return m, b

# In practice you'd let scikit-learn solve it for you:
# from sklearn.linear_model import LinearRegression
# model = LinearRegression().fit(X, y); model.coef_, model.intercept_

This is how nearly every model learns: start with some parameters, measure how wrong the predictions are, work out which direction reduces the error, take a small step, and repeat. Linear regression walks a simple landscape with two parameters. A neural network walks one with millions. The principle is identical. It's not magic; it's systematic optimization.

What Can Our Line Explain?

For each prediction, there are three important values:

$y$: the actual value.
$\hat{y}$ ("y-hat"): our prediction.
$\bar{y}$ ("y-bar"): the average of all actual values. This is what you'd guess if you knew nothing about the features.

These split each data point's story into two parts:

Explained: $\hat{y} - \bar{y}$. How much better our prediction is than just guessing the average.
Unexplained: $y - \hat{y}$. The error, the part our model can't account for.

Example. On a 75°F day, the stand actually sold 60 cones. The model predicted 50, and the average day sells 40.

Explained: 50 − 40 = 10 cones. The model knows hot days sell more.
Unexplained: 60 − 50 = 10 cones. Something else helped that day. Maybe it was a holiday, or there was a festival nearby. The model only knows about temperature.

A good model makes the explained part as large as possible and the unexplained part as small as possible.

Evaluating the Model

R-squared

Add up those pieces (squared) across every data point:

SSR (Sum of Squares Regression): the total explained variation.
SSE (Sum of Squares Error): the total unexplained variation.
SST (Sum of Squares Total): all the variation in the data, meaning how much sales differ from the average overall. For a least-squares line, SST = SSR + SSE.

R-squared is the share of the variation the model explains:

$$R^2 = \frac{\text{SSR}}{\text{SST}} = 1 - \frac{\text{SSE}}{\text{SST}}$$

It runs from 0 to 1. An $R^2$ of 0.8 means the model explains 80% of the variation in sales, and 20% is still unexplained. An $R^2$ of 0.3 means 70% is a mystery, so maybe you need more features, like humidity, day of the week, or local events.

Because SST is fixed, explaining more (higher SSR) automatically means less error (lower SSE). That's why minimizing squared error works: it's the same thing as maximizing what the model explains.

R-squared is a handy scorecard for comparing models. If adding humidity raises $R^2$ from 0.6 to 0.8, humidity helped. But keep two limits in mind:

It always goes up when you add features, even useless ones. A complex model can score higher and still predict worse on new data.
It says nothing about the size of the errors. A model can explain 90% of the variation and still be off by hundreds of cones on individual days.

It also only tells you how well the model fits the data it was trained on, not how it will do on new data.

Error-based metrics

These tell you how big the errors actually are, in real units:

Mean Squared Error (MSE): the average squared error. It's in "squared cones," which is hard to interpret.
Root Mean Squared Error (RMSE): the square root of MSE, back in cones. "On average, predictions are off by about 7 cones."
Mean Absolute Error (MAE): the average of the absolute errors. "A typical error is 5 cones, in either direction."

RMSE punishes big errors more than MAE, because it squares them first. If RMSE is much bigger than MAE, your model is making a few large mistakes.

Worked example: ice cream sales

Five days of predictions:

Day	Predicted	Actual	Error (actual − predicted)	Squared
1	50	45	−5	25
2	30	35	+5	25
3	60	70	+10	100
4	40	40	0	0
5	25	40	+15	225

MAE = (5 + 5 + 10 + 0 + 15) / 5 = 7 cones
MSE = (25 + 25 + 100 + 0 + 225) / 5 = 75
RMSE = √75 ≈ 8.7 cones

RMSE is higher than MAE because day 5's 15-cone miss gets heavily penalized when squared.

For R-squared: the average of the actual sales is 46 cones. SSE is 375, and SST (the squared distances of each actual value from 46) is 770. So:

$$R^2 = 1 - \frac{375}{770} \approx 0.51$$

The model explains about half the variation in sales.

Which metric should you use?

R-squared to see how much of the variation the model captures, and to compare feature sets.
RMSE when large mistakes are especially costly.
MAE for the most intuitive "typical error," when all mistakes matter equally.

In practice, report all three. What counts as "good" depends on the problem: an RMSE of 2 cones might be excellent for an ice cream stand, while an RMSE of $50,000 might be terrible for house prices. Always compare against a sensible baseline, like predicting the average.

Key Takeaways

Machine learning finds patterns in data instead of following hand-written rules.
Data becomes vectors, points in a feature space. Once you see data this way, machine learning becomes geometry.
Classification draws boundaries between groups; regression maps features to a number. Both are supervised learning.
Linear regression learns a mapping: weights that turn a feature vector into a prediction.
The best line minimizes the sum of squared errors, and gradient descent finds it by stepping downhill on the error landscape.
Every prediction splits into an explained part and an unexplained part. $R^2$ measures the explained share; RMSE and MAE measure error size.
The same ideas (vectors, mappings, minimizing error) power neural networks, decision trees, and everything else you'll meet next.

Next: Classification Models