Machine Learning Fundamentals

For decades, software meant writing rules: if this, then that. Machine learning flips that on its head. Instead of writing the rules, you show the computer thousands of examples and let it figure out the rules itself. This course explains how that works — and, just as important, when it goes wrong.

Imagine writing software to spot spam email. The rules approach: if it contains "free money", mark as spam. Spammers write "fr3e m0ney" and you are back to square one. You add another rule. They adapt again. You lose.

The machine learning approach: show the computer a million emails already labelled "spam" or "not spam," and let it learn the patterns that separate them — patterns far too subtle and numerous to write by hand. When spammers adapt, you feed in new examples and the model relearns.

Almost all machine learning falls into three families, and the difference comes down to what the computer is shown.

Supervised learning is by far the most common. You give it labelled examples — photos tagged "dog" or "cat," houses with their sale prices — and it learns to predict the label. Predicting a category (dog/cat) is called classification; predicting a number (a price) is called regression.

Unsupervised learning gets no labels. You hand it your customers and say "find natural groups." It might discover that you actually have three kinds of customer — something nobody told it to look for. This is how "customers like you also bought…" begins.

Reinforcement learning learns by doing. It tries an action, gets a reward or penalty, and gradually learns a strategy that earns the most reward. It is how programs master games — and, notably, part of how modern chatbots are tuned to be helpful (more on that later in the track).

At heart, training a model means finding the line (or curve, or boundary) that best fits your examples. Picture house prices: bigger houses cost more. Plot size against price, and the model draws the line that best runs through the dots. Now, given a new house's size, it reads the price off the line.

  price
    |                         •
    |                    •  /
    |                •   /  <- the line the model learned
    |            •     /
    |        •      /
    |    •       /
    |________/_______________ size
              ^ new house: find its size, read the price off the line

For classification, instead of a line through the dots, the model draws a line between the groups — a boundary with "dogs" on one side and "cats" on the other. Fancier models draw wiggly boundaries that separate complicated groups. But the idea never changes: fit the examples, then use the fit to judge new cases.

How does it find the best line? Exactly the "walking downhill" idea from the maths course — it starts with a bad line and nudges it, step by step, until the error is as small as it can be.

Here is the most important warning in all of machine learning. A model can do too well on the examples it studied — by memorising them instead of learning the real pattern. This is called overfitting, and it is the number one reason AI projects disappoint in the real world.

An analogy: a student who memorises the exact answers to last year's exam will ace a re-run of that exam — and fail this year's, because they learned the answers, not the subject. An overfit model is that student.

  UNDERFIT          GOOD FIT           OVERFIT
  (too simple)      (the pattern)      (memorised noise)

   •  •              •  •                •__•
  ____•___          __/‾‾\__            _/‾\_•/‾\_
  •     •          •/      \•          •/ \•/    \•
  (misses it)     (smooth, general)   (wiggly, brittle)

The cure is simple to state: always test the model on data it has never seen. You hold back some examples, train on the rest, then check the held-back set. If it does well on examples it studied but badly on the held-back ones — it overfit. This hold-back check (called validation) is non-negotiable. A model is only as good as its performance on data it has never seen.

"Accuracy" sounds like the obvious way to judge a model — what fraction did it get right? But accuracy can lie, badly. Suppose 1 in 1,000 transactions is fraud. A lazy model that says "never fraud" is 99.9% accurate — and completely useless, because it never catches a single fraud.

So we use sharper measures. The two that matter most:

• Precision — when the model raises an alarm, how often is it right? High precision = few false alarms.
• Recall — of all the real cases, how many did the model catch? High recall = few misses.

There is almost always a trade-off. Catch more fraud (high recall) and you will also flag more innocent transactions (lower precision). Which matters more depends entirely on the job: