Language is messy, ambiguous, and deeply human — which is exactly why teaching computers to handle it was one of AI's hardest, longest quests. This course tells that story, from the clumsy early methods to today's language models, and along the way gives you the fundamentals that still matter even in the LLM era.
- Understand why human language is so hard for computers
- Trace NLP's evolution: counting words → meaning → context → LLMs
- Grasp embeddings — turning words into meaning you can measure
- Know the everyday NLP tasks (classification, entity extraction) still in wide use
Why language is so hard
To a computer, "bank" in "river bank" and "savings bank" is the same string — yet the meanings are opposite worlds. Human language is full of this: ambiguity, sarcasm, context, idioms ("it's raining cats and dogs"), and meaning that depends on who is speaking and when.
Early systems tried to handle language with hand-written grammar rules, and drowned — there are too many exceptions, too much context, too much that humans "just know." This is the same lesson as the ML course: when the rules are endless, learn from examples instead. NLP's whole history is the move from rules to learning.
- Language is ambiguous and context-dependent — the same word can mean opposite things
- Hand-written grammar rules can't keep up with human language
- NLP advanced by learning from examples instead of coding rules
From counting words to capturing meaning
The first practical trick was simple: count the words. Represent a document by how often each word appears (a "bag of words"). Spam filters worked this way — emails heavy with "free," "winner," "click" scored as spam. Crude, but useful, and still fine for many tasks today.
The problem: counting words ignores meaning. "Great" and "excellent" are treated as totally unrelated, and word order vanishes entirely. The breakthrough was the embedding — turning each word into a vector (from the Maths course) positioned so that words with similar meanings sit near each other.
This was huge: meaning became something you could measure with distance. "Find similar documents," "recommend related articles," and semantic search all became possible. Embeddings are still everywhere today — they are exactly what powers the retrieval step in RAG.
- Bag-of-words (counting) is crude but still useful for simple tasks like spam
- Embeddings turn words into vectors where similar meanings sit close
- This made meaning measurable — powering search, recommendations, and RAG
- Why does counting words fail to capture that "great" and "excellent" are similar?
- In one sentence, what does an embedding do for a word?
The everyday NLP tasks still in use
Long before chatbots, NLP quietly powered features you use daily — and these still run on simpler, cheaper models where an LLM would be overkill:
- Text classification — sorting text into categories. Spam vs. not, positive vs. negative review (sentiment analysis), routing a support ticket to the right team.
- Named entity recognition — pulling out the things in text: names, dates, places, amounts. ("Invoice #4471 from Acme Ltd dated 3 March for £2,000" → company, date, amount.)
- Intent detection — figuring out what a user wants ("book a flight" vs. "cancel a flight") — the backbone of older voice assistants.
- Classification, entity extraction, and intent detection are the workhorse NLP tasks
- They power spam filters, sentiment analysis, ticket routing, and form-filling
- For high-volume, narrow tasks, small NLP models often beat an LLM on speed and cost
The leap to language models
The old methods had a ceiling. Bag-of-words ignored order; early embeddings gave each word one fixed meaning (so "bank" could not be both river and money). The missing piece was context — understanding each word in light of the words around it.
That is precisely what attention delivered (the course earlier in this track). Modern language models give each word a meaning that shifts with its sentence, so "river bank" and "savings bank" finally land in different places. That single advance turned NLP from a box of separate tools — one for classification, one for translation, one for summarising — into a single model that can do all of them.
- Old methods lacked context: word order and shifting meaning were lost
- Attention gave words context-dependent meaning — the key NLP breakthrough
- LLMs unified the once-separate NLP tasks into one general model
- Give one NLP task where a small classic model still beats an LLM, and why.
- What did "context" add that older embeddings lacked, and which idea delivered it?