Tutorial T-02 · Unit 1
A Brief History of Artificial Intelligence
The field of AI is more than seventy years old and has been declared dead several times. Understanding why it survived each setback — and what changed in 2012 to make machine learning the dominant approach — is essential context for everything that follows.
01 Learning Objectives
By the end of this tutorial you will be able to
- Outline the major eras in the history of artificial intelligence from the 1940s to the present.
- Define the term AI winter and explain what caused each of the two major winters.
- Describe the shift from symbolic AI to statistical learning, and explain why it happened.
- Identify the three factors that converged in 2012 to trigger the deep learning era.
- Explain why historical context matters for understanding current machine learning practice.
02 Motivating Example
In March 2016, a program called AlphaGo defeated Lee Sedol, one of the strongest Go players in the world, four games to one. The match was watched by over 200 million people. It was widely described as a milestone for artificial intelligence.
Nineteen years earlier, in May 1997, a program called Deep Blue had defeated Garry Kasparov at chess. That was also described as a milestone for artificial intelligence. Two board games. Two victories over world champions. Two headlines that sounded almost identical.
But the two programs worked in fundamentally different ways.
- Deep Blue was built by IBM engineers who hand-coded its evaluation function — assigning values to piece positions, material advantage, king safety, and dozens of other factors. The rules were written by humans. Deep Blue searched an enormous number of positions, but every judgment it made came from a human-designed heuristic.
- AlphaGo was built by researchers at DeepMind who did not tell it how to evaluate positions. They showed it hundreds of thousands of human games and let it learn which moves led to wins. Its evaluation of a board position was not written down anywhere; it was encoded in the parameters of a neural network that had been trained on data.
Two programs. Two victories. Two completely different approaches. And the gap between them — the shift from writing rules to learning them from data — is the central story of AI’s history.
Why history matters
The field has gone through this shift before, in both directions. AI was originally built on hand-coded rules. It abandoned that approach after repeated failures. It returned to learning from data only when three conditions finally converged. Knowing what those conditions were is what this tutorial is about.
03 Concept Explanation
The history of AI is not a smooth progression. It is a sequence of waves, each driven by an optimistic new idea, each followed by a period of disappointment when the idea failed to deliver on its promises. The pattern repeats, and recognising it is useful.
Before the field existed (1943–1956)
The intellectual foundations were laid before anyone called this work “artificial intelligence.” In 1943, Warren McCulloch and Walter Pitts proposed a mathematical model of a neuron — a simple unit that summed its inputs and fired if the sum exceeded a threshold. The model was crude, but it established the idea that computation could be described in terms of networks of neuron-like units. That idea would lie dormant for decades before becoming central.
In 1950, Alan Turing published a short paper titled Computing Machinery and Intelligence. Rather than asking whether machines could think — a question he considered too vague to answer — he proposed a practical substitute: if a human could not distinguish a machine’s responses from another human’s in a text conversation, we should be willing to say the machine could think. This became known as the Turing Test, and it framed the question of machine intelligence in operational terms for the first time.
The Dartmouth Conference and the birth of a field (1956)
In the summer of 1956, a group of researchers gathered at Dartmouth College for a two-month workshop. The proposal, written by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon, contained a sentence that is now famous:
Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.
The conference is generally considered the founding event of artificial intelligence as an academic discipline. The name “artificial intelligence” was coined there. And the participants left with a level of confidence that would look, in hindsight, remarkably misplaced: many believed that a machine matching human intelligence was perhaps a generation away.
Symbolic AI and the first optimism (1956–1974)
The dominant approach for the next two decades was what we now call symbolic AI. The idea was that intelligence consisted of manipulating symbols according to logical rules. If you could represent knowledge as facts and rules, and reason over them, you could build a system that solved problems.
Early successes were real. The Logic Theorist (1956) proved mathematical theorems. The General Problem Solver (1957) could solve simple puzzles by working backwards from a goal. Programs like ELIZA (1966) simulated a psychotherapist convincingly enough to unsettle some users. Funding flowed generously, and optimism was high.
But the approaches did not scale. Programs that worked on toy problems failed on realistic ones. Machine translation, in particular, became a public embarrassment: the U.S. government had funded it heavily, and results were poor. In 1966, a report from the ALPAC committee concluded that machine translation was not viable, and funding collapsed.
In 1973, the British mathematician James Lighthill produced a report for the UK government that was similarly damning. It concluded that AI had failed to deliver on any of its major promises and was not worth continuing to fund at scale. Combined with the earlier ALPAC findings, this triggered what is now called the first AI winter.
The first AI winter (1974–1980)
Funding for AI research in both the United States and the United Kingdom was drastically reduced. For roughly six years, the field contracted. Researchers who remained worked on narrower problems, and the field’s profile in the public imagination fell sharply. Many who had entered the field in the 1960s left it.
Expert systems and the second boom (1980–1987)
AI revived in the 1980s through a narrower, more commercially focused approach: expert systems. These were programs that encoded the knowledge of a human specialist as a set of if-then rules. Unlike the ambitious general reasoning systems of the 1960s, expert systems made no claim to general intelligence. They solved specific, well-bounded problems in medicine, engineering, and finance.
For a while, they worked well enough to be profitable. Companies like DEC and DuPont deployed them internally. Japan launched an ambitious national project, the Fifth Generation Computer Systems, to build a new generation of AI hardware. Investment surged.
But expert systems had a structural weakness: the knowledge had to be extracted from human experts and encoded by hand, which was slow, expensive, and brittle. A system that worked in one hospital might fail in another because the rules were subtly different. Maintaining the rule bases became increasingly difficult as they grew. By the late 1980s, the limitations were clear, and the market for specialised AI hardware collapsed.
The second AI winter (1987–1993)
The second winter was longer and deeper than the first. Research funding contracted again, and public enthusiasm fell further. The term “AI” itself became somewhat disreputable in academic and industrial contexts. Many researchers relabelled their work to avoid it: “machine learning,” “knowledge systems,” and “data mining” all gained currency partly as ways of describing AI work without using the word.
The statistical turn (1993–2012)
Quietly, starting in the late 1980s and accelerating through the 1990s, the field underwent a fundamental shift. Researchers stopped trying to encode human knowledge as rules and started building systems that learned patterns from data instead.
Several things made this possible. Statistical methods for learning from data had been maturing for decades, in fields as diverse as statistics, pattern recognition, and neural network research. Computing power was increasing steadily. And new applications — spam filtering, credit scoring, search ranking — provided both data and demand.
By the 2000s, machine learning had become the dominant approach to AI. The word “AI” itself came back into fashion, partly because the methods now worked well enough to justify the ambitious term. Deep Blue’s victory over Kasparov in 1997 belonged to this era: it was a triumph of carefully engineered search, not of learning, but it happened at a moment when the public was beginning to see AI as credible again.
The deep learning era (2012–present)
In 2012, a neural network called AlexNet won the ImageNet image classification competition by a margin that shocked the field. Its error rate was roughly ten percentage points lower than the next best entry — a gap that, in a competition where improvements of one or two percentage points had been typical, seemed almost impossible.
What made AlexNet work was not a single breakthrough. It was the convergence of three things that had each been developing for years:
- Data. ImageNet, a dataset of over a million labelled images, had been assembled by Fei-Fei Li and colleagues starting in 2006. It provided the scale that neural networks needed.
- Compute. Graphics processing units, originally designed for video games, turned out to be well-suited to the matrix operations at the heart of neural network training. Using GPUs made training large networks practical for the first time.
- Algorithms. A series of incremental improvements — the ReLU activation function, dropout regularisation, better weight initialisation — made deep networks trainable in practice.
None of these three was new in 2012. Neural networks had existed since the 1950s, GPUs since the 1990s, and large datasets since at least the 2000s. What was new was their combination. The field has not looked back.
Since 2012, deep learning has spread from image recognition into speech, translation, recommendation, and eventually into the large language models that define the current public conversation about AI. The pattern of history — enthusiasm, disappointment, revival through a narrower and better-founded approach — has not disappeared, but the current wave has lasted longer and delivered more than any previous one.
04 Visual Illustration
The timeline below shows the seven major eras. The two grey segments are the AI winters — periods when funding and public interest contracted sharply. Notice how narrow they are compared with the periods around them. Each winter lasted a few years; each era of active development lasted much longer.
Reading the timeline
It is worth noting that the timeline is not drawn to scale in terms of impact. The current deep learning era has lasted only about a decade, but it has produced more commercially deployed AI systems than every previous era combined. Conversely, the two winters together span less than fifteen years, but their effect on the field’s direction was enormous.
05 Worked Example
Let us examine the 2012 moment closely, because it illustrates a pattern that recurs throughout AI’s history: a technique that has existed for decades suddenly becomes practical, because the surrounding conditions finally align.
Why did deep learning take off in 2012?
Neural networks were not invented in 2012. The basic idea — layers of simple units that learn from data — had been described in the 1950s and refined repeatedly. The backpropagation algorithm, which makes training multi-layer networks possible, was popularised in the mid-1980s. Yet for twenty-five years, neural networks remained a niche approach that most of the field considered inferior to simpler methods.
The question is: what changed?
- Data. Neural networks have a large number of parameters and need a large number of examples to learn from. Through the 1990s and 2000s, most machine learning datasets contained hundreds or thousands of examples. In 2006, Fei-Fei Li began assembling ImageNet, a dataset eventually containing over 14 million labelled images. When it was released for competition in 2010, it offered a scale that had not existed before.
- Compute. Training a deep neural network involves millions or billions of matrix multiplications. On a CPU, this is prohibitively slow. Graphics processing units, designed to render video games, perform exactly these kinds of parallel numeric operations — thousands of them at once. By the late 2000s, researchers had begun experimenting with GPUs for neural network training. The speedup was often ten to thirty times.
- Algorithms. Several small but crucial improvements made deep networks trainable in practice. The ReLU activation function, which replaced the sigmoid used in earlier networks, avoided the vanishing gradient problem that had limited network depth. Dropout, introduced in 2012, provided a simple way to prevent overfitting. Better weight initialisation strategies reduced the frequency of training failures.
AlexNet combined all three. It was a deep convolutional network, trained on ImageNet, using two consumer GPUs, incorporating ReLU and dropout. Its victory in the 2012 ImageNet competition was not a theoretical breakthrough. It was the first time that decades of theory, suddenly matched with enough data and enough compute, had been shown to work at scale.
Within two years, essentially every competitive entry in the ImageNet competition was a deep neural network. Within five years, deep learning had spread to speech recognition, machine translation, and medical imaging. The pattern was clear: once the conditions were right, the technique that had been waiting in the wings for decades rapidly became the standard.
This pattern — a promising technique waiting for the surrounding conditions to catch up — is common in the history of Computing It is worth recognising, because it suggests that some techniques currently dismissed as impractical may simply be waiting for their own moment.
06 Interactive Activity
Place the event in its era
Below are ten events from the history of AI. For each one, decide which era it belongs to. This is not a memory test — the goal is to develop a feel for the sequence, so that when you encounter a reference to, say, “expert systems” or “the second winter,” you can place it in the broader narrative.
| # | Event | Your answer |
|---|---|---|
| 1 | Alan Turing publishes Computing Machinery and Intelligence. | |
| 2 | The Dartmouth Conference coins the term “artificial intelligence.” | |
| 3 | The Lighthill Report leads to drastic cuts in UK AI funding. | |
| 4 | Japan launches the Fifth Generation Computer Systems project. | |
| 5 | IBM’s Deep Blue defeats Garry Kasparov at chess. | |
| 6 | McCulloch and Pitts propose a mathematical model of a neuron. | |
| 7 | AlexNet wins the ImageNet competition by a large margin. | |
| 8 | The ALPAC report concludes machine translation is not viable. | |
| 9 | Fei-Fei Li begins assembling the ImageNet dataset. | |
| 10 | The term “machine learning” gains currency as researchers avoid saying “AI.” |
Show suggested answers
1. Foundations (1943–1956). Turing’s paper appeared in 1950, before the field had a name.
2. Foundations / Symbolic AI boundary (1956). The Dartmouth Conference is generally considered the founding event of the field.
3. First AI Winter (1974–1980). The Lighthill Report was published in 1973, and its consequences unfolded over the following years.
4. Expert Systems boom (1980–1987). Japan launched the Fifth Generation project in 1982.
5. Statistical Turn (1993–2012). Deep Blue won in 1997. Note that Deep Blue used symbolic search, not learning — but it belongs to the statistical era chronologically.
6. Foundations (1943–1956). The McCulloch-Pitts neuron was published in 1943.
7. Deep Learning era (2012–present). AlexNet won in 2012, the event that opens the current era.
8. First AI Winter (1974–1980). The ALPAC report was published in 1966, contributing to the funding collapse that became the first winter.
9. Statistical Turn (1993–2012). Li began the project around 2006. This is a nice example of an event within one era that set up the next.
10. Second AI Winter (1987–1993). The retreat from the word “AI” is characteristic of the second winter, when the term itself became a liability.
07 Self-Check Questions
Try each question before revealing the answer. If you get one wrong, re-read the section it refers to before continuing.
Question 1
What is an “AI winter”?
Show answer
B. The term refers to periods of reduced funding and public interest following disappointment with earlier promises. The field did not stop — research continued — but its scale and profile contracted sharply.
Question 2
Which best describes the difference between symbolic AI and statistical learning?
Show answer
B. The essential difference is where the knowledge comes from: humans writing rules, or algorithms extracting patterns from examples. This distinction returns in T-03, which examines the shift in more depth.
Question 3
Which of the following was not one of the three factors that converged to enable the 2012 deep learning breakthrough?
Show answer
C. The theoretical result that neural networks can approximate any continuous function — the universal approximation theorem — dates to the late 1980s and was not the limiting factor. What changed in 2012 was the practical combination of data, compute, and engineering improvements.
Question 4
True or false: the field of artificial intelligence has progressed steadily since its founding, with each decade building directly on the last.
Show answer
False. The history of AI is characterised by waves of optimism followed by disappointment, with two major winters. Recognising this pattern is useful context for interpreting current enthusiasm about AI — though the current wave has lasted longer and delivered more than any previous one.
Question 5
Why is it useful to know that neural networks existed for decades before becoming practically successful?
Show answer
B. The recurring pattern is that an idea can be sound but impractical until the ecosystem around it — data, compute, tooling — catches up. This is worth remembering when evaluating current claims about which techniques will or will not work.
11 Summary
Key takeaways from this tutorial
- The field has passed through seven distinct eras since the 1940s, each characterised by a dominant approach and a level of optimism that has not always been justified.
- Two AI winters — roughly 1974–1980 and 1987–1993 — followed periods of unmet promises. Both contracted the field substantially but did not stop it.
- The statistical turn of the 1990s replaced symbolic, rule-based AI with data-driven learning. This is the approach the rest of the course is built on.
- The 2012 breakthrough was not a single discovery but the convergence of three long-developing trends: large datasets, GPU compute, and algorithmic refinements such as ReLU and dropout.
- Historical context matters. Understanding where the current approach came from — and what previous approaches failed to do — is essential for judging what machine learning can and cannot achieve.
12 Further Reading
The following sources expand on the history covered in this tutorial. None are required reading, but the first two in particular are worth returning to as the course progresses.
-
Russell, S., & Norvig, P. (2021). Artificial
Intelligence: A Modern Approach (4th ed.). Pearson. Chapter 1.
The standard AI textbook. Its introductory chapter includes a concise history that overlaps with this tutorial but goes into more depth on the symbolic era. -
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012).
“ImageNet Classification with Deep Convolutional Neural Networks.”
Advances in Neural Information Processing Systems, 25.
The AlexNet paper itself. Dense in places, but the introduction provides a useful first-hand account of what made the breakthrough possible. -
Haenlein, M., & Kaplan, A. (2019). “A Brief
History of Artificial Intelligence: On the Past, Present, and Future of
Artificial Intelligence.” California Management Review,
61(4), 5–14.
A short, accessible overview aimed at a non-specialist audience. Useful as a complement to this tutorial if the material here felt dense. -
MIT Technology Review. “The AI Hype Cycle”
(various articles, ongoing).
A running series that traces the current wave of AI against previous ones. Worth checking occasionally as the field develops.
Next up
T-03 examines the shift from symbolic AI to statistical learning in more depth. The history you have just read provides the context; T-03 explains why the shift happened, and what it means for how we think about building intelligent systems today.