A History of Artificial Intelligence and Deep Learning

Abstract

Artificial intelligence has a history spanning seven decades, but its present form is almost entirely the product of one tradition within it: neural networks, now called deep learning. This essay traces that story from its origins in the 1950s through to the large language models and generative systems of today. It is a history of an approach that was twice dismissed as a dead end, survived in the margins of the field, and eventually came to absorb almost everything else within artificial intelligence.

Five Traditions, One Winner

The history of artificial intelligence is not a single continuous story. It consists of several intellectual traditions that developed partly independently, competed for attention and funding, and periodically merged or diverged. Five strands are worth naming at the outset, because all five contributed to what exists today, though not equally.

The first is symbolic AI: search, planning, logic, formal knowledge representation, and expert systems. The second is statistical machine learning: classification, probabilistic modelling, kernel methods, decision trees, and statistical natural-language processing. The third is neural and connectionist learning: perceptrons, multilayer networks, convolutional and recurrent architectures, and modern deep learning. The fourth is adaptive agents: reinforcement learning, control theory, robotics, and game playing. The fifth is enabling infrastructure: increasing computational power, digital data, benchmark competitions, software frameworks, and industrial investment.

This essay is concerned primarily with the third strand. The reason is simple: it won. Deep learning is now so thoroughly dominant that the term “artificial intelligence” is used by most people to refer exclusively to it. The other traditions have not disappeared, but they either operate inside systems whose central component is a deep neural network, or they have been pushed to the margins of a field they once dominated. Understanding how this happened requires following the neural and connectionist tradition from its earliest days, through its periods of marginalisation and obscurity, to its eventual ascendancy.

Before Dartmouth: 1930s to 1955

Several developments laid the intellectual foundations for artificial intelligence before the field had a name.

Warren McCulloch and Walter Pitts published a paper in 1943 showing that networks of highly simplified, idealised neurons could compute any logical function. This was not an engineering proposal but a mathematical result, and it was as much a contribution to neuroscience and logic as to anything resembling computer science. The significance was theoretical: it established that the kind of processing thought to occur in biological neural systems was, in principle, computationally tractable. This paper is the genuine starting point of the neural network tradition, predating the Dartmouth conference by thirteen years.

Donald Hebb, a Canadian psychologist, proposed in 1949 that learning resulted from changes in the strength of synaptic connections between neurons. When two neurons fire together repeatedly, the connection between them is strengthened. This idea, later compressed into the phrase “neurons that fire together wire together,” became an important conceptual basis for learning rules in artificial neural networks.

Alan Turing’s 1950 paper “Computing Machinery and Intelligence” reframed the philosophical question of whether machines could think as a behavioural question. Turing did not claim that machines could think, but he proposed an operational criterion: if a machine could sustain a conversation indistinguishable from a human one, that would be sufficient evidence of intelligence for practical purposes. The paper also anticipated nearly every objection that has since been raised against machine intelligence and disposed of most of them efficiently.

Norbert Wiener’s work on cybernetics, published in 1948, contributed a framework for thinking about feedback, control, and communication in biological and mechanical systems. Cybernetics was particularly concerned with systems that continuously interacted with and adapted to their environments. Its influence on early AI researchers was substantial, though the relationship was not always harmonious, and John McCarthy later coined the term “artificial intelligence” partly to distinguish his programme from Wiener’s.

These developments established three enduring views of what intelligence might be: intelligence as symbolic computation, intelligence as adaptive control, and intelligence as an emergent property of networks of simple processing units. The tension between the first and third of these views would drive the history of the field for the next fifty years.

The Dartmouth Conference and Two Traditions

The term “artificial intelligence” was introduced in a 1955 proposal written by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon. The resulting Dartmouth Summer Research Project on Artificial Intelligence took place in 1956. McCarthy’s framing was deliberately ambitious: the conjecture was that every aspect of learning and intelligence could, in principle, be described sufficiently precisely for a machine to simulate it.

The conference itself was loosely structured. It ran for several weeks and participants came and went. It did not produce a unified research programme or a set of agreed methods. What it did was gather in one place a group of people who would go on to define the field.

Allen Newell and Herbert Simon were there with a working programme, the Logic Theorist, which could prove theorems from Whitehead and Russell’s Principia Mathematica. They continued this work with the General Problem Solver, which attempted to operationalise general-purpose reasoning through a technique called means-ends analysis. Their broader theoretical position became known as the physical symbol system hypothesis: the claim that intelligence is the manipulation of symbols according to formal rules, and that any physical system doing this is therefore capable of intelligent action. This was the foundational claim of symbolic AI.

The neural network tradition was also present at Dartmouth, but it was not the dominant programme. The symbolic approach was more immediately productive and more aligned with the computational resources available. Two traditions were therefore established from the very beginning of the field, and they would be in tension throughout its subsequent history.

The Perceptron

Frank Rosenblatt, a psychologist working at the Cornell Aeronautical Laboratory in Buffalo, introduced the perceptron in 1957. The laboratory conducted military and aviation research, and Rosenblatt’s work was funded substantially by the US Office of Naval Research, which was interested in automatic pattern recognition for surveillance and reconnaissance applications.

The perceptron was a simple computational model of a neuron. It took a set of inputs, multiplied each by a learned weight, summed the results, and produced a binary output depending on whether the sum exceeded a threshold. Crucially, Rosenblatt also proposed a learning rule: if the perceptron made an error, adjust the weights in the direction that would have produced the correct output. He proved that if a correct set of weights existed, the learning rule would find it in finite time.

The New York Times covered the perceptron in 1958 in terms that seem, in retrospect, reckless. The Navy was quoted announcing expectations of a machine that would walk, talk, see, write, reproduce itself, and be conscious of its existence. Rosenblatt himself was more measured in his technical writing, but the coverage created expectations that would later make the backlash sharper. Rosenblatt eventually moved to the Ithaca campus as a faculty member in the Department of Neurobiology and Behavior, where he remained until his death in a sailing accident in 1971, aged 43.

The perceptron attracted considerable attention because it demonstrated that a machine could learn rather than merely execute explicitly programmed rules. This was a conceptually important distinction. Rather than encoding knowledge by hand, as symbolic AI required, the perceptron adjusted its own internal parameters from examples. That idea of learning from data would prove to be the defining characteristic of the tradition that eventually won.

The 1960s: Optimism and Microworlds

The early 1960s were a period of genuine excitement and, in retrospect, considerable overconfidence. Programs were being written that could solve algebra problems, prove geometric theorems, and play checkers at a reasonable level. Researchers made predictions about machine translation and general AI being solved within a decade or two that proved badly wrong.

John McCarthy created Lisp, which became the characteristic programming language of AI research. Newell and Simon continued their work on heuristic problem solving. McCarthy investigated formal representations of common-sense knowledge. Automated theorem proving, planning, and symbolic mathematics all advanced.

Much research took place in restricted artificial environments later called microworlds. Joseph Weizenbaum created ELIZA between 1964 and 1966, a programme that simulated a psychotherapist by recognising patterns in the user’s input and reflecting them back as questions. It had no understanding of what it was reading. Weizenbaum was disturbed, not pleased, by how readily people attributed genuine understanding and empathy to it. His point was precisely that the appearance of intelligence was easily manufactured and easily mistaken for the real thing.

Terry Winograd built SHRDLU around 1968 to 1972. SHRDLU could parse and respond to natural language instructions about a simulated microworld of coloured blocks, and it did so with genuine linguistic machinery rather than pattern matching. Its limitation was that it could not generalise beyond its narrow domain: the linguistic understanding it appeared to have was entirely dependent on the constrained world it inhabited.

Seymour Papert’s 1966 Summer Vision Project proposed that a group of students construct significant components of a visual-recognition system over a single summer. The project reflected the period’s confidence that perception could be decomposed into manageable subproblems and solved relatively quickly. Computer vision would prove to be one of the hardest problems in the field, and one that would eventually be definitively solved not by hand-engineered approaches but by convolutional neural networks.

Minsky, Papert, and the First AI Winter

In 1969 Minsky and Papert published Perceptrons, a rigorous mathematical analysis of what single-layer perceptrons could and could not compute. The central result was that perceptrons could only solve linearly separable problems: they could not correctly compute the exclusive-or function, a basic logical operation. The proof was clean and the result was correct.

The book was influential beyond what the mathematics strictly warranted. Minsky and Papert did note, in passing, that multilayer networks might overcome these limitations, but they expressed scepticism about whether training methods for such networks could be developed and did not investigate the question seriously. They did not prove that multilayer neural networks were incapable of solving such problems. Multilayer networks could represent exclusive-or and many other nonlinear functions. The practical difficulty was that no reliable and computationally feasible method for training their hidden layers was then widely available.

The effect on research and funding was disproportionate to the actual argument. Neural network research declined sharply through the 1970s, partly because of the book and partly because funding agencies including DARPA had grown frustrated with the gap between AI’s promises and its deliverables. A critical report on machine translation, the ALPAC report of 1966, had already triggered substantial funding cuts in that area. The Lighthill report of 1973, commissioned by the British Science Research Council, produced a similarly damaging reassessment of AI research in the United Kingdom. The combination of failed promises and pointed criticism produced what is now called the first AI winter.

The symbolic AI programme was not similarly affected. Research on search, theorem proving, and knowledge representation continued through the 1970s and became better organised and better funded. The neural network tradition migrated to the margins, surviving primarily in psychology, cognitive science, and neuroscience, where the question of whether network models could explain human cognition was of direct relevance.

Expert Systems and Knowledge Engineering

The dominant paradigm of the 1970s and 1980s was the expert system. The idea was that the bottleneck in building intelligent programmes was knowledge: if you could represent the knowledge of a human expert in a formal, machine-readable way, the programme could apply that knowledge systematically in ways no human could match for speed and consistency.

These systems encoded knowledge as production rules of the form “if X and Y then Z,” and used an inference engine to chain rules together. DENDRAL, developed at Stanford from the mid-1960s onward, could identify the molecular structure of organic compounds from mass spectrometry data. MYCIN, also from Stanford, in the early 1970s, could diagnose bacterial infections and recommend antibiotics with a performance that compared favourably with human specialists in controlled studies. DEC’s XCON system, which configured VAX computer orders, reportedly saved the company tens of millions of dollars annually.

The 1980s saw commercial investment in expert systems at a scale that had not previously been directed at AI. Governments, particularly Japan with its Fifth Generation Computer project, invested heavily. Specialist companies sold expert-system software, knowledge-engineering services, and Lisp machines designed for AI development.

The limitations eventually became apparent. Expert systems were brittle outside the situations anticipated by their designers. The process of extracting knowledge from human experts and encoding it by hand was called knowledge engineering, and its difficulty was systematically underestimated. Large rule bases were expensive to construct, verify, and maintain. Cheaper general-purpose computers also displaced specialised Lisp machines. The commercial collapse produced a second AI winter beginning in the late 1980s, and the term “artificial intelligence” became professionally problematic for a decade.

Judea Pearl’s work on probabilistic reasoning, which culminated in his 1988 book Probabilistic Reasoning in Intelligent Systems, offered a more principled alternative to rule-based systems. Rather than encoding rules, Pearl’s Bayesian networks represented variables and their conditional probabilistic relationships in a graphical structure from which probabilities could be computed efficiently. This statistical approach to reasoning under uncertainty was not initially mainstream, but it proved enormously influential and gave rise to the entire field of probabilistic graphical models.

Connectionism Beyond the Mainstream

Neural network research had not disappeared during the first AI winter. It had migrated to the margins of computer science and was being pursued more actively in psychology, neuroscience, and cognitive science.

A group centred on David Rumelhart and James McClelland at the University of California, San Diego, with Geoffrey Hinton as a central contributor, became the nucleus of what was called connectionism or parallel distributed processing. Connectionist researchers rejected the idea that cognition necessarily depended on explicit symbolic rules. They proposed instead that knowledge could be distributed across patterns of connection weights and that cognitive behaviour could emerge from the collective activity of many simple processing units.

Hinton had studied experimental psychology at Cambridge and completed a PhD in artificial intelligence at Edinburgh in 1978, supervised by Christopher Longuet-Higgins. Edinburgh was an environment where connectionist ideas had survived the post-Minsky-Papert cold somewhat better than elsewhere. He subsequently worked at Sussex and the University of California, San Diego, before joining Carnegie Mellon and later the University of Toronto, which became the base for his most important later work.

Important technical developments during this period included Hopfield networks, self-organising maps, and Boltzmann machines. These were models of how networks of simple units could store memories, organise input patterns, or learn probabilistic distributions. Terry Sejnowski, whose work with Hinton on Boltzmann machines connected the technical programme to neuroscience, was based at the Salk Institute. The psychological and philosophical implications of the connectionist approach were as important as the technical ones: the argument was that cognition itself might be better understood as the collective behaviour of many simple units rather than as the execution of explicit symbolic rules.

Backpropagation and the Connectionist Revival

The key technical development that made connectionism scientifically productive was backpropagation, a method for computing how much each weight in a multilayer network contributed to the output error.

The algorithm had been discovered independently several times. Paul Werbos derived it in his 1974 PhD thesis. Seppo Linnainmaa had described related ideas earlier. But the paper that reached and persuaded the field was published by Rumelhart, Hinton, and Ronald Williams in Nature in 1986. The same year, Rumelhart and McClelland edited the two-volume Parallel Distributed Processing, which became the founding document of the connectionist programme.

Backpropagation made it possible to train multilayer networks by computing gradients through the chain rule and adjusting weights accordingly. This was the technical answer to the limitation Minsky and Papert had identified: hidden units in a multilayer network could now learn their own internal representations rather than requiring every useful feature to be programmed manually. A network trained to recognise handwritten digits would learn, from examples alone, internal representations that captured relevant features of the input, without anyone specifying what those features should be.

The Neural Information Processing Systems conference, originally called the Neural Information Processing Systems conference and now known as NeurIPS, began in 1987 at the intersection of neural computation, neuroscience, psychology, statistics, and computer science. Its founding reflected the interdisciplinary character of the connectionist revival. It would go on to become one of the most important venues in all of machine learning.

The revival, however, did not last. By the mid-1990s, interest had declined again. Backpropagation networks were difficult to train reliably, prone to getting stuck in local minima, and there was no good theoretical account of why they should generalise well. Other approaches were delivering better results with cleaner theoretical foundations.

The Second AI Winter and the Statistical Turn

The commercial expert systems bubble burst in the late 1980s. Maintenance costs were high, the systems were difficult to update, and they did not deliver on the broader promises that had attracted investment. By the early 1990s, the AI industry had contracted substantially.

What replaced expert systems was not a new concept but a methodological shift. Machine learning, the study of algorithms that learn from data, had existed as a subfield of AI from the beginning but had always been somewhat marginal relative to the symbolic mainstream. In the 1990s it moved to the centre.

Data was becoming available at a scale that had not previously existed. The internet was generating text, images, and structured records in quantities that made statistical learning feasible. The theoretical foundations of machine learning were also being strengthened. Vladimir Vapnik’s statistical learning theory provided rigorous accounts of when and why learning algorithms should generalise from training data to new examples. Vapnik’s support vector machine, published in its modern form in 1995, became the dominant method in applied machine learning through the late 1990s and most of the 2000s. Support vector machines were theoretically principled, practically effective, and produced results competitive with or superior to neural networks on most benchmarks, with more reliable training.

The statistical turn in natural language processing during the 1990s was substantial. Hidden Markov models for speech recognition, probabilistic context-free grammars for parsing, and statistical approaches to machine translation all replaced rule-based systems. These methods were less linguistically interpretable but more accurate and far easier to build.

The defeat of world chess champion Garry Kasparov by IBM’s Deep Blue in 1997 was a cultural milestone, though its technical significance was more ambiguous. Deep Blue relied primarily on brute-force tree search, examining hundreds of millions of positions per second using alpha-beta pruning, with evaluation functions partly hand-tuned by grandmasters. It was impressive engineering, but it did not involve learning in any meaningful sense and told researchers little about general intelligence. Its contrast with later game-playing systems would become instructive.

By the late 1990s and early 2000s, the word “artificial intelligence” had almost vanished from research culture. Researchers worked on machine learning, computational linguistics, computer vision, speech recognition, and robotics, but framed these as separate engineering disciplines rather than contributions to a unified project. NeurIPS, the conference founded on neural network research, had by the late 1990s become a machine learning conference in which neural networks themselves were regarded with scepticism.

Neural Networks in the Wilderness

Neural network research continued through this period, though at reduced scale and with reduced prestige.

Kunihiko Fukushima’s Neocognitron, introduced in 1980, was a hierarchical architecture designed to recognise patterns regardless of their spatial position, inspired by the structure of the mammalian visual cortex. Yann LeCun combined related convolutional structures with backpropagation to build practical systems for handwritten-character recognition. LeCun’s convolutional neural networks were deployed commercially for reading digits on bank cheques, and his LeNet-5 architecture, published in 1998, demonstrated that deep convolutional networks could achieve excellent performance on real pattern recognition tasks.

Sepp Hochreiter and Jürgen Schmidhuber published the long short-term memory architecture in 1997. LSTM networks addressed a fundamental problem in training recurrent networks: gradients either vanish or explode as they are propagated backwards through many time steps, making it difficult to learn dependencies over long sequences. The LSTM introduced gating mechanisms that allowed the network to maintain information across long spans of input. It would become the standard approach to sequential data in speech and language throughout the 2000s.

Despite these advances, by the early 2000s, kernel methods were generally regarded as more reliable and theoretically satisfactory than neural networks. Support vector machines became the standard solution to a wide range of classification problems. Neural networks survived, but they were not the tool of choice for most practitioners. A small persistent community continued to believe that deeper networks with more data and more compute would eventually prove their value.

Deep Learning Re-emerges

The decisive break came in 2006. Geoffrey Hinton and Ruslan Salakhutdinov published a paper in Science showing that deep networks with many layers could be trained effectively using a technique called greedy layer-wise pretraining. The idea was to train each layer independently, using a type of model called a restricted Boltzmann machine, before fine-tuning the whole network with backpropagation. This bootstrapped the training process and provided a way around the problem of gradients vanishing as they were propagated back through many layers.

Yoshua Bengio at the University of Montreal and his collaborators developed related layer-wise methods. Bengio’s group had also produced important earlier work on word embeddings and distributed representations of language, including a 2003 neural language model that is a genuine precursor of later developments in natural language processing.

The specific pretraining methods that Hinton and Bengio developed were later found to be unnecessary for many tasks. Better initialisation methods, improved activation functions, and more data meant that deep networks could often be trained directly with backpropagation. But the 2006 work was historically important because it restored confidence in deep neural architectures and demonstrated that networks with many layers could learn useful representations.

The term “deep learning” was coined deliberately around this time, partly to distinguish the revived approach from the discredited “neural networks” label and partly to emphasise the key distinguishing feature: the use of many layers of representation, each transforming its input into a progressively more abstract form.

By the late 2000s, deep learning was no longer considered marginal in research circles, but it was not yet dominant. The mainstream of machine learning remained kernel methods and probabilistic graphical models. The attitude toward deep networks was more that they had been unfairly dismissed than that they were about to take over.

AlexNet and the Modern Era

The decisive moment came in 2012. The ImageNet Large Scale Visual Recognition Challenge was an annual competition in which systems competed to classify images from a database of over a million labelled photographs across a thousand categories. In 2012 a team consisting of Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton submitted a convolutional neural network trained on GPUs, subsequently known as AlexNet, which achieved an error rate roughly ten percentage points lower than the previous year’s winner. The gap was large enough that its significance was immediately apparent.

AlexNet combined several elements that together made the result possible. GPU-based parallel training made large networks feasible. The ReLU activation function trained faster and more reliably than the sigmoid functions previously standard. Dropout regularisation reduced overfitting. A deep convolutional architecture automatically learned hierarchical image features rather than relying on hand-engineered representations. The network learned, from data alone, feature detectors that corresponded recognisably to edges, textures, and object parts at successive layers.

The response from the technology industry was immediate. Google acquired the AlexNet team for a reported forty-four million dollars. DeepMind, a London-based AI research company founded by Demis Hassabis, Shane Legg, and Mustafa Suleyman in 2010, was acquired by Google in 2014 for a reported five hundred million dollars. Investment in deep learning research, both in academia and industry, scaled up rapidly.

The ImageNet result did not just demonstrate that deep learning worked. It demonstrated that the gap between deep learning and everything else was substantial. Within a few years, deep learning had replaced most of what had come before in computer vision.

Deep Learning Spreads

The years following AlexNet saw deep learning displace existing methods across machine learning’s application areas.

Speech recognition systems based on deep networks outperformed the previous generation of hidden Markov model systems substantially. The deep learning approach to speech had been developing since the early 2010s through a collaboration between Hinton’s Toronto group and researchers at Microsoft, IBM, and Google, and the results were consistent enough that by 2014 deep learning was the standard approach.

Recurrent neural networks, particularly the long short-term memory architecture, became the standard approach to sequential data including text and speech. Deep learning entered natural-language processing before the transformer arrived: neural word embeddings represented words as points in continuous spaces, capturing semantic relationships in a form that earlier bag-of-words models could not.

DeepMind’s Deep Q-Network, published in 2015, learned to play Atari games directly from the pixel output of the screen, using essentially the same general learning architecture across many different games. This was reinforcement learning combined with deep representation learning, and it demonstrated that the combination was powerful: a single system could learn to play dozens of games to superhuman performance without any game-specific programming.

AlphaGo, also from DeepMind, defeated the world Go champion Lee Sedol four games to one in March 2016. Go had been considered a much harder problem than chess for AI systems because the branching factor of the game tree was far too large for brute-force search. AlphaGo combined deep reinforcement learning with Monte Carlo tree search, learning through self-play. Its successor AlphaZero, trained entirely through self-play from scratch, achieved superhuman performance in Go, chess, and shogi simultaneously within hours of training. The contrast with Deep Blue’s approach could hardly have been sharper: where Deep Blue was specialised hardware executing handcrafted evaluation functions, AlphaZero was a general learning system that acquired its knowledge entirely from experience.

The Transformer

The most consequential architectural innovation of the current period was published by a group of Google Brain researchers in 2017 under the title “Attention Is All You Need.”

Attention mechanisms had antecedents. Bahdanau and colleagues had introduced an attention mechanism for recurrent networks in 2014, allowing a model to focus on different parts of the input while generating each element of the output. This reduced the need to compress an entire sequence into a single fixed representation, which had been the main bottleneck of earlier sequence-to-sequence models.

The 2017 paper went further. The transformer architecture dispensed with recurrence entirely, replacing it with self-attention: a mechanism that allows each element in a sequence to attend directly to every other element, computing the relevance of each to the others in parallel. This made training far more parallelisable than recurrent networks, which processed sequences step by step. The practical consequence was that transformers could be trained at a scale that recurrent networks could not.

Transformers were initially introduced for machine translation but rapidly became the standard architecture for language modelling. The attention mechanism proved to be general enough to work across many different types of data, and transformer-based architectures eventually spread into computer vision, audio processing, and scientific applications.

Foundation Models and Large Language Models

Two major transformer traditions developed from 2018 onwards.

BERT, from Google in 2018, used a bidirectional masking approach to pretraining: it learned to predict words that had been masked out of a sentence, seeing context from both directions. This produced representations that were powerful for tasks requiring understanding of a complete context. BERT and its successors became the dominant approach for natural-language understanding tasks through fine-tuning on specific downstream datasets.

GPT, from OpenAI, also in 2018, took an autoregressive approach: it learned to predict each word from the preceding context. GPT-2 in 2019 showed that a larger model trained on broader internet data could perform some tasks without task-specific training. GPT-3 in 2020, with 175 billion parameters, demonstrated that sufficiently large language models could perform many tasks from instructions and examples supplied directly in their context, without any additional training. This few-shot learning behaviour was unexpected and sparked substantial debate about what, if anything, these models understood.

An important empirical finding during this period was that performance often improved predictably as model size, training data, and computation increased according to consistent scaling laws. Scaling therefore became a systematic research programme rather than simply an engineering convenience. If you trained a bigger model on more data with more compute, you would get a better model, and the relationship was smooth and predictable enough to plan around.

These large pretrained systems became known as foundation models: a single model trained on a sufficiently broad prediction task acquired representations and capabilities that transferred across many domains. One model could provide the basis for many applications, changed through lightweight fine-tuning or instruction following rather than training from scratch.

Generative Media

Deep learning also transformed generative media, producing systems that could synthesise images, audio, and video at quality levels that had not previously been possible.

Generative adversarial networks, introduced by Ian Goodfellow and colleagues in 2014, trained a generator and a discriminator in competition. The generator attempted to produce synthetic examples indistinguishable from real data, while the discriminator attempted to tell them apart. This adversarial dynamic drove both networks toward increasingly realistic outputs. GANs produced remarkable results in image synthesis through the late 2010s but could be difficult to train and sometimes covered only part of the data distribution.

Diffusion models became practically important around 2020. They learned to reverse a gradual process of adding noise to data, allowing new samples to be generated by starting from noise and denoising step by step. By 2021 and 2022, diffusion models were producing image quality that generally surpassed GANs. Systems such as DALL-E, Stable Diffusion, and Midjourney brought high-quality image generation to general users.

Modern generative systems do not form separate language and visual technologies. Text encoders, transformers, vision encoders, autoregressive models, and diffusion decoders are frequently combined in single architectures. The relevant object is increasingly a multimodal model rather than a model devoted exclusively to language, images, or sound.

ChatGPT and the Present

Several developments were required to transform pretrained language models into practical assistants.

Instruction tuning trained models to respond to requests expressed in ordinary language rather than to complete text prompts in the style of their training data. Reinforcement learning from human feedback used demonstrations and preference judgements to make responses more useful and conversational. These techniques, combined with the raw capability of large pretrained models, produced systems that could be used directly for writing, explanation, programming, analysis, and planning.

OpenAI released ChatGPT on 30 November 2022. Its underlying language-modelling technology was evolutionary rather than wholly new, but the conversational interface and instruction-following behaviour made its capabilities accessible to a mass audience. ChatGPT reached one hundred million users in two months, the fastest adoption of a consumer technology in recorded history.

The subsequent releases of GPT-4, Claude, Gemini, and competing models from various organisations established the large language model as the dominant form of AI in both public understanding and commercial application. Earlier machine-learning systems had usually been invisible components of larger products. A conversational language model could instead be used directly and interactively for an enormous range of tasks. Its generality made the broader concept of artificial intelligence seem plausible again.

Current AI systems combine several previously distinct traditions. Their central representations are learned by deep neural networks, but they may also use reinforcement learning, explicit search, external memory, retrieval, code execution, symbolic tools, and conventional software. Recent development has concentrated on multimodal models that process combinations of text, images, audio, and video, and on reasoning models that use additional computation before answering. Agentic systems that repeatedly plan, call tools, inspect results, and revise their actions are an active area of development.

The Historical Pattern

The broad historical movement can be compressed into a single observation: machine learning absorbed much of artificial intelligence, and deep learning absorbed much of machine learning.

This does not mean that earlier approaches disappeared. Search, planning, probabilistic reasoning, reinforcement learning, and symbolic computation now frequently operate inside systems whose principal representation learner is a deep neural network. But the terms of the relationship have reversed. Where once neural networks were a marginal approach within a field dominated by symbolic reasoning, symbolic and statistical tools are now often auxiliary components within systems whose core is a neural network.

Deep learning’s success was not purely an intellectual victory. It resulted from the conjunction of several things that had to arrive together: better training algorithms, unprecedented quantities of digital data, GPU-based computing that made large networks feasible, benchmark competitions that concentrated effort and measured progress objectively, open-source software frameworks that lowered the barrier to entry, and eventually unprecedented industrial investment. The neural network tradition had been right about the broad approach for decades before it had the infrastructure to demonstrate it convincingly.

Modern language models have overcome problems that many researchers believed required explicit grammars, manually encoded concepts, and detailed symbolic models of the world. That achievement is historically extraordinary. Language and intelligence cannot reasonably be regarded as completely solved. Reliable factuality, grounding in the physical world, causal understanding, robust mathematical reasoning, and long-horizon autonomous action remain open problems.

What has been achieved is the construction of remarkably general statistical models that can mediate between human language, knowledge, perception, and action. Whether those systems constitute general intelligence, approximate it, or represent a fundamentally different kind of computational competence remains the central unresolved question in the history now unfolding.