What Is a Neural Network?

If you have ever used Claude, ChatGPT, Gemini, Midjourney, or any modern AI tool, you have used a neural network. They are the engine underneath every AI system that appeared after 2017. Understanding what a neural network actually is, in plain English, is the difference between using AI as a magic box and using AI like someone who knows what is going on inside.

What is a neural network, in one sentence?

A neural network is a piece of software made of many small calculators (called “neurons”) wired together in layers, where each connection has an adjustable strength, and the network learns by repeatedly tuning those strengths until it produces good outputs.

That is the whole thing. Not magic. Not consciousness. Just a very large set of adjustable connections that get tuned by feedback.

Why is it called a “neural” network?

The name comes from a loose analogy to brain cells (neurons). In your brain, neurons connect to other neurons via synapses, and learning happens when those synaptic connections get stronger or weaker. An artificial neural network borrows that idea: each artificial “neuron” receives signals from other neurons, multiplies them by adjustable weights, adds them up, applies a simple math function, and passes the result on.

The analogy is loose. A real neuron is vastly more complex than an artificial one. But the core principle, learning by adjusting connection strengths, transferred over and turned out to work astonishingly well.

How does a neural network actually learn?

Imagine you are teaching a child to recognize dogs. You show them a picture and ask “is this a dog?” If they say yes and it is a dog, you say “right.” If they say yes and it is a cat, you say “no, that is a cat.” Over thousands of examples, they get better.

A neural network learns the same way, just with math instead of words.

1. Show the network a labeled example. A picture of a cat, labeled “cat.”
2. Let the network guess. Its current set of weights produces some output, say “70% dog, 30% cat.”
3. Measure the error. The correct answer was “100% cat.” The network was off by 70 percentage points in the wrong direction.
4. Adjust every weight a tiny bit to make the error smaller. This is the magic step, called backpropagation.
5. Repeat millions of times with millions of different examples.

After enough rounds, the weights settle into a pattern that does a good job on the kind of input you trained it on. The network has “learned” without anyone telling it the rules of cat-vs-dog. It found the rules by itself, encoded in the millions of weights.

What is a “deep” neural network?

“Deep” just means “many layers.” A neural network with two or three layers is called shallow. A network with dozens or hundreds of layers is called deep, and the field of working with them is called deep learning.

Why does depth matter? Because each layer learns a slightly more abstract version of the input. In an image-recognition network, the first layer might detect edges. The next layer combines edges into shapes. The next builds shapes into parts (an eye, a wheel, a leaf). The next combines parts into whole objects (a cat, a car, a tree). Depth gives the network room to build up complex understanding from simple raw pixels.

What is a “weight” in a neural network?

A weight is a single adjustable number on one connection between two neurons. Modern neural networks have billions of weights. GPT-4, Claude 3.5, and Gemini Ultra each have hundreds of billions of weights inside them.

Every weight starts as a random number. Training is the process of nudging each weight slightly, billions of times, until the whole set produces good outputs on the training data. The “model” you download or interact with is, at the end of the day, just an extremely long list of weights.

What are the different kinds of neural networks?

• Convolutional Neural Network (CNN). The workhorse for image recognition. Used by every photo app that detects faces and every self-driving car that reads road signs.
• Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM). Older designs for processing sequences of data like text and audio. Mostly replaced by transformers since 2017.
• Transformer. The architecture behind GPT, Claude, Gemini, and almost every modern language model. Reads all parts of an input in parallel using a mechanism called attention. See our transformer explainer.
• Diffusion model. The kind behind Midjourney, DALL-E, Stable Diffusion, Veo, Runway. Learns to reverse a process of adding noise to images, which lets it generate new images from text descriptions.
• Graph Neural Network (GNN). Designed for data shaped like networks of connections, like molecules in drug discovery or social networks.

Why did neural networks suddenly get so good after 2012?

The math of neural networks has been around since the 1940s. So what changed?

• 2012, AlexNet. Fei-Fei Li released the ImageNet dataset (millions of labeled images). A team at the University of Toronto trained a deep neural network called AlexNet on GPUs and crushed every competing approach. This was the public proof that scale + data + compute = breakthrough performance.
• GPUs and data. The same chips that render video game graphics turned out to be perfect for neural-network math. As GPUs got faster and the internet produced more labeled data, networks got more accurate.
• 2017, the transformer. A small team at Google published “Attention Is All You Need”, which introduced the architecture that powers every modern language model. Before this, AI mostly read text one word at a time. Transformers read everything at once.
• 2020 onward, the scaling laws. Researchers found that bigger networks, with more data and more compute, kept getting better in predictable ways. This is why every modern AI lab is racing to train bigger models.

Where can a non-developer see neural networks in everyday life?

• Every time you talk to Claude, ChatGPT, or Gemini, you are sending input through a transformer neural network with hundreds of billions of weights.
• Every photo your phone auto-tags, every face it recognizes, every “best shot” it picks, used a convolutional neural network.
• Every Spotify or Netflix recommendation, a neural network somewhere in the recommendation engine.
• Every voice assistant transcription, a neural network converts audio to text.
• Every AI image you have ever generated with Midjourney, DALL-E, Stable Diffusion, or any video model, a diffusion neural network.

Related glossary entries and deeper guides

• What is Deep Learning?
• What is Machine Learning?
• What is a Transformer?
• What is Attention (in AI)?
• What are Weights (in AI)?
• What is Backpropagation?
• Full AI Glossary, 130 plain-English entries