What Is Reward Prediction Error and How Does the Brain Learn From Rewards?

Last month my local café gave me a free coffee on my seventh stamp. The card says ten. I stood there grinning like I’d won the lottery over a flat white.

Then last week I hit stamp ten, got the free coffee I was owed, and felt nothing much. A free coffee is a free coffee, so that’s odd.

There’s a fairly neat explanation, and I want to get my head around it properly.

So what is it, in plain words?

It’s called reward prediction error. Sounds like a software bug, I know. It’s just the gap between the reward you expected and the one you got.

Better than expected, and the gap is positive. Exactly as expected, and it’s zero. Worse than expected, or nothing turns up at all, and it goes negative.

Stamp seven was a big positive. Stamp ten was zero, because I saw it coming for weeks.

That’s more or less the whole idea. Slightly deflating for something with such a fancy name.

The monkey experiment everyone brings up

In the 1990s, Wolfram Schultz and his colleagues recorded from single dopamine neurons in the midbrain of monkeys. The monkeys got little squirts of juice, and the researchers watched what the neurons did.

At first, when juice arrived out of nowhere, the neurons fired in a burst. Fine, nice surprise.

Then they added a cue, a light or a tone, just before the juice. After a while, the neurons stopped firing at the juice itself. They fired at the cue instead. The juice wasn’t news anymore. The cue was.

Here’s the part that got people excited. When the cue came, but the juice didn’t, the neurons went quiet right at the moment the juice should have shown up. Their activity dipped below normal, like a tiny “huh?”

That dip is what convinced people the signal wasn’t just “reward detected.” It was tracking the reward prediction error, the difference between what was predicted and what happened.

I think that’s lovely, honestly. A single cell doing arithmetic on disappointment.

It wasn’t a brand-new idea

The math was already around. In 1972, Robert Rescorla and Allan Wagner built a model out of behavioral psychology, saying that animals learn when something is surprising, and barely at all when it’s predictable. Before that, Pavlov’s dogs were basically the same story.

Later, computer scientists Richard Sutton and Andrew Barto came up with temporal difference learning, a way for machines to learn from rewards by tracking prediction errors. Then in 1997, Schultz, Peter Dayan, and Read Montague noticed that the monkey neurons behaved a lot like the signal in those models.

Biology and computer science are landing on the same trick. I find that a bit spooky. It’s also why the AI systems that learn by trial and error use a very similar idea.

Dopamine isn’t the “pleasure chemical.”

For years, the popular story was that dopamine equals pleasure. Eat chocolate, get dopamine, and feel good. It’s a nice, tidy picture, and it’s mostly wrong, or at least not the whole thing.

What the monkey work suggested is that dopamine release works more like a teaching signal. Not “this feels good,” but “this was better than I thought, remember what you just did.”

That’s why you hear people talk about boosting dopamine levels with cold showers and phone detoxes, and I’d take most of it with a big pinch of salt. Brain chemistry is a lot messier than a single number going up or down.

And even the prediction error story doesn’t cover everything. Dopamine is involved in movement, motivation, attention, and probably more than I know. Researchers still argue about how much it does, and some recent work suggests different groups of dopamine neurons do different jobs. So treat this as the best-known piece of a bigger puzzle, not the full picture.

It’s a good example of how much brain function gets squeezed into one headline.

How the learning actually happens

So, a rough version. Something happens, you do something, and a result arrives. If the result beats your guess, the connections behind that choice get strengthened. Do it again, probably.

If the result is worse than your guess, those connections weaken a bit. Maybe don’t do it again.

If it matches your guess, not much changes. You already knew.

That last bit explains why getting good at something feels flatter than learning it. The early days are full of surprises, so the learning process moves fast. Later there’s less to be surprised by, so progress slows down. It’s not that you’ve stopped improving, it’s that the teacher has less to say.

I think this is also why novelty feels so good. New things have big gaps between expectation and reality, so the signal is louder.

Calling this brain learning in its simplest form isn’t quite fair, since people also learn from words, from watching others, and from just thinking. But the trial-and-error version is probably the oldest.

Where you can see it in daily life

The loyalty card, obviously. Also the compliment from a boss who rarely gives them, which lands harder than constant praise from someone who says “great job” about everything.

Positive reinforcement works partly for this reason. A dog gets a treat when it sits, and the surprise of the treat helps tie the sit to the reward. Once the dog expects a treat every time, the effect on learning gets smaller. Good trainers mix it up, I think.

Mixing it up is also what slot machines and social media feeds do. You never know when the next good thing is coming. Sometimes it’s great, usually it isn’t, so every pull of the lever or flick of the thumb carries a little bit of hope. The unpredictability keeps the signal alive. That’s partly why a reward prediction error account is so popular in explanations of gambling, though I’d say it’s one piece of the story and not all of it.

Some research suggests near-misses on a slot machine light up reward areas too, even though you lost. I’ve read that more than once, but the details are still being worked out.

When it goes a bit wrong

A few ideas, and these are theories rather than settled facts.

One theory about addiction is that some drugs force a dopamine surge directly, so the brain keeps getting a “better than expected” signal that never fades the way a normal reward would. The learner never gets to say, “Okay, I’ve got it.” That’s a simplification, and addiction has plenty of other layers, including stress, environment, and habit.

In Parkinson’s disease, dopamine-producing cells are lost, and that affects movement first but also how people learn from rewards and mistakes. Some Parkinson’s medicines have been linked to impulse control problems like compulsive gambling in a minority of people. If that rings any bells for you or someone close to you, speak to a doctor rather than guessing.

There are also theories connecting weaker or blunted signals to low mood and loss of pleasure in depression. Interesting, but still being tested. Please don’t diagnose yourself off a blog post. I’d hate that.

Does knowing this help?

A bit, I think. It explains why you stop being thrilled by things you used to love. They became predictable, and the prediction error shrank to nothing.

It also hints that expectations matter more than you’d think. If you expect something to be amazing, it has to clear a high bar to feel good at all. Lowering the bar isn’t exactly a life plan, but it’s worth noticing.

And in reward system psychology textbooks, this is one of the cleaner, more testable ideas, which is why it keeps coming up.

So next time something minor makes you unreasonably happy, or a great thing feels like nothing, that’s probably your brain doing its sum. Seventh stamp, tenth stamp. Same coffee.

I did get another free one today, by the way. It tasted fine. Zero error, zero drama.

FAQs

What is reward prediction error?

It’s the difference between the reward you expected and the one you actually got.

Is dopamine the pleasure chemical?

Not really. It’s better described as a learning and motivation signal, though its full role is still debated.

Why do surprises feel better than planned treats?

Unexpected rewards also create a larger positive difference between expectation and outcome, and hence a stronger brain signal.

What happens when an expected reward doesn't show up?

Dopamine neuron activity dips below its normal level, which acts as a “that was worse than I thought” signal.

Does the brain keep learning when rewards become predictable?

Much less so. When nothing surprising happens, there’s little for the brain to update.