Confirmation bias: checking only the answers you hope are right
Nobody thinks they cherry-pick. But a classic number puzzle shows how easily people test only the cases that can say yes, and come away feeling confirmed when they shouldn't.
WrongEvery example I checked fits my theory, so it must be right.
RightEvery example I checked fits, but I only checked the cases my theory predicts. What result would prove it wrong?
What people get wrong#
Psychologist Raymond Nickerson's widely cited review defines confirmation bias as "the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand." If you had to pick the one flaw in human reasoning most worth attention, he argues, this would have to be among the candidates.
The common misunderstanding is that confirmation bias is something stubborn other people do: ignoring facts, believing what they want. The more useful version is less dramatic. Honest, curious people show it all the time, simply through which questions they think to ask.
The 2-4-6 puzzle#
In a 1960 experiment by the psychologist P. C. Wason, people were shown the numbers 2, 4, 6 and told they fit a hidden rule. They could propose other sets of three numbers and would be told whether each one fit, then announce the rule when they were sure.
A typical player guesses "even numbers going up by two," then tests 8-10-12, 20-22-24, 100-102-104. Every one comes back "yes." The guess feels confirmed, and it's wrong. The real rule was simply any three numbers in ascending order.
Nickerson explains the trap: people "typically tested hypothesized rules by producing only triplets that were consistent with them." Since every set that fits "up by two" also fits "increasing," no test could ever come back "no." The run of yeses felt like proof but proved almost nothing. One try of 1-2-3 or 5-10-40, a set your theory says should fail, would have shown the mistake at once.
It's not just testing#
The bias also changes how we weigh evidence we already have. Nickerson cites the finding that two people with opposing views can examine the same evidence and both come away more confident, each picking out the parts that suit them and finding fault with the rest. You see the same thing in performance reviews ("I knew she was a star"), investing ("see, the stock went up") and debugging ("the logs confirm my theory," when nobody checked the logs that would contradict it).
A fair caveat#
Looking for cases where your idea should hold isn't always irrational. Klayman and Ha argued that much of what gets called confirmation bias is better described as a "positive test strategy," which is often a very good heuristic under realistic conditions. It fails in particular situations, such as when your hypothesis is narrower than the truth, as in 2-4-6. So the fix isn't "never look for confirmation." It's knowing when confirming cases can't tell you anything.
Why it happens#
Checking for "yes" takes less effort than imagining how you could be wrong. Agreeing evidence feels good; contradicting evidence feels like criticism. And once we've said a belief out loud, keeping it becomes partly about our reputation.
The fix#
The new checkout page is better: conversions are up since launch.
The new checkout page is better: conversions beat the old page in a randomized test, and we checked the segments where we expected it to lose.
A test that couldn't have come out against you isn't evidence for you.
- Write down what would change your mind before you look. The philosopher Karl Popper made falsifiability his test for whether a theory is scientific at all. A claim that fits every possible result tells you nothing.
- Run the test that should fail. In code, write the unit test for the input your fix shouldn't handle. In planning, ask what would be true if you were wrong, and check whether it is.
- Steelman the other side, or ask someone who disagrees to pick the evidence.
- Blind yourself where you can. Grade work, review résumés or compare options without seeing which one you favored.
More on related traps in our guide to cognitive biases.
Sources
Every factual claim on this page links to one of these. Checked November 4, 2026. See how we source.
- Review of General Psychology (UCSD copy): Confirmation Bias: A Ubiquitous Phenomenon in Many Guises (Nickerson, 1998) pages.ucsd.edu
- Psychological Review (UCSD copy): Confirmation, Disconfirmation, and Information in Hypothesis Testing (Klayman & Ha, 1987) pages.ucsd.edu
- Stanford Encyclopedia of Philosophy: Karl Popper plato.stanford.edu