The Testing Fallacy
Say we have a drug test that is 99% accurate (it answers correctly 99% of the time.) What is the probability that a random person that tests positive actually took the drug? The answer: far from 99%.
The issue occurs when you ignore the fact that you don’t have exactly half your population taking the drug. So let’s take 1,000,000 people, and say 1% of them (10,000 people) are taking the drug (ignore reality for a moment here.) 99% of the ones who are taking the drug (9,900 people) will be caught. Here’s the problem: 1% of the remaining 990,000 people that haven’t taken the drug have their test results come back positive. That’s 9,900 people out of the 19,800 people testing positive—only half of them actually took the drug. These are not good odds to base any sort of punishment on.
I’ve heard this described multiple times in lectures, but I haven’t seen it written down or diagrammed. I found it difficult to visualize—hopefully this will help others.
