## Bayes' theorem

- Details
- Category: Basics
- Published on 02 January 2012
- Written by Richard D. Morey
- Hits: 12400

## Bayes' theorem

Trisomy 21, or Down syndrome, is caused by having an extra copy of chromosome 21. Children with the disease typically show impairments in cognitive abilities and certain stereotypical physical features. Approximately 1 in 800 children have the disease. Prenatal tests for trisomy 21 exist, including the multiple-marker screening test performed in the second trimester of pregnancy.

The outcome of this screening test is either positive, meaning that the test indicates that the fetus has trisomy 21, or negative, indicating that it doesn't. Screening tests, however, are not perfect; sometimes a test will indicate the presence of the disease when the fetus does not have the disease (a "false positive"); likewise, the test can fail to detect the disease when the fetus actually does have the disease (a "false negative"). The false positive and false negative rates for the trisomy 21 screening are known, and are about 0.05 and 0.19, or 5% and 19%, respectively. A pregnant woman who obtains a positive screening result on the screening test must wonder: what is the probability that the baby will have trisomy 21?

It should be obvious that this question can be written as a conditional probability: that is, the mother wants to know the probability that the fetus has trisomy 21, given the positive test result, or, in notation, \[ Pr(D\mid+) \] where \(D\) represents the event that the fetus has the disease and \(+\) represents the event that the test was positive. But although we can write our question using conditional probability, it is not obvious from the preceding sections how to answer it using joint probability. In the previous sections, we had access to a table that contained all the joint probabilities, from which we could compute marginal and conditional probabilities. In this problem, we aren't given any of the information about the joint probabilities; we only have the following probabilities:

- \(Pr(D)\): the marginal probability of the disease, which is 1/800=0.00125.
- \(Pr(+\mid H)\): the false positive rate, which is 0.05.
- \(Pr(-\mid D)\): the false negative rate, which is 0.19.

where \(H\) is the event that the fetus is "healthy" (i.e. does not have the disease) and \(-\) is the event that the test is negative. Because probabilities must always sum to 1, we also know the following probabilities:

- \(Pr(H)\): the marginal probability that a fetus does not have the disease, which is 1-0.00125 = 0.99875
- \(Pr(+\mid D)\): the correct positive, or detection, rate, which is 1-0.19=0.81.
- \(Pr(-\mid H)\): the correct negative rate, which is 1-0.05=0.95.

We know \(Pr(+\mid D)\) (among other quantities); we *want to know* \(Pr(D\mid +)\). How can be get from what we know to what we want to know? We can use the definition of conditional probability, which says that \[ Pr(A\mid B) = \frac{Pr(A,B)}{Pr(B)} \] which, by multiplying both sides by \(Pr(B)\), implies that \[ P(A,B) = Pr(A\mid B)Pr(B). \] But, by the same logic, \[ Pr(A,B) = Pr(B\mid A)Pr(A). \] Since we have two quantities that are equal to one another, we can write \[ Pr(A\mid B)Pr(B) = Pr(B\mid A)Pr(A). \] Dividing both sides by \(Pr(B)\) yields the equation \[ Pr(A\mid B) = \frac{Pr(B\mid A)Pr(A)}{Pr(B)} \] This equation is called Bayes' theorem. We can extend the equation a bit; because of the way we rewrote the marginal probability in the article Conditional probability, we can rewrite Bayes' theorem as: \[ Pr(B\mid A) = \frac{Pr(A\mid B)Pr(B)}{\sum_{k=1}^K Pr(A\mid B_k)Pr(B_k)} \] By writing Bayes' theorem this way, we can see that Bayes' theorem is a proportion; the numerator is the probability of the combination of interest (e.g., a positive test and the disease), and the denominator is the sum of all possible ways to achieve the result conditioned on (e.g., a positive test). If we know we have a positive test, we must account for all possible ways of obtaining a positive test result; in this case, by both having the disease and correctly detecting it, and not having the disease and getting a false positive.

Returning to our example and applying Bayes' theorem, we can see that \[ Pr(D\mid +) = \frac{Pr(+\mid D)Pr(D)}{Pr(+\mid D)Pr(D) +Pr(+\mid H)Pr(H)}. \] If you scan the list above of probabilities that we know, you'll see that we know every probability needed to apply Bayes' theorem. We need the base-rate of trisomy 21 (\(Pr(D)\), from which \(Pr(H)\) can easily be calculated), the detection rate (\(Pr(+\mid D)\)), and the false positive rate (\(Pr(+\mid H)\)). All that remains is to use the numbers provided:

That is, the probability of a fetus having trisomy 21 given that the screening test was positive is only 2%. Although the parent has some reason for concern, a positive test result is by no means a definitive indication of the disease. In fact, a 2% probability corresponds to 50 to 1 odds that the child does *not* have the trisomy 21. The low base-rate of the disease (1 in 800) means that much more evidence than one positive test is needed to conclude that the fetus has trisomy 21.

Given that a positive screening test gives only a 2% chance of having the disease, it may be temping to say that the screening test failed or is not valuable. However, this would be a mistake. Recall that before the test was performed - that is, before conditioning on the test result - the probability that the fetus had the disease was \(Pr(D)\)=0.00125, or 0.125%. The positive test result increased the probability of the disease by a factor of 16, to 2%. By this metric, the positive test was quite informative. However, the positive test was not informative enough to push the probability of the disease over 50%, which is the point where we should begin to believe that the fetus has the disease. In order to increase the informativeness of the test, we would need to increase the test's detection rate or decrease the false positive rate.

At this point, it is important to note that the use of Bayes' theorem is not what makes one a Bayesian analyst. Bayes' theorem is a simple consequence of the definition of conditional probability; all kinds of analysts make use of Bayes' theorem. The dividing line between Bayesian analysts and frequentist analysts is what kinds of events one discusses using the rules of probability. The example above is uncontroversial; however, for questions of real interest to analysts - such as inference about the mean of a population, numbers of whales in the ocean, or single event probabilities - Bayesian statisticians interpret the terms in Bayes theorem in a special way.