Bayes’ Theorem with Example for Data Science Professionals

Bayes Theorem is the extension of Conditional probability. Conditional probability helps us to determine the probability of A given B, denoted by P(A|B). So Bayes’ theorem says if we know P(A|B) then we can determine P(B|A), given that P(A) and P(B) are known to us.

In this post I am concentrating on Bayes’ theorem assuming you have good understanding of Conditional probability. In case you want to revise your concepts, you may refer my previous post on Conditional probability with examples.

Formula derivation:

From conditional probability we know that

P(A|B) = P(A and B)/P(B)
P(A and B) = P(B) * P(A|B) ————–[1]

Similarly

P(B|A) = P(B and A)/P(A) = P(A and B)/P(A) [In Joint Probability order does not matter]
P(A and B) = P(A) * P(B|A) ————–[2]

From equation [1] and [2],

P(B) * P(A|B) = P(A) * P(B|A)
P(A|B) = P(A) * P(B|A) / P(B)

Which mean if we know P(A|B) then we can easily determine P(B|A) and vice versa. Assuming we know the total probabilities P(A) and P(B).

More Generalized Definition:

Let A1, A2, A3… Ak be a collection of mutually exclusive and exhaustive events with probability P(Ai), I = 1, 2, 3…k. Then for any event B for which P(B) > 0

P(Ai|B) = P(Ai and B)/P(B)

= P(B|Ai)*P(Ai)/∑[P(B|Ai)*P(Ai)

Below is the tree representation of the Bayes’ Theorem.

Bayes' Theorem using Probability Tree — Bayes’ Theorem using Probability Tree

Now let’s solve some example to get a feeling of Bayes’ theorem.

Example 1

Technicians regularly make repairs when breakdowns occur on an automated production line. Janak, who services 20% of the breakdowns, makes an incomplete repair 1 time in 20.Tarun, who services 60% of the breakdowns, makes an incomplete repair 1 time in 10. Gautham, who services 15% of the breakdowns, makes an incomplete repair 1 time in 10 and Prasad, who services 5% of the breakdowns, makes an incomplete repair 1 time in 20. For the next problem with the production line diagnosed as being due to an initial repair that was incomplete, what is the probability that this initial repair was made by Janak?

Solution:

First let’s try to visualize using probability tree.

Example 2 Clinical trials

Epidemiologists claim that probability of breast cancer among Caucasian women in their mid-50s is 0.005. An established test identified people who had breast cancer and those that were healthy. A new mammography test in clinical trials has a probability of 0.85 for detecting cancer correctly. In women without breast cancer, it has a chance of 0.925 for a negative result. If a 55-year-old Caucasian woman tests positive for breast cancer, what is the probability that she in fact has breast cancer?

Solution:

P(Cancer) = 0.005
P(Test Positive | Cancer) = 0.85
P(Test Neg|No cancer) = 0.925
P(Cancer|Test Positive)= P(Cancer) * P(Test Positive | Cancer) / P(Test Positive)

Probability Table and Tree Visualization for the given question

Example 3

SpamAssassin works by having users train the system. It looks for patterns in the words in emails marked as spam by the user. For example, it may have learned that the word “free” appears in 20% of the mails marked as spam. Assuming 0.1% of non-spam mail includes the word “free” and 50% of all mails received by the user are spam, find the probability that a mail is spam if the word “free” appears in it.

Solution

Data Given:

P(Free | Spam) = 0.20
P(Free | Non Spam) = 0.001
P(Spam) = 0.50 => P(Non Spam) = 0.50
P(Spam | Free) = ?

Using Bayes’ Theorem:

P(Spam | Free) = P(Spam) * P(Free | Spam) / P(Free)
P(Spam | Free) = 0.50 * 0.20 / (0.50 * 0.20 + 0.50 * 0.001)
P(Spam | Free) = 0.995

Now visualize using probability tree:

Bayes' Theorem Example 3 — Bayes’ Theorem Example 3

So that is all about Bayes’ Theorem for data Science. In next posts I will be writing about Probability Distributions which will complete the Probability for Data Science Series.

Previous articles from Probability for Data Science Series are:

I hope you enjoyed the post. For any comments, use the below comment section to write back to me or even you can directly write to me using https://ashutoshtripathi.com/contact/. I will try to answer all your queries related to DS, ML and AI.

Thank You!