A probability is conditional on a model
An alarm may be likely when a machine is faulty, yet a positive alarm need not make a fault likely if faults are rare. Inference combines the prior frequency of a hypothesis with the likelihood of what was observed. The result is a probability under a specified model, not a guarantee that the model describes every machine or environment.
Let be a hypothesis and be binary observations. Specify a prior with and conditional probabilities . These prior bounds ensure both conditioning events have positive probability.
Definition 9.1.1
(Prior, likelihood and posterior)
The prior is . For observed values the likelihood of hypothesis is , the probability of the evidence if were true. The posterior is , defined when the evidence has positive probability.
Definition 9.1.2
(Observations independent given the hypothesis)
The observations are mutually independent given if, for every value of and every joint observation , the likelihood is the product of the individual conditional probabilities:
This is a modeling assumption, not a consequence of having separate sensors or repeated tests.
From a joint model to a posterior
For observed values , define
Theorem 9.1.3
(Posterior under conditional independence)
Assume and that the observations are mutually independent given . Then for . If , the posterior is
If , the observed event has probability zero and the posterior is undefined.
Proof
(Multiply, sum, divide)
By the multiplication rule, the joint probability of and the evidence is times the likelihood, which by the stated independence equals . The events and are disjoint and exhaustive, so the law of total probability gives the evidence probability . When this is positive, the definition of conditional probability divides the joint probability by it. When it is zero, the denominator of the definition vanishes.
Proof X-ray
Where conditional independence is used
The assumption is used exactly once: to turn the likelihood of the whole evidence into a product, inside each hypothesis. Summation over the two hypotheses and the final division need no independence at all. Nor is independence of the observations after averaging over required; the counterexample below shows that it typically fails. What breaks without the assumption is the factorization, not the structure of the formula: if the second alarm merely repeats the first, the likelihood of two positive readings under each hypothesis is the likelihood of one, so squaring it is wrong.
For the empty product is one, so the posterior is the prior. If the evidence probability is zero, the displayed ratio is undefined; it is not legitimate to normalize a zero vector. For example, if both hypotheses give , the model assigns zero probability to observing and must be reconsidered or extended before this calculation applies.
One alarm and two alarms
Worked example 9.1.4
(A rare fault and a positive result)
Suppose , , and . Here one means “faulty” or “alarm”, respectively. A positive alarm has joint masses
Thus , approximately . The likelihood of an alarm under a fault is ; the posterior after an alarm is a different quantity.
Worked example 9.1.5
(A second conditionally independent alarm)
Suppose a second alarm has the same conditional rates and is independent of the first given . For two positive observations the masses are and . The posterior is , approximately . This calculation relies on the stated independence; common noise or a shared sensor failure can invalidate it.
Concept lensAlgebraic
Posterior odds as prior odds times likelihood ratios
Whenever , dividing by gives
and the posterior probability is recovered as . In the alarm example the prior odds are and each positive alarm contributes the likelihood ratio . After one alarm the odds are , giving ; after two they are , giving . Each new conditionally independent observation multiplies the odds by one factor, which makes the effect of every piece of evidence visible separately.
Worked example 9.1.6
(Conflicting readings and an impossible observation)
With the rates of the previous examples, let alarm 1 be positive and alarm 2 negative. The negative reading has likelihood under a fault and under no fault, so
The posterior is : the negative reading cancels most of the effect of the positive one, but not all of it, because the two readings have different likelihood ratios.
Edge case: change the model so that . One positive alarm gives and , so the posterior is . The formula remains valid because ; a zero likelihood eliminates a hypothesis rather than making the calculation undefined.
Counterexample mode
Conditional independence does not imply marginal independence
In the two-alarm example , but , which is not . The shared unknown can make observations dependent after averaging over its values. One must not multiply marginal probabilities as if the observations were independent without conditioning.
Common mistakes
Common mistake
(Reading the likelihood as the posterior, or ignoring the prior)
After one alarm, “the alarm is reliable” is a statement about , not about the probability of a fault. Comparing only the likelihoods and would suggest a fault probability of . The prior must be multiplied in, and the correct value is .
Common mistake
(Counting a repeated reading as independent evidence)
Suppose a second alarm is a copy of the first, so that with certainty. Then , so , and the posterior is still . Squaring the likelihoods, as the independent two-alarm calculation does, would wrongly report . Two readings count as two pieces of evidence only when conditional independence is justified.
What the calculation does and does not establish
For a binary hypothesis and observations, evaluating the two products and their sum takes arithmetic operations under constant-cost arithmetic and access to the specified probabilities. Products may underflow in finite-precision computation; numerical implementations need suitable scaling or log-domain methods. If or but their sum is positive, the formula remains valid. The result is a posterior under this model, not evidence of a causal mechanism or an automatic calibration guarantee.
Exercises
Quick check
(In the one-alarm example, compute the probability of a positive alarm and the posterior probability of no fault after it.)
The alarm probability is the evidence probability .
Solution · Answer
The alarm probability is ; the no-fault posterior is .
Quick check
(Under the two-alarm assumptions, compute .)
Sum the two joint masses.
Solution · Answer
It is .
Quick check
(Why is it invalid to divide by when both values are zero?)
Recall what represents.
Solution · Answer
The model gives the observed event probability zero, so the conditional-probability ratio has a zero denominator and is undefined.
Quick check
(If two alarms share a failure mechanism even after conditioning on , is squaring the one-alarm likelihood automatically justified?)
Consider the extreme case in which the second alarm copies the first.
Solution · Answer
No. The joint likelihood must be specified or justified; a shared mechanism may violate conditional independence. In the extreme case of an exact copy the joint likelihood of two positive readings equals the one-alarm likelihood, and the posterior stays at instead of rising to .