A count of successes is not always enough. An experiment may produce one of several mutually exclusive outcomes, and we may want the counts in all categories. The counts share a fixed total, so knowing one count affects what is possible for the others. By contrast, if the total itself is Poisson, independently assigning its events to categories produces independent counts. Understanding the difference requires keeping the sampling mechanism in view.
A fixed number of trials
Definition 27.1.1
(Multinomial distribution)
Suppose independent trials each fall into exactly one of categories with probabilities , where and . Let count outcomes in category . The law of the vector is the multinomial distribution, written . Its support consists of nonnegative integer vectors with .
Theorem 27.1.2
(The multinomial probability)
For a vector on this support,
Each is binomial with parameters , while for ,
Proof
(Count arrangements, then use indicators)
A specified sequence with appearances of category has probability , by independence of the trials. There are such sequences, obtained by choosing which trial positions belong to each category. Summing their equal probabilities gives the formula; the usual convention handles absent categories of zero probability.
Let indicate category on trial . Then , and are independent Bernoulli variables with mean , so is binomial with the stated mean and variance. On the same trial for , hence
while cross-trial covariances vanish by independence. Summing the same-trial terms proves the covariance formula.
Proof X-ray
Where the negative covariance comes from
The proof uses two different features of the model. Independence across trials is what makes every sequence with the same category counts equally likely, and what kills all cross-trial covariances. Exclusivity within a trial, the fact that a trial falls into exactly one category, is the sole source of the covariance: it forces , which turns into .
Thus a negative covariance is a consequence of this fixed-total model, not a universal feature of category counts. If the trials themselves were dependent, cross-trial terms would reappear and the sign could change; if the total were random, the constraint that the exclusivity encodes would disappear, which is exactly what happens in Poisson splitting below.
Worked example 27.1.3
(Three possible outcomes)
In four independent trials let the probabilities of red, blue and green be . Exactly two red, one blue and one green has probability
The expected numbers of red and blue are and , and their covariance is . They cannot be independent: the event of four red trials rules out even one blue trial, though each event has positive probability.
A random total changes the answer
Theorem 27.1.4
(Poisson splitting)
Let , with . Conditional on , independently assign each of the events to category with fixed probability , where and the probabilities sum to one. Then the category counts are independent, and .
Proof
(Multiply the Poisson mass by the multinomial mass)
If , then almost surely and every count is zero almost surely, proving the claim in that case. Now suppose . For nonnegative integers , put . The counts determine , and , so the multiplication rule and the multinomial formula give
The second equality uses to write , and . Each factor is a Poisson probability mass, including the degenerate zero-rate case. Factorization of the full joint mass proves independence.
Proof X-ray
Why the factorials cancel exactly
The key move is the cancellation of : the Poisson mass of the total supplies , and the multinomial coefficient supplies . The Poisson mass of the total carries exactly the factorial that the coefficient needs, so what remains is a product of separate functions of ; for a general law of , the factor need not split in this way. Two further ingredients are used: splits as because , and splits because the sum to one. Independent assignment is used in invoking the multinomial formula given . Conditioning again on when restores the fixed-total multinomial dependence, so the independence lives in the randomness of the total.
Worked example 27.1.5
(Thinning and conditional reconstruction)
Suppose a Poisson number of arrivals has mean , and each arrival is independently labeled A with probability or B otherwise. Then and independently. Thus . If we are told that the total was exactly three, however, , and .
Counterexample mode
A random total alone is not enough
One might expect any random total to produce independent counts. If is always one and the two category probabilities are both , the counts are or , each with probability . Then , while , so the counts are dependent. The Poisson law and independent assignment are both essential to the stated splitting result. Likewise, assigning every event the same randomly chosen label generally produces dependence even with a Poisson total.
Rare categories and the Poisson limit
The split Poisson model also appears as a limit of fixed-size experiments. In row , let each of independent trials have category probabilities for , with one further category of probability . Here are fixed, and is large enough that this last probability is nonnegative. Let count category .
Theorem 27.1.6
(Poisson limit of rare categories)
For fixed nonnegative integers ,
Thus converges in distribution to a vector of independent Poisson variables with means .
Proof
(Take the limit factor by factor)
Put and take . The multinomial formula, with trials in the extra category, gives
The factorial ratio is , a product of factors tending to one, and the last factor tends to . Thus each fixed joint mass approaches the mass of independent Poisson variables. These limiting masses sum to one, so the joint distributions converge to that law.
The limiting independence does not assert exact independence at finite : for , the covariance is , which tends to zero but is nonzero whenever .
Worked example 27.1.7
(How close the limit is for a thousand trials)
Take trials with two rare categories of probabilities and , so and , and a third category of probability . The exact probability of no category-1 outcome and exactly one category-2 outcome is
The limit theorem gives , a relative error of about . The exact covariance of the two counts is : small, but not zero.
Common mistakes
Common mistake
(Multiplying binomial marginals to get a joint probability)
Each multinomial count is binomial, but the joint mass is not the product of the marginals. In the three-color example, , because there are only four trials. The product of marginals is .
Common mistake
(Dropping the covariance when adding counts)
In the three-color example, . But counts the merged category of probability , so it is with variance . The difference is , and .
Practice with solutions
Quick check
(In ten independent trials, three categories have probabilities . Find the probability of counts , and the covariance of the first two counts.)
Use the multinomial coefficient .
Solution · Answer
The coefficient is , so the probability is . The covariance is .
Quick check
(A Poisson number of defects has mean . Independently classify each as minor with probability or major otherwise. Find the probability of exactly two major defects and no minor defects.)
Split first, then use independence.
Solution · Answer
Minor and major counts are independent Poisson variables with means and . The answer is .
Quick check
(With and the labels as in the splitting theorem and , derive the conditional distribution of given , and state whether remain independent under this conditioning when and .)
Given the total, the construction is a fixed number of trials.
Solution · Answer
Given , the construction is a multinomial experiment with trials, so . The counts are not conditionally independent: their conditional covariance is .