Introduction
A great deal of applied data analysis begins with two categorical variables recorded on the same units: treatment arm and outcome, region and product choice, device type and conversion. The natural summary is a two-way contingency table, and the first question is usually whether the two variables are associated at all. Pearson's chi-square test answers a narrow version of that question. It compares the observed table with the table we would expect if the joint distribution factored into its margins, and it calibrates the discrepancy against an asymptotic reference law.
This note sets out exactly what is assumed, what is computed, why the reference law has degrees of freedom, and where the method breaks. The calculation is easy. The hard part is knowing what the test claims and what it does not.
The sampling model
Let be independent and identically distributed pairs, where takes values in and takes values in , with and fixed. Write
The observed count is the number of pairs falling in cell . The row totals, column totals and grand total are
Because the pairs are iid and each lands in exactly one cell, the vector is multinomial with trials and probabilities . Everything below depends on this. Each unit contributes one count, units do not influence one another, and the category set was fixed before the data were seen.
Independence as a hypothesis about the joint law
The marginal probabilities are and . The null hypothesis of independence is
with every and every .
is a statement about the joint distribution: it equals the product of its margins. It says nothing about mechanism. Rejecting means the data are hard to reconcile with that factorization under the sampling model. It does not mean that causes , that causes , or that intervening on one would change the other. Confounding, selection into the sample and reverse dependence all produce a joint law that does not factor.
Concept lensAlgebraic
Independence as a rank-one probability matrix
Arrange the cell probabilities as an matrix . Then says exactly that : the matrix has rank one, and every row is a multiple of the column margin. Conversely, a nonnegative rank-one matrix with entries summing to factors as the product of its own margins. For a table the condition is the single equation , an odds ratio of . The fitted table is the rank-one table with the observed margins.
Estimated expected counts
Under the cell probabilities are unknown because and are unknown, so they must be estimated.
Proposition 7.5.1
(Maximum-likelihood fit under independence)
Under , if every observed row total and column total is positive, the maximum-likelihood estimates of the margins are and . The fitted, or estimated expected, counts are therefore
Proof
(Separating the log-likelihood)
Under the multinomial log-likelihood is, up to a constant,
The problem separates. With every , the function is strictly concave on the simplex , so its stationary point is the maximum; a Lagrange multiplier gives , hence . Similarly . A zero observed margin instead places the maximum on the boundary of the closed parameter space; under strictly positive parameters it is only a supremum.
The fitted table reproduces the observed margins exactly: and .
Positive margins are required. If some , then for every in that row. The fitted law then contradicts the assumption , and the Pearson terms are undefined. The computed statistic is therefore defined only when every observed row total and column total is positive. An individual cell may have . That is harmless, because its is still positive whenever its margins are.
Under the model, zero margins are a finite-sample phenomenon. Since , and the same holds for columns, the probability that the statistic is undefined vanishes as . At finite it can still happen. Adding an arbitrary pseudocount to rescue the formula is not a remedy, because it produces a different statistic with no established reference law. The honest options are to report that the statistic is undefined for the planned table, or to state openly that the analysis was changed (for example, that an empty category was removed) and that the new table has different degrees of freedom.
Pearson's statistic and its asymptotic law
Definition 7.5.2
(Pearson residuals and Pearson’s statistic)
When every observed margin is positive, the Pearson residual of cell and the Pearson statistic are
exactly when the observed table equals its own product fit.
Theorem 7.5.3
(Asymptotic null law of Pearson's statistic)
Suppose the pairs are iid, the categories are fixed with , and holds with all . Then, as ,
This is a limit statement. For finite , is a discrete random variable and its law is not exactly chi-square. The approximate p-value is .
The degrees of freedom are often explained by counting parameters. The saturated model has free probabilities, the independence model has , and the difference is . That count is a useful mnemonic but not a proof. The proof below shows where the number comes from: it is the rank of a projection.
Proof
(Projection proof of the chi-square limit)
Step 1: standardized fluctuations. Let and define
The multinomial covariance of is with . By the multivariate central limit theorem, . Pre- and post-multiplying by gives
where is a unit vector because .
Step 2: linearize the fitted product. Write and , and arrange as an matrix . Then with and , where . Hence
Using and , and dividing by , we get
Since exactly, we may add the term for free. The bracket then becomes . In vectorized form this is , with
Step 3: Slutsky. The Pearson residual equals the left-hand side above, except that its denominator is rather than . By the law of large numbers the ratio of the two denominators tends to , so Slutsky's theorem gives with .
Step 4: geometry of . Each factor of is the orthogonal projection onto the complement of a unit vector, with ranks and . A Kronecker product of orthogonal projections is again symmetric and idempotent, and its rank is the product of the ranks: . Moreover . Therefore
Step 5: the norm. Write , where has orthonormal columns. Then with , so the limiting Gaussian is standard on the subspace onto which projects. By continuous mapping, .
Proof X-ray
Why a singular covariance still gives a chi-square
The null hypothesis is used in Step 1, to center at . Under an alternative, has a mean of order and diverges, which is what gives the test power. Positivity of every and is used for the divisions by and in Slutsky's step.
The non-routine move is Step 2. Replacing the unknown by the fitted product is, to first order, a linear map. The exact identity lets a fourth term be added at no cost, and this turns that map into the Kronecker product of two projections.
The limit in Step 1 has the singular covariance , because cell proportions sum to one. This does no harm, because : the one direction in which the CLT limit is degenerate is a direction the projection deletes anyway. So exactly, and appears as the rank of , not as a parameter count.
Two consequences of the proof deserve emphasis.
First, the residuals are not independent standard normals. Their limiting covariance is , whose diagonal entries are and whose off-diagonal entries are generally nonzero. Even at finite the residuals obey exact constraints:
Second, the rank comes from estimating exactly the two sets of margins. If additional parameters are fitted, or if some cells are impossible by design, the projection changes and so does the rank.
Worked example: evidence versus strength
Worked example 7.5.4
(A balanced table)
Observe
Every margin is , so for all cells. Each equals , so
A variable is the square of a standard normal, so the approximate p-value is
At level we do not reject . This is not a demonstration of independence. The data are simply not inconsistent enough with independence, at this sample size, to clear the chosen threshold.
Strength of association. For a table with cells read row by row,
describes association in this sample. It is not a causal effect.
Scaling. Multiply every count by . The proportions, and therefore , are unchanged. However, now and , so . Equivalently, . The approximate p-value is about .
The sample association is identical in both tables. The evidence against the product model differs enormously because grows linearly in at fixed proportions. A p-value measures evidence under a reference law. It does not measure how strong the association is.
Residuals. In the original table every . The residual pattern shows which cells carry the discrepancy (the diagonal is overfull), but the four residuals are tied together by the margin constraints. In this balanced table they share a common absolute value. In a general table the unstandardized discrepancies share a magnitude, while their standardized residual magnitudes need not. Their limiting variance is , so the variance-adjusted residual is , and the table carries only one degree of freedom of information.
Common mistake
(Comparing a single Pearson residual with 1.96)
Declaring an individual cell “significant” because is miscalibrated even for one cell. The limiting variance of is , which is why the adjusted residual divides by an estimate of its square root; in the balanced table above, becomes . When several cells are inspected, a multiplicity correction is needed as well, or a model aimed at the specific pattern of interest.
Worked example 7.5.5
(Reordering the categories of the balanced table)
Swap the two rows of the balanced table, so that it reads . All margins are still , every and every , so again. But
In general a row swap replaces by and the product difference by , so changes sign, while , a sum over cells, only has its terms permuted. Swapping columns behaves the same way. Swapping both rows and columns returns the original table and . The sign of depends on how the categories are ordered; does not.
What the p-value is, and is not
The p-value is computed under and the sampling model, using an approximate reference law: the probability, if independence held, of a statistic at least as large as the one observed. It is not , and is not the probability that the variables are associated.
Common mistake
(Reading “not significant” as “independent”)
A nonsignificant result is compatible both with independence and with a modest dependence that the sample was too small to detect. A significant result is compatible with dependence that is practically negligible, when is large. The worked example shows the first effect: with the table of units is not significant at level , while the same proportions with give .
Do not replace the independence question with a single correlation either.
Worked example 7.5.6
(Zero score correlation with perfect dependence)
Assign scores to the rows and to the columns of
The row mean is and the column mean is , while . The covariance, and so the score correlation, is exactly zero.
Yet the column is perfectly determined by the row. Every row total is and the column totals are and , so and . Rows 1 and 3 each contribute , and row 2 contributes , so . Zero correlation does not establish independence.
When the reference law is wrong
Sampling designs. Fixing the row totals in advance (product-multinomial sampling, as in a study with a set number of units per group) turns into homogeneity of the conditional column distributions. The same asymptotic law holds. Conditioning on both margins gives a hypergeometric reference law that does not depend on the nuisance parameters. That is the basis of Fisher's exact test, which uses a different reference distribution, not merely a more accurate version of the same one.
Clustered data. Repeated measurements on the same person, or patients grouped within institutions, can violate the iid assumption. Within-cluster dependence can change the covariance of counts and invalidate the quoted chi-square calibration; inflation and anti-conservative nominal p-values occur in common settings, but the effect depends on the joint dependence structure.
Data-dependent categories. Choosing cutpoints or merging categories after looking at the table makes the result of a search. The reference law no longer applies.
Structural zeros. If some cells are impossible by design (), then independence with positive margins cannot hold. Testing that null is meaningless. Quasi-independence models on the remaining cells have different degrees of freedom. A sampling zero ( with ) is a different thing and is not a problem in itself.
Small expected counts. Rules such as "every " are heuristics about when the approximation tends to be adequate. They are not theorems, and meeting them guarantees neither accuracy nor any substantive conclusion.
Marginal versus conditional dependence
Counterexample mode
Pooling across institutions with unequal allocation
Two institutions treat different case mixes and assign treatment in very different proportions.
| Treated: success/total | Control: success/total | |
|---|---|---|
| Institution X (milder cases) | ||
| Institution Y (severe cases) | ||
| Pooled |
Within each institution the treated success rate is higher. In the pooled table it is far lower, because most treated patients come from the severe-case institution.
This is Simpson's reversal. The marginal association in the pooled table differs in direction from the conditional association within each stratum. A chi-square test on the pooled table answers a question about the pooled joint law. That law mixes institutions and is not a valid summary of either one. It also gives no causal reading.
Pooling makes things worse when units within an institution are correlated: then the counts are not even multinomial.
Exercises
Exercise 7.5.7
(Test independence for .)
Foundation
Solution(2)
- 1
Expected counts
and , so in every cell.
- 2
Statistic and interpretation
Hence with , and . The sample table factors exactly. That is a statement about the sample, not a proof that the population law factors.
Exercise 7.5.8
(Analyze : compute , and the approximate p-value, and comment on their reliability.)
Core
Solution(3)
- 1
Statistic and effect size
All margins are and , so and . Then
- 2
Approximate p-value
With , the approximate p-value is .
- 3
Caution
The expected counts sit exactly at the usual heuristic boundary, and is small. The conditional (Fisher) reference law gives a two-sided p-value of , computed from a different reference distribution, not from a corrected version of the chi-square approximation. Both values fall below , but they differ by a factor of about three. Moreover, is a sample quantity that is estimated very imprecisely at .
Exercise 7.5.9
(Analyze the table , then the variants and .)
Core
Solution(3)
- 1
The original table
The row totals are , the column totals are , and . The expected counts are and . These match the observed table exactly, so with and .
- 2
Zero cells with positive margins
The margins of the second table are the same, so is unchanged, and the zero cells cause no difficulty. Each of the four cells in rows 2 and 3 contributes , so .
- 3
A zero margin
In the third table row 2 has total , so and the statistic for the planned table is undefined. Dropping the row produces a different, analysis with , and that change must be reported. Inventing a pseudocount is not acceptable.
Exercise 7.5.10
(A study records 20 students, each measured 30 times. Each measurement is classified by condition and by outcome, and a chi-square test on all entries gives . What can be concluded?)
Synthesis
Solution(3)
- 1
The sampling model fails
Key step
The data-collection description does not justify treating all entries as iid multinomial observations. The effective sample size depends on the dependence, design and estimand, and need not be approximately . The p-value therefore has no guaranteed calibration from the stated information alone.
Because: Measurements from one student may share characteristics and be dependent.
- 2
Even a calibrated p-value would say little
A nonsignificant result would not show that the conditional distributions are equal. It certainly would not show that the condition has no causal effect, since causal claims also require a design such as randomization.
- 3
A sound remedy
Treat the student as the independent unit: analyze student-level summaries, fit a model with student-level dependence, or resample whole students (a cluster bootstrap). Each of these still relies on having enough independent clusters and a suitable design. None of them makes validity automatic.
Summary
Pearson's test compares a table with its fitted product of margins, . Under iid multinomial sampling with fixed categories and positive margins, converges in distribution to , because the fitted residuals converge to a standard Gaussian projected onto a subspace of exactly that dimension.
The test measures evidence against independence, not the strength of the association, and it says nothing about causation. Its reference law is only as good as the sampling model behind it.
Prerequisites: the multinomial distribution, the multivariate central limit theorem, Slutsky's theorem, and orthogonal projections.