What is the McNemar Test?
The McNemar test is a non-parametric statistical test used to analyze paired nominal (binary) data. It assesses whether the marginal proportions of two binary outcomes are equal when the data are paired — i.e., when each observation in one group is matched with a corresponding observation in the other group. Common applications include comparing two diagnostic tests on the same subjects, measuring changes in a before‑after study, or evaluating the effectiveness of two treatments applied to matched pairs.
Unlike the standard chi-square test for independence, which assumes independent groups, the McNemar test accounts for the dependence inherent in paired data. It focuses exclusively on the discordant pairs — those where the two methods disagree (cells b and c in the 2×2 table) — because concordant pairs (a and d) provide no information about differences between the two methods.
H₀: p₁ = p₂ vs. H₁: p₁ ≠ p₂
where p₁ and p₂ are the marginal proportions of positive results for Method 1 and Method 2, respectively.
When to Use the McNemar Test
- Paired binary outcomes: Each subject or unit contributes a pair of binary responses (e.g., positive/negative, yes/no, success/failure).
- Matched case-control studies: Comparing exposure history between cases and controls matched on confounding variables.
- Before‑after interventions: Measuring the same binary outcome before and after an intervention on the same subjects.
- Diagnostic test comparison: Evaluating two diagnostic or screening tests applied to the same set of patients.
- Inter-rater agreement: Assessing whether two raters (or two measurement methods) produce systematically different binary classifications.
Important: The McNemar test is not appropriate for independent samples. If your data are unpaired, use the standard chi-square test for independence or Fisher's exact test instead.
Formula & Derivation
Given a 2×2 contingency table of paired binary outcomes:
| Method 2+ | Method 2− |
|---|
| Method 1+ | a | b |
|---|
| Method 1− | c | d |
|---|
The McNemar test statistic, without continuity correction, is:
χ² = (b − c)² / (b + c)
With Yates' continuity correction (recommended for small samples):
χ²Yates = (|b − c| − 1)² / (b + c) if |b − c| ≥ 1, else 0
Under the null hypothesis, the test statistic follows a chi-square distribution with 1 degree of freedom (for large samples). For small samples (typically b + c < 25), the exact binomial test is preferred:
p = 2 × P(X ≤ min(b, c)), where X ~ Binomial(b + c, 0.5)
This exact test computes the probability of observing a deviation as extreme as the observed one, assuming that discordant pairs are equally likely to fall in either direction under the null hypothesis.
The odds ratio (OR) for discordant pairs is defined as:
OR = b / c
An OR > 1 indicates that Method 1 is more likely to be positive when Method 2 is negative than vice versa. The 95% confidence interval for the OR is computed using the standard error of the log-odds:
SE(ln OR) = √(1/b + 1/c)
and the CI is: exp(ln OR ± 1.96 × SE).
Step-by-Step Guide
- Organize your data into a 2×2 contingency table with cells a, b, c, and d as described above.
- Enter the counts into the calculator's input fields. Ensure all values are non-negative integers.
- Click "Compute McNemar Test" to obtain the results.
- Review the output: The calculator displays the chi-square statistic (with and without Yates' correction), p-values (both chi-square approximation and exact binomial), significance at α = 0.05, odds ratio, and its 95% confidence interval.
- Interpret the results: If the p-value is less than your chosen significance level (typically 0.05), reject the null hypothesis and conclude that there is a statistically significant difference between the marginal proportions.
- Examine the visualization: The bar chart compares the counts of discordant pairs (b vs. c) and the expected counts under the null hypothesis, providing a visual aid for understanding the test result.
Real-World Applications
Medical Diagnostics: Comparing Two Screening Tests
A hospital compares two rapid antigen tests for COVID-19 on 100 symptomatic patients. Each patient receives both tests. The results show: a = 45 (both positive), b = 8 (Test A positive, Test B negative), c = 15 (Test A negative, Test B positive), d = 32 (both negative). The McNemar test examines whether the two tests have different sensitivity profiles. With b=8 and c=15, the test statistic is χ² = (8−15)²/(8+15) = 49/23 ≈ 2.13, p ≈ 0.144 (uncorrected). At α=0.05, we fail to reject the null hypothesis — there is no statistically significant difference between the two tests in this sample. The odds ratio OR = 8/15 ≈ 0.533 suggests that Test B tends to be more often positive when Test A is negative, but the difference is not statistically significant.
Social Science: Before-After Intervention Study
A public health campaign aims to increase awareness of a health risk. A survey of 200 individuals is conducted before and after the campaign. The paired responses (aware / not aware) are: a = 50 (aware both times), b = 15 (aware before, not aware after), c = 35 (not aware before, aware after), d = 100 (not aware both times). Here, b=15 and c=35. The uncorrected McNemar χ² = (15−35)²/(15+35) = 400/50 = 8.0, p ≈ 0.0047. The exact binomial p-value (two-tailed) for b+c=50, min=15 is approximately 0.007. At α=0.05, we reject H₀ — the campaign significantly increased awareness. The odds ratio OR = 15/35 ≈ 0.429, indicating that the odds of losing awareness vs. gaining awareness are about 0.43, meaning awareness was more likely to increase than decrease.
Agriculture: Comparing Two Pesticide Formulations
An agricultural researcher applies two different pesticide formulations to paired plots (left and right halves of the same field) to compare their effectiveness against a pest. The binary outcome is infestation (present/absent). Data: a = 28 (both effective), b = 6 (formulation 1 effective, formulation 2 not), c = 18 (formulation 1 not effective, formulation 2 effective), d = 8 (both ineffective). With b=6 and c=18, the corrected McNemar χ² = (|6−18|−1)²/(6+18) = (11)²/24 = 121/24 ≈ 5.04, p ≈ 0.025. The exact p-value for b+c=24, min=6 is approximately 0.022. At α=0.05, the result is significant — formulation 2 performs better than formulation 1. The OR = 6/18 ≈ 0.333, with a 95% CI of [0.13, 0.85], confirming that formulation 2 is superior.
Common Misconceptions
- "McNemar test is the same as chi-square test for independence." — No. The chi-square test for independence assumes independent groups, while McNemar is specifically for paired data. Using the wrong test can lead to incorrect conclusions.
- "The test only works for large samples." — While the chi-square approximation is valid for large b+c, the exact binomial version provides accurate p-values for any sample size, making the test robust and reliable.
- "Concordant pairs (a and d) matter." — In the McNemar test, a and d do not enter the test statistic because they represent agreement between the two methods and carry no information about the direction of disagreement. Only discordant pairs (b and c) are informative.
- "A significant result means there is a large effect." — Statistical significance depends on both effect size and sample size. A very large sample can yield significance even for a trivial difference. Always consider the odds ratio and its confidence interval alongside the p-value.
Practical Considerations & Limitations
- Small b+c: When b+c < 25, the chi-square approximation may be unreliable. Use the exact binomial p-value provided by the calculator.
- Zero cells: If b = 0 or c = 0, the odds ratio is undefined (infinite or zero). The calculator handles this by adding a continuity correction (0.5) to both b and c for the OR and CI calculation.
- Marginal proportions: The McNemar test directly tests the equality of marginal proportions. It is equivalent to testing whether the proportion of discordant pairs that fall into cell b equals 0.5.
- Matched design: The test requires that each pair is independent of other pairs. The matching should be meaningful (e.g., same subject, same location, matched controls).
Frequently Asked Questions
The McNemar test is used for paired binary data, where each observation is matched with another (e.g., before/after, two tests on the same subject). The standard chi-square test for independence is used for unpaired data, where the two groups are independent. Using the chi-square test on paired data ignores the pairing and can lead to inflated or deflated p-values.
Yates' correction is recommended when the sample size (specifically b+c) is small — typically when b+c < 30 or when any expected frequency is less than 5. It reduces the chi-square value slightly, making the test more conservative. For large samples, the uncorrected and corrected values converge. The calculator provides both, so you can compare them.
If b+c = 0, there are no discordant pairs — the two methods agree perfectly. The McNemar test cannot be computed because there is no information about differences between the methods. The calculator will display a warning and indicate that the test is not applicable.
The odds ratio (OR = b/c) compares the odds of Method 1 being positive when Method 2 is negative (cell b) versus Method 1 being negative when Method 2 is positive (cell c). An OR > 1 means Method 1 is more likely to be positive in discordant pairs; OR < 1 means Method 2 is more likely to be positive. If the 95% confidence interval for the OR does not include 1, the difference is statistically significant at α = 0.05.
The classic McNemar test is for binary (2×2) data. For paired data with more than two categories, extensions such as the Stuart-Maxwell test or the Bhapkar test are used. These are generalizations of the McNemar test for multinomial outcomes.
References:
Wikipedia: McNemar's test;
McNemar, Q. (1947). "Note on the sampling error of the difference between correlated proportions or percentages."
Psychometrika, 12(2), 153–157;
Agresti, A. (2018). "An Introduction to Categorical Data Analysis," 3rd ed. Wiley.
This calculator implements both the chi-square approximation and the exact binomial test, following the recommendations of the Cochrane Handbook for Systematic Reviews.