Where Is the Reverend Bayes When Needed?
Originalism is an interpretation of the U.S. Constitution holding that only what was in the minds of the drafters should be used in determining what it now means. The doctrine is alive and well today, influencing contemporary decisions of the conservative faction of the Supreme Court and on the stage in Washington, DC, where Justice Antonin Scalia is currently being portrayed in “The Originalist” at the city’s Arena Stage. Might an originalist think that the Second Amendment’s “the right of the people to keep and bear arms shall not be infringed” means that one can walk the streets with automatic weapons, since in 1789 they were not prohibited? What might originalism mean with respect to statistics and the law?
What constitutes admissible scientific—including statistical—evidence has been an evolving concept. The first formal guidance came in 1923 from Frye v. United States: the proffered evidence must be generally acceptable as reliable in the relevant scientific community, as indicated, for example, by having been peer-reviewed. For federal and many state courts, the Frye test has been replaced by a ruling making the judge the gatekeeper for admissibility of expert evidence, evidence that must be “relevant to the task at hand” and “rest on a reliable foundation,” as formulated in 1993 in Daubert v. Merrell Dow Pharmaceuticals. Further, the Daubert court decreed that a conclusion will qualify as “scientific knowledge” only if it can be demonstrated that it is the product of sound “scientific methodology.” What is to be considered in establishing the validity of scientific testimony was codified in Federal Evidence Rule 702 (but as factors, not as a “rule”):
- Empirical testing: whether the theory or technique is falsifiable, refutable, and/or testable.
- Whether it has been subjected to peer review and publication.
- The known or potential error rate.
- The existence and maintenance of standards and controls concerning its operation.
- The degree to which the theory and technique is generally accepted by a relevant scientific community.
Subsequently, in General Electric Co. v. Joiner, in 1997, the court gave trial judges the responsibility to evaluate not just experts’ conclusions, but the experts’ methodology.
How then does the originalism doctrine deal specifically with statistical evidence? Can only that methodology familiar to the Founding Fathers be used in presenting expert evidence?
The U.S. Constitution in Article I, section 2, clause 3 describes the counting of the population for the purposes of the apportionment of the House of Representatives: “The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.” However, problems with the undercount of the decennial census have been recognized since the mid-twentieth century, with most statisticians concurring that statistics-based post-survey adjustments would produce a more accurate “Enumeration.” To do so would no doubt satisfy the “living Constitution” doctrine of Constitutional interpretation, but what might be the opposing originalist view?
In 1976, the Census Act authorized the Secretary of Commerce to “take a decennial census in such form and content as he may determine, including the use of sampling procedures.” However, in 1996 the act was amended to provide: “Except for the determination of population for purposes of apportionment, the Secretary shall, if he considers it feasible, authorize the use of statistical ‘sampling.’” Apparently reading this as permissive, not prohibitive, of sampling for apportionment, leaving it optional for other purposes, the Census Bureau announced its intention to use two sampling techniques to obtain a more accurate count for apportionment in the 2000 census. The U.S. House of Representatives, four counties, and residents of 13 states, some of which were likely to lose a seat if the sampling results were used, sued the Bureau. Affirming decisions in two federal district courts, the Supreme Court barred the use of sampling for apportionment purposes, citing as precedent the fact that for nearly 200 years the Commerce Department had acquiesced to sampling not being used.
In his concurring opinion, Justice Scalia quoted the definition of “enumeration” from Samuel Johnson’s 1773 Dictionary of the English Language—“The act of numbering or counting over; number told out”—as evidence that sampling was not what the drafters of the Constitution had in mind. A different but also plausible originalist interpretation of the Constitution’s “actual Enumeration” might find that it simply meant to distinguish the count to be made after three years from the population estimates made to determine the apportionment of the House the first two years of the Constitution.
While sampling techniques proposed to adjust the census totals were not in use in 1787, sampling is mentioned in the Bible, and in 1786, Pierre-Simon Laplace estimated the population of France by using sampling, along with Bayes’ theorem. In the decision to prohibit sampling there is no suggestion that one must go back to the door-to-door personal contact of the original census takers who swore an oath to make “a just & perfect enumeration,” since mailed-out forms had been in use for several censuses at the time the case came to the Supreme Court.
After the 2000 census, for which no post-survey sampling adjustments were made for apportionment, the state of Utah sued the Secretary of Commerce in part because hot-deck imputation was used to adjust the apportionment count. In Utah v. Evans (2002), the Supreme Court declared that hot-deck imputation—essentially used to create a person where otherwise there would be none—was not in fact “sampling” and so let it stand, even though it was certainly not what was contemplated at the time of the drafting of the Constitution.
The use of statistics in court is not confined to census issues. The fairness of the treatment of a group protected by the Constitution or by statute often must be judged by comparison of its representation among those receiving favorable treatment to its representation in the pool of those seeking consideration of the benefit. For example, the Sixth Amendment (extended to the states by the Fourteenth Amendment) guarantees one the right to an “impartial” jury, which the courts have interpreted to mean a jury with a “fair representation of the community,” without recourse to an originalist interpretation that would certainly have excluded African-Americans, women, and in many cases the economically less well off.
Similar considerations arise in selections for employment actions such as hiring, firing, and promotions, with in many cases considerable dispute as to what groups are actually appropriate for comparison. For example, are minorities overrepresented among those ineligible to vote because of age or other conditions—and thus should not be considered to be part of the eligibility pool for jury service since generally to serve as a juror requires being registered to vote? Voter registration itself is an area where statistical analyses have shown great disparities. In the 2014 film “Selma,” the voter registration drive is motivated by the declaration that 50% of the population of the city of Selma are African American but only 2% of the group are registered to vote. The missing comparison is the no-doubt strikingly higher percentage of registered voters among the white population. The Voting Rights Act of 1965 was designed to remedy such disparities, but voting rights continues to be a problem often addressed by statistical analyses.
Courts have sanctioned both methods of comparing actual or comparative disparities in percentages and what has been characterized as “standard deviation” methods—that is, whether a disparity might have been achieved by chance. There has been judicial notice of problems with methods that do not account for sample size—especially in the case of the Equal Employment Opportunity Commission’s 4/5 rule (“pass” rate for protected group less than 4/5 of that of the other group is considered evidence of discrimination), but little discussion in general of the power of statistical tests. The disparities in percentages have continued to be used as the indication of discrimination or lack thereof even though methods involving probabilities were introduced nearly 40 years ago.
Some issues of disparity address constitutional issues whereas others arise from statutes, as for example disparate impact of employment practices under Title VII of the Civil Rights Act of 1964. Techniques such as regression (which dates back to the mid-nineteenth century) have been introduced, resulting in competing models presented by opposing expert statisticians; however, novel approaches must pass the Daubert test. Even simple procedures have nonetheless often been misused although not often challenged based on originalist-style interpretations as in the census litigation.
As early as 1950, it was suggested that Bayesian methodology was well-suited to the presentation of evidence. Bayesian methods have not yet achieved general acceptance despite continued advocacy for their use; the evolution of accessible software to assist in their use; and even occasional instances of their use—for example in in 1997. Nor would their adoption eliminate controversy, as Bayesians are also known to present rival analyses, as happened in United States v. Delaware in 2004. A court in Great Britain, in the 2010 case R v. T, decreed that “Bayes theorem and likelihood ratios should not be used” (except in DNA cases), because
“[T]o introduce Bayes theorem, or any similar method into a criminal trial plunges the jury into inappropriate and unnecessary realms of theory and complexity deflecting them from their proper task.” (R v. Adams, 1998)
Opposition to Bayesian methodology in the United States rests primarily on misunderstanding of the meaning of prior probabilities, especially in criminal cases, as well as on their computational intricacy. Some believe that it must be contrary to the principle of “innocent until proven guilty” to assume a prior non-zero probability.
It is not clear what originalists make of Bayesian methods, but the Reverend Bayes (1701-1761) and his theorem do predate the drafting of the Constitution.
Mary Gray is professor of mathematics and statistics at American University in Washington, DC. Her PhD is from the University of Kansas, and her JD is from Washington College of Law at American. A recipient of the Elizabeth Scott Award from the Committee of Presidents of Statistical Societies, she is currently chair of the American Statistical Association Scientific and Public Affairs Advisory Committee. Her research interests include statistics and the law, economic equity, survey sampling, human rights, education, and the history of mathematics.
