The Statistical Role in Voter Identification (ID) Laws

Abstract

Many public policy issues have at their core statistical issues. It is extremely important that these issues be addressed by statisticians, and their findings must be explained to the public, legislators, and judges. The recent Voter Identification bills passed by a number of state legislatures are a case in point. This article summarizes the statistical issues at the heart of the recent court case between the State of Texas and the Federal government over the Voter ID legislation passed in that state. The Federal court ruled that, “all of Texas's evidence… is some combination of invalid, irrelevant, and unreliable.”

KEY WORDS::

• Response rate
• Statistical matching
• Weighting

1. OVERVIEW

There has been a great deal of publicity recently concerning legislation passed in a number of states which sponsors assert are aimed at reducing election fraud by voter impersonation. The laws vary from state to state, but the basic approach is to require voters to not only present their proof of being a registered voter but also to have one of several forms of additional identification, to demonstrate that they are who they say they are.

In some states (e.g., Virginia), the forms of allowable identification include drivers licenses, utility bills, college student picture IDs, and many other forms of ID. However, in other states, the forms of allowable ID are very restricted. For example, in Texas, SB 14 requires that the IDs must be a (Texas Legislature On-line 2011)

• Driver's license, election identification certificate, or personal identification card issued to the person by the Texas Department of Public Safety (DPS);

• United States military identification card that contains the person's photograph;

• United States citizenship certificate issued to the person that contains the person's photograph;

• United States passport; or

• License to carry a concealed handgun issued to the person by DPS.

Student IDs and out-of-state drivers licenses with photographs are not allowed. In a third of Texas counties (81 of 254), there are no locations where someone can obtain any of the required forms of state-issued ID, and many of these counties have a very high-percentage minority population.

A recent nationwide examination by News21 (Washington Post 2012) of more than 2,000 cases of alleged election fraud since 2000 found only 10 were of the impersonation type that might be prevented by such laws. Four of the 10 were when someone voted for themselves and their spouse. The Republican National Lawyers Association has strongly argued for such laws, referring to a list of 375 election fraud cases. But the investigation found that none of these involved voter impersonation fraud.

Regardless of whether voter impersonation is or is not a major problem, the legal question is whether the laws are constitutional. They are unconstitutional if they have a discriminatory purpose. In general, it is necessary for someone challenging the laws to prove this. However, all or parts of 16 states had a history of discrimination and were thus covered by Section 5 of the U.S. Voting Rights Act of 1965 (U.S. Department of Justice 2012). Under this Act, these jurisdictions needed preclearance from the Federal government before they could change their election laws. The Federal government refused to give Texas clearance, claiming the Voter ID law would be discriminatory. (In contrast, Virginia's less strict law did receive clearance.) So Texas sued the Federal government; but in this case the burden is on Texas to prove the law is not discriminatory.

Notice this very important difference. In general, the null hypothesis is that the law is nondiscriminatory and the burden of proof is to prove it is discriminatory. In the Texas case rejected under Section 5, it is reversed, with the null hypothesis that the law is discriminatory and the alternative that it is not.

These are fundamentally statistical questions, and it is reasonable for statisticians to be asked to contribute to these legal disputes. The author was asked to be an expert witness in the Texas case (State of Texas vs. Eric H. Holder Jr).¹ My experiences are related below.

To try and prove lack of discrimination, the State of Texas had their Secretary of State match their voter list against the State Motor Vehicle list of people with driver's licenses. A survey was then conducted of a sample of those who did not match (thought to be “at risk” of not being able to vote in person under the new legislation), asking if they had any of the forms of ID included in the law.

I was asked to review and respond to the Expert Declaration of Daron R. Shaw. Dr. Shaw is a Political Science Professor at the University of Texas who was hired by the State of Texas to conduct the survey. In particular, I was asked to focus on the two surveys of Texas voters he conducted, one from the entire list of nonmatches, the second with a subset whose last name was thought to be Hispanic. Texas' voter registration list does not identify race or ethnicity, so Hispanic surname was used as a proxy for selecting a sample of Hispanics.

Based on these surveys, Dr. Shaw concluded that, “The bottom-line, then, is that the polling data suggest that between 3.3% and 5.8% of the 795,955 ‘at risk’ registrants are truly ‘at risk.’ The upper bound of this estimate, 46,245 constitutes 0.3% of Texas's 12,892,280 registered voters.”

2. FINDINGS

There were four main statistical concerns with Dr. Shaw's study worth discussing, the first and fourth had the largest impact on Dr. Shaw's conclusion. These topics were matching errors, weighting, survey question clarity, and response rates.

2.1 Matching Databases

Matching databases is a very appropriate procedure. When matching is evaluated, it is important to examine both types of errors (false failure to match and false positive match) to get a full, unbiased understanding.

Matching databases without a common identifier is very difficult and must be evaluated carefully. Dr. Shaw only discussed one type of error (false failure to match) but not the other (false-positive match); both types can be expected from any matching process. (NAS 2010) He found near 80% error from failure to match (claimed to indeed have a Texas drivers license). The 795,955 registered voters who were matched to the Drivers Licence database represent only 6% of all 12,892,280 registered voters. If, say, the false match rate for positive matches (that is, people who the Secretary of State identified as having a State ID but actually don't and thus would be prevented from voting on election day) is only 2 (or 5)% (compared to 80% for failure to match), it would mean another 240,000 (645,000) does not have Texas IDs. So while some of these would have Federal forms of ID (passport, etc.) the “upper bound” of 46,245 might be significantly understated.

Another expert on behalf of the Federal Government, Harvard Government Professor Stephen Ansolabehere examined this issue in depth (Ansolabehere 2012). He conducted his own match of the Texas voter and motor vehicle lists, with more strict matching criteria. (Ansolabehere required exact matches on identifiers, such as name and social security number, or name and birth date.) He identified from 1.5 million to 1.9 million nonmatches (depending upon how he treats ambiguous cases) when compared against the State's drivers’ license and License to Carry lists. This implies that one-seventh of the eligible voters were at risk of not having these forms of identification (Dr. Ansolabehere did not know if these had other forms of allowable ID such as birth certificates or passports) and either would have to present another form of identification or would be prevented from voting.

It is important to note that neither 800,000 nor 1.9 million is the “true” number of people without an ID required by SB 14. The huge discrepancy between these two estimates demonstrates how difficult matching two lists without common numeric identifiers can be. The only way to accurately determine the number without an ID is to interview a sample of the population and ask them directly.

Ansolabehere used a private vendor (Catalist LLC) to provide its best estimate of the race/ethnicity of the 1.9 million at risk of not having a required ID. Catalist also has estimates of the likely racial identity of all persons on the Texas voter list. While a little more than 60% of all Texas-registered voters were thought to be White and 35% Black or Hispanic, the 1.9 million possible nonmatches were only 50% White and more than 45% Black or Hispanic. Minorities were a statistically significant 1.5 times as likely to be on the list as Whites.

Whether they not only are on the list of nonmatches, but also do not have any of the forms of ID allowed by SB 14 will be discussed further in Section 2.4.

2.2 Weighting

Dr. Shaw presented results of his surveys including both weighted and unweighted estimates (Shaw 2012). He presented the unweighted results because he did not “have good information on the demographic characteristics of the underlying population of interest,” and “the definition of the at-risk population is problematic.” It is true that the list of nonmatching records only had age, not race. For ethnicity, only the likelihood that a surname was Hispanic was available. However, presenting unweighted results can be very misleading when the likelihood of responding was not equal among all those surveyed.²

Weighting survey data can be very important to help reduce nonresponse bias when there are differential response rates. Table 1 (taken from Table 7 of Shaw 2012) shows that his General survey was much more likely to include responses from Whites and the elderly than other people. Whites were three times more likely to participate than those with a Hispanic surname, those 65+ are 3.6 times as likely as 18–29 or 45–64-year olds and 22 times as likely as 30–44-year olds (key differences are shown in bold in the table). For the Hispanic surname survey, race/ethnicity seems consistent, but those 65+ were eight times as likely as 30–44 and five times as likely as 18–29-year olds.

Table 1. Weighted and unweighted demographics

	General sample		Hispanic surname sample
	Unweighted	Weighted	Unweighted	Weighted
Anglo/White	57.7	38.8	5.8	5.0
Black	15.8	14.9	1.0	1.7
Hispanic	17.0	34.9	89.5	90.8
Other	9.0	8.0	3.7	2.3
18–29	7.9	12.1	6.3	13.3
30–44	2.5	23.5	7.8	26.2
45–64	22.6	35.6	28.0	36.8
65+	67.0	28.5	57.8	23.7

As demonstrated by the differences in Table 1, Dr. Shaw should only have presented weighted survey estimates of number of Texans without valid IDs.

It is very important to use weighted numbers, rather than unweighted, to adjust for this overrepresentation of Whites and the elderly in such surveys. Differential weighting does increase the margin of error due to sampling (Kish 1965), but this is generally well worth the trade-off for reduction in nonresponse bias.

2.3 Response Error

Another source of possible error in surveys is whether the respondents properly understood the questions. This can be evaluated by comparing survey results against known totals. One of the exceptions to the requirements in SB14 is that a photo ID is not required if you have been determined to be disabled by the U.S. Social Security Administration (SSA). Those who did not report having any of the allowable IDs were asked, “Has the U.S. Social Security Administration determined that you have a disability?”

The SSA reports that 3.7% of the 18–64-year-old population in Texas was disabled beneficiaries (SSA 2010). Unfortunately, since this question was not asked of all respondents (only of those who did not have any of the other forms of ID), the answers are not directly comparable to SSA estimates for Texas. Still, 69% (weighted) of the general survey and 11% of the Hispanic surname survey answered affirmatively.

To show how unreasonable these responses are, assume that those without an ID are like the overall sample, 39% Hispanic surname (the following computation is only worse if we assume those without an ID are more likely Hispanic). If that group has the 11% disability rate reported in the Hispanic surname survey, the only way for the overall estimate to be 69% is for 106% of the non-Hispanics to have a disability (39%*11% + (1 – 39%)*106% = 69%).

Clearly, there is something wrong with the responses to this question, or those who do not have any of the forms of ID are much more disabled than the population in general. Response error is an example of nonsampling errors that have to be considered when using surveys.

2.4 Response Rate

The fundamental problem with the Texas survey results is that they have a 2% survey response rate. Out of more than 44,000 names with telephone numbers selected for inclusion in the General survey, only 1,102 were completed. Given that 27 of the numbers were listed as “untouched,” it appears that only one attempt was made to reach each person. This approach will include mostly those who are regularly at home and willing to participate. Dr. Shaw's Disposition Report for the two surveys can be collapsed as shown in Table 2. I have collapsed codes that indicate completed interviews, those that could be tried again, bad telephone numbers, refusals, ineligibles (death), and those of unclear status.

Table 2. Disposition codes from two surveys of Texas voters

Response	General	General	Hispanic	Hispanic
category	pop	pop%	surname	surname%
Completes	1,102	2.5%	600	2.2%
Try again	30,338	68.5%	19,818	72.3%
Bad number	8,797	19.9%	5,854	21.4%
Refusal	1,493	3.4%	318	1.2%
Unclear	2,571	5.8%	821	3.0%
Total eligible	44,301	100.0%	27,411	100.0%
Ineligible	283		136
Total	44,584		27,547

As a comparison, here are recent typical response rates for Federal Government-sponsored surveys, such as those done by the Census Bureau or Westat, where the data are to be used for policy purposes.

• High quality in-person surveys (e.g., American Community Survey (ACS) or the National Health and Nutrition Examination Survey (NHANES)) achieve response rates at or above 80%.

• Telephone surveys from a list of phone numbers typically achieve response rates of 30–50%.

• Telephone surveys using random digit dialing (RDD, not a list of known phone numbers) where anyone can answer for the household achieve 30–50%.

• RDD surveys with a randomly selected adult respondent needing to respond (so a call back to reach that specific respondent is often needed) can achieve 20–30% percent.

The U.S. Office of Management and Budget requests that any “survey with an overall unit response rate of less than 80% conduct an analysis of nonresponse bias.” (OMB 2006) OMB will not approve surveys with much lower response rates without extensive documentation of why there is not an alternative.

The key point of my testimony was that the Federal government has clear standards; policy must be based on high-quality data. Poor quality surveys cannot be depended upon because their findings are highly suspect and subject to large potential biases.

The Federal court agreed, “The results of surveys with such low response rates, Dr. Marker bluntly concluded, are ‘really irrelevant.’ Thus, taking our cues from the polling industry itself, we have little trouble finding that Dr. Shaw's response rates fall well short of acceptable.” (U.S. District Court 2012, p. 40)

The importance of achieving a high response rate is that it protects against the situation where the nonrespondents are different than the respondents even after what can be done with weighting. So are the Hispanics that Dr. Shaw was not able to reach different from those he was able to reach? The same question can be asked for Whites, Blacks, and different age groups. Typically, the answer is that for many questions respondents and nonrespondents are very similar, but for a few key items they can be quite different.

As one example, NHANES spends considerable resources trying to maximize its response rate because research has shown that those that are harder to get to participate have very different health conditions.

As the Federal court quoted the author in its decision, “[weighting] only goes so far. The issue is, are the young people who responded, the 2%, are they like all of the other young people? Are the elderly who responded like the elderly who did not? Are the Blacks who responded like the typical Blacks? And that doesn't get help by weighting.” (U.S. District Court 2012, p. 41)

Dr. Shaw reported that “3.3%–5.8% of the 795,955” don't have an allowable form of ID. Let us assume the low end of this range, 3.3%. But that is based on his 2% response rate; what if the remaining 98% had a higher rate? If the nonrespondent's rate is higher, the overall Texas rate will reflect that second rate, not that found for the 2%. Table 3 demonstrates how poorly a 2% response rate can reflect the true levels. If, for example, the nonrespondents have a 30% rate of not having an ID, the overall Texas rate would be 29.5%, not 3.3%. In contrast, if the survey had obtained a 50% response rate and the nonrespondents had a 30% rate of no IDs, the overall Texas rate would be 16.7%. If the survey had an 80% response rate, the overall Texas rate would be 8.6%, much closer to that reported by survey respondents.

Table 3. Percentage without a photo ID as a function of response rate

2% Response rate	98% Nonresponse rate	Total
3.3%	3.3%	3.3%
3.3%	10.0%	9.9%
3.3%	20.0%	19.7%
3.3%	30.0%	29.5%
3.3%	40.0%	39.3%
50% Response rate	50% Nonresponse rate	Total
3.3%	3.3%	3.3%
3.3%	10.0%	6.7%
3.3%	20.0%	11.7%
3.3%	30.0%	16.7%
3.3%	40.0%	21.7%
80% Response rate	20% Nonresponse rate	Total
3.3%	3.3%	3.3%
3.3%	10.0%	4.6%
3.3%	20.0%	6.6%
3.3%	30.0%	8.6%
3.3%	40.0%	10.6%

This uncertainty is why no government policy is based on surveys with such a low response rate. There are many reasons to hypothesize that the nonrespondents are indeed different from respondents, even after controlling for ethnicity. They may well be more poor and less connected (telephone and otherwise). We do not know for sure, but we normally protect against that uncertainty by putting in the resources to get a higher response rate.

3. CONCLUSIONS

Many public policy decisions require high quality statistical information to make informed decisions. There has been a great deal of research into how to improve the collection, processing, and analysis of survey data (Lyberg (2012) or Groves et al. (2009)). Statistical procedures and survey data that are consistent with these approaches are more likely to meet the needs of policymakers and judges. Unfortunately, the data presented in State of Texas vs. Eric H. Holder Jr. did not come close to meeting those standards.

To show that SB 14 will not adversely affect minorities, students, or other subpopulations, one needs to first estimate the number of potential Texas voters who may not have a State-approved ID, then compare their incidence rate across the different subpopulations. The Texas Secretary of State estimated the upper end of this population as approximately 800,000 people (some of whom may have had an acceptable Federal ID), while an alternative definition of a match estimated this population as up to 1.9 million. Statistical matching is difficult. When estimating the number of people who might be affected by legislation, it is important to recognize that there may be false positives as well as false negatives from any matching operation.

Survey findings need to be weighted to control for some forms of nonresponse bias.

Pretesting surveys is always important. This is even more so when the population affected is thought to have limited English language skills. Nonsampling errors (including misunderstood questions and differences between respondents and nonrespondents) can have a major effect, so the margin of error of a survey cannot be limited to the sampling error. Attempts should be made to minimize nonsampling errors and to estimate the size of remaining errors.

It is very important that efforts be made to conduct a high-quality survey of the population to estimate the true impact of legislation. If in-person surveys cannot be conducted, then at the very least multimodal surveys with multiple contact attempts must be conducted.

The Federal court understood the importance of the null hypothesis and the need for high-quality survey data (or the effect of poor-quality data). “Texas bears the burden of proving that nothing in SB14 ‘would lead to a retrogression in the position of racial minorities with respect to their effective exercise of the electoral franchise.’ Beer, 425 U.S. at 141. Because all of Texas's evidence on retrogression is some combination of invalid, irrelevant, and unreliable, we have little trouble concluding that Texas has failed to carry its burden.” (U.S. Court, 2012, p. 44)

On June 25, 2013, the U.S. Supreme Court ruled unconstitutional Section 4 of the 1965 Voting Rights Act. This is the section under which the State of Texas was required to prove its Voter ID law did not discriminate. The Supreme Court ruling effectively negated the federal court ruling against Texas. Two hours after the Supreme Court decision was announced, the State of Texas announced that they would enforce the Voter ID law that had been ruled unconstitutional. On July 25, 2013, the U.S. Attorney General announced that they would argue that under Section 2 of the 1965 Voting Rights Act a judge could still require Texas to get preclearance, and that they would push for this to happen. The final determination of this case and the status of preclearance in other states had not been decided when this article was published.

Simultaneously, some state Voter ID cases are being brought to court for violating provisions of state constitutions, rather than the U.S. constitution (e.g., Applewhite et al. v. the Commonwealth of Pennsylvania). In these cases, the burden of proof is on the plaintiffs, who must show the state laws are unconstitutional. Both the federal and these state cases are going to be decided in the next few years. The burden of proof will vary from case to case, but all will require judicial decisions that will turn on statistical evidence.

Voter ID laws are just one of the many public policy issues that end up in the court system. The role of statisticians can be vital in clarifying for judges the actual findings, and limitations, from studies of their impact. It is extremely important that professional statisticians get involved in these issues; apply their knowledge to improve the quality of the studies conducted; and clearly describe the findings that can be drawn from the studies.

Notes

I testified as an expert witness on behalf of the Texas State Conference of the NAACP Branches and the Mexican American Legislative Caucus of the Texas House of Representatives, intervenors on behalf of the Federal government.

After Dr. Ansolabehere conducted his matching, Dr. Shaw conducted a second set of surveys based on this revised database. These had similar results and quality although only unweighted results were presented to the court. This article only discusses the findings from the initial two surveys.