On election day we cast our ballots, and hear the results that evening. Most votes are counted on computers, but we rarely question the accuracy of these computer-generated results. Who makes sure that these programs are accurate? Let's look at the 1985 mayoral race in Dallas, Texas, to see what can go wrong.

As is frequently the case, the city limits of Dallas had changed since precincts were drawn, resulting in precincts containing both city and non- city residents. This situation is called a Òsplit precinct.Ó Elections officials purchased the wrong computer software to handle these split precincts.

On election night, there was a power glitch which interrupted counting on the central computer counting the votes. The leading candidate before the glitch was trailing soon after the power was restored, arousing the suspicions of one candidate's campaign manager. Because of one of the incorrect pieces of software, the official vote count included three different totals for the number of votes cast, a fact that was only discovered later.

Because the election was close, a recount was ordered. When the votes were recounted, 161 of the 250 precincts showed vote shifts one way or the other, but these largely canceled out one another and didn't change the overall election. Dallas County used pre-punched ballots for this election. A pre- punched ballot is voted by placing it into a jig and punching a pre- perforated "Chad" out. If this Chad is loosened but not removed, it is called a "hanging chad." These changes in votes were blamed on hanging Chad, but no physical evidence to support this contention was ever produced. In several precincts, the number of ballots counted was different from all three previous ballot totals. A defect in precinct procedures has been suggested as the cause, but again, no evidence has been produced.1 Later, charges were raised that all or part of the mayor's race results had been fabricated.2 These have never been disproved .

As a result of this fiasco, at least one election official was soon no longer working for Dallas County Elections. The New York Times ran five articles on the situation, and the Texas legislature conducted hearings, ultimately amending state election law (Texas HB 1412). Dallas County also changed vote- counting systems. Dallas County was not an isolated incident. In fact, the original system the county bought is in use in many other counties today.3

In the U.S., elections are administered locally. While states set the rules for their respective elections, counties, a few large cities, and small units like townships actually register voters, distribute ballots, and count them. Naturally, these jurisdictions purchase equipment and software for the elections they conduct.

The earliest use of computers to count votes was in 1964 when five counties used punch-card ballots.4 In 1970, fraud in the a Los Angeles election using punchcard ballots was alleged.5 An investigation found no evidence of fraud, but IBM, the vendor for Los Angeles' system, decided to leave the business. To this day, most elections systems are purchased from small firms, some with just a few employees.

As of early 1988, about 55% of the popular vote was counted by computer systems. Mechanical lever systems make up the bulk of the remaining 45%.6 By far the most common type are punch card systems, where voters vote by punching holes in a computer punch-card. Systems where voters fill in boxes, as in standardized tests, are also used. Electronic voting machines, though rare now, will tend to replace mechanical lever systems over time. Finally, computer systems often tabulate manual entry of lever machine totals. Thus, except for small jurisdictions, computerized vote-counting systems are or will be the norm.

Computers are used to count votes because they are cheaper and faster. But as Dallas County's experience indicates, the use of computers in itself does not guarantee better elections. Computerized vote counting as practiced today evolved in the "frontier days" of computing. During that period (from the mid-1950's to about 1970), the implementation of computing systems focused purely on the necessary functionality for the application, and systems were operated by experts for experts. Structured design and design for maintainability were not well understood. The 80's have seen dramatic improvements in the reliability and user-friendliness of computer systems, but computerized voting systems continue to exhibit a frontier mentality. This results in three problems: unreliable programs, user-hostile programs, and inadequate administrative controls

Elections computer programs are not subject to design or source code inspections by independent auditors outside the vendor, as banking software is. Some programs still in use consist of unstructured COBOL, patched over the years. In some cases, special purpose code is written for a specific election, then discarded. There are no requirements that the programs be written in a high level language, so assembly language is frequently used. These features make it difficult to determine if the program is designed correctly.

Even if elections officials had access to source code for vote-counting programs, few would be able to obtain the resources to determine its quality.

The users of these systems are the voters and the elections system operators. Though the process of voting seems simple, many systems are dependent on the voter accurately lining up the ballot in a jig so that the vote punches are counted as the voter intends. Elderly and handicapped voters have special problems with these systems.

Elections operators have to lay out the ballot and configure the program for counting, often by binary fields on punch cards. The reports that are produced showing the results are sometimes not comprehensible by anyone but the programmer who designed them.

Good operational practice in the counties is necessary to protect even the best computerized voting systems. Errors in configuration programs frequently go undetected because of inadequate pre-election testing. In one extreme case, a test of 13 ballots each in 2 of 63 precincts for one election was used. This inadequate testing failed to alert election officials to errors that caused them much grief later.7 An audit trail for the counting process is essential, but on some systems it may be disabled or even be an optional software module!

One critical area of weakness is administrative controls on the programs themselves. Controls to insure the object code corresponds to correct source are not in place. Procedures to insure the operating system or other programs running during the count don't affect the vote tabulation are largely absent. Some counties permit dial-up modem access to the computer during the count. Finally, some jurisdictions actually allow the software vendor to operate the system or modify the vote-counting program during the election.

The improvements that election systems need are well known within the computing profession. If elections officials, elected officials and the public want fair elections, why do problems persist? The three forces which could make a difference are frozen in a web of dependencies which stifle change.

Elected officials, primarily at the state level, could pass laws requiring better systems. However, they were elected by the existing elections systems and thus don't tend to question their reliability. Replacing existing systems sounds expensive, and expenditure on vote counting is not of visible benefit to the voters.

Elections officials could make known their dissatisfaction with existing systems. However, they see the computer systems as part of the larger elections process for which they're responsible. It's threatening to question the reliability of elections systemsÑin many cases they could lose their job over real or even alleged problems. Worse, they must choose among the products available, but they often lack the expertise and information to make an informed decision, or to demand better products.

Elections Vendors could provide better systems, but they are small companies, and the market is small, providing little revenue to be plowed back into development. Moreover, the fragmented market of individual counties is not demanding better systems. Finally, for vendors to admit deficiencies would open them to lawsuits or cause existing users to question their investment in current systems.

The technical solutions to the problems of elections system reliability are well within the state of the art in data processing. Some of these solutions were proposed as early as 1970 as a result of the problems with the Los Angeles election.8 Even now, some of those recommendations have not been put in place across the U.S. Numerous reports since have made specific recommendations, which, if applied, would have prevented most of the problem elections to date.

To address the problem of program correctness, elections system vendors need to catch up to current industry practice in system development. This would require vendor quality assurance programs with practices such as source code and design walkthroughs. Independent testing would also be required. The quality of designs and of source code should be examined, and full-system tests should be conducted with a worstcase number of realistic test ballots. Currently, each state does its own testing of vote-counting systems. If this rigorous testing were to be conducted, testing would become very expensive. To make this testing costeffective, states will have to cooperate on testing. Ideally, testing would be done once for the entire U.S. Requiring the use of high-level languages throughout vote-counting programs would also reduce the cost of testing, because experts in dozens of different assembly languages wouldn't be required. Instead, one expert in each permitted high-level language is all that would be needed.

Draft voluntary standards from the Federal Election Commission propose a National Testing Facility that would test each vote-counting program once. The test would include optional sections that would be relevant to only some state election laws. States would ignore the results of sections that aren't relevant to their state laws. However, a test facility need not serve the whole country, at least not initially. A consortium of states could begin a testing facility. States that are concerned about the quality of their vote- counting systems could join the consortium. Regardless of whom it serves, the test facility could be administered by independent testing labs, the National Institute of Standards and Technology (formerly the National Bureau of Standards), a public interest group or another federal agency, as politics determine.

A spring-loaded punch is now available, for use with plain ballot stock that is not pre-punched. This punch completely ejects the "Chad" from the hole to prevent it from floating around loose in the deck and creating random variation in the count. This new punch and ballot are compatible with existing pre-punched vote-counting system, requiring the replacement of only a few components. There is therefore no longer any excuse for pre-punched ballots .

The elections official's interface needs to be brought out of the realm of bits. Elections officials in small jurisdictions are sometimes the county clerk and assessor who are not computer experts, so what might be merely an inconvenient interface in one county becomes a hazard to accurate elections in another.

The input formats of all vote-counting programs should be standardized. Imagine a world where compiler vendors supported only their own proprietary programming languages, and each language was written from scratch, totally different from every other. This is the situation in the vote counting world. Standardizing input languages for vote-counting programs would allow election officials to treat configuration for an election as a generic problem. Training would be vendor independent. In addition, standard tests for vote- counting systems could be devised and run unmodified on several manufacturers' hardware. Many election systems contain a program that prints the ballots for an election. This program should accept the same configuration file that the counting program uses. Besides reducing the work of election preparation, this arrangement would nearly eliminate errors caused by desynchronizations between the ballot-printing program and the counting program.

The report of the election results and the log of election night activities should also be standardized. The same testing and training benefits would also accrue for these interfaces. Furthermore, these outputs ought to be public records (as they are in many places), and therefore must be intelligible to interested voters. At the least, there should be a useable reference available that explains the format of the reports, and the terms used.

Finally, the interface that the voter sees should be improved and standardized, so that voters wouldn't have to learn a new way of voting whenever they move to a new county.

Most elections problems that have been discovered have been at least partly caused by inadequate administration. The best computer program in the world can be brought down by poor administrative practices.

Some jurisdictions have carefully thought-out plans, covering a wide variety of possible circumstances and compiled into books which are actually read and used. Some states, notably Florida, provide guidance (sometimes backed up by legal requirements) in this area. Many jurisdictions are not so fortunate.

One of the administrative practices that has caused the most trouble in the past is pre- and post-election testing. Since a vote-counting system is configured differently for each election, it must be tested prior to use. It is also wise to test the system after use to be sure that no accidental or malign change has occurred in the configuration. For the test to be useful, it must test every ballot position for every ballot style, contain ballots from every precinct, and contain more ballots than the largest number expected.

After the election, this same test should be re-run, and the same (correct) results should be generated. A valuable check on the accuracy of a vote- counting system is a random recount on an independently managed system or by hand. One per cent of precincts should be recounted at a minimum, increasing to one-hundred per cent of precincts when the results are very close.

As computer professionals, we have little direct responsibility for the machinery of democracy. However, when an important component of the election process is inadequate, and this component is a computer system, we have a responsibility as a profession to investigate and speak out.

Of course, it is individuals in the profession who must discharge this responsibility. We can inform ourselves about what kind of system each of us votes on. We can get to know our election officials, tell them about our concerns, and learn about their systems from the election official's perspective.

Computer scientists can also be expert observers of elections. They can be a resource for election officials and elected officials, suggesting ways to make their systems more secure, or evaluating prospective systems.

We can provide an analysis of the problem to public interest groups and the public, and answer questions from decision makers on the available courses of action. This is an important issue, one that cries out for our special expertise. Working together, we can make elections more reliable and deserving of our trust.

1. Roy G . Saltman, Accuracy, Integrity, and Security in Computerized Vote- Tallying, preprint, (Washington, D.C.: U.S. Department of Commerce, National Institute of Standards and Technology [formerly National Bureau of Standards], NBS Special Publication 500-158, 1988), section 4.3.

2. Erik Nilsson, A Bucket of Worms: Computerized Vote Counting in the United States, (Palo Alto, CA: Computer Professionals for Social Responsibility, 1988), p. 3; Terry Elkins, Testimony Before the Texas House of Representatives, November 25, 1986.

3. Lance J. Hoffman, Making Every Vote Count: Security and Reliability of Computerized Vote-Counting Systems, (Washington, D.C.: The George Washington University, 1988), p. 8.

4. Roy G. Saltman, Effective Use of Computing Technology in Vote-Tallying, (Washington, D.C.: U.S. Department of Commerce, National Bureau of Standards NBSIR 75-687, 1975), p. 10.

5. Saltman, 1975, op. cit., p. 16-18.

6. "Type of vote-counting Equipment by County, Elections Data Services and the Elections Center, 1988, p. 1.

7. Saltman, 1988 op. cit., section 4.4.1.

8. Saltman, 1975, op. cit., p. 35-38

-------------------------------------

Erik Nilsson - September 2001

What has a dozen years changed? Since, the last presidential election, everyone knows what a chad is. Roy Saltman’s 1975 paper established the problems with computerized elections and what should be done. In 1988, when this article was written, elections wonks thought we’d stumbled onto this important research that just needed a little attention. But attention hasn’t been enough. People still think elections are simple. During the last presidential election, I spent a small fortune in cell-phone charges, explaining over and over to reporters why computers don’t necessarily count ballots better than people, and why elections aren’t simple. Unfortunately, the recommendations of the 1988 article still stand, because they remain mostly unimplemented. I wish the article was more dated. Since 1988, the pipe dream of voting by phone has been replaced by the pipe dream of voting via the Internet. Internet-voting arguments are more technically refined but just as wrong. In 1988, there were no elections-specific equipment standards. Today, it’s not unusual for equipment to actually have independent testing for standards compliance. People are starting to recognize that elections quality will remain poor so long as it is unmeasured. Rebecca Mercuri started researching voting systems about the time of this article, and today is a forceful voice for reform. Slowly, fitfully, things change, perhaps for the better.

"What's Inside"