The Black Liszt

Long-Standing Software Techniques Can Help Election Integrity

There is a massive secret process in election vote counting. It’s invisible, so observers won’t help. It’s subject to error and fraud. Existing standards, even Virginia Governor Youngkin’s Executive Order 35 ignores it. It’s not a hard problem to solve. Standard practices in database and data warehousing software that have been proven and refined in decades of use can be applied. The fragmented group of semi-custom voting equipment, largely driven by software-ignorant bureaucrats and regulators ignores this technology, if they’re even aware of it – no one is asking for it, so why bother? The process is simple in principle; it’s just arithmetic. It’s the process of registering each vote as it’s made, accumulating them with strict transaction logging, and gathering the votes recorded by multiple machines in a location, summing and displaying them in real time, and further gathering the votes from multiple locations and doing the same – by County and then by State.

A version of this process takes place today. But no one will tell you exactly how it’s done. What software exactly is used? How do the votes get from a machine to totals for a location? How is the flow done up the line to larger groups: towns, counties and states? Does someone read a display on one machine and type it into another? What’s the machine that does the summing, using what software? What are the formats used for transmission? No one wants to spell it out, keeping the whole thing invisible.

You think this is trivial? Think about the fact that ballots are different in each town to handle local elections and issues. The ballots are designed and entered into the various machines from different makers that are used. When the votes are reported, exactly how is each vote identified – by the name of the person being voted for? Is the name exactly what appears on the ballot, which is always entered locally? When voting for president, is the president’s full first name used? Middle name or initial? Is the vice president named, and if so exactly how? If there is so much as a single character mis-match in the naming, the totals may not be correct, depending on how they’re done, by whom and/or by what software. The problem gets worse when the votes (and names) are passed up the line to a county, which could have scores of separately programmed machines. Apart from naming issues, the right amounts have to be added to the right places – how is this arranged exactly? Not only is this an opportunity for error, it’s a massive opportunity for corruption and fraud.

The real solution to this problem is to eliminate local machines and controls altogether. This may sound impossible, but it’s not. See this for a description of the approach, with links for more detail. But so long as paper ballot counting machines are used, this problem must be solved to assure election integrity. Fortunately, the problem is one that has been addressed and solved in the commercial computer industry. It’s been used for dozens of years in various forms. The fundamental technology is database (DBMS) technology, of the kind that all the major vendors provide and in open-source. It is available in every major cloud platform. A form of DBMS technology called data warehousing has long been used for accumulating the results of transactions in databases for reporting, display and analysis. Moreover, there are long-established technologies for performing the essential operation of ETL (Extract, Transform and Load). ETL first enables a programmer to view the schema (data definitions) of the source and target DBMS and easily define how the data from the source is transferred and transformed as needed to the destination.

The process starts with the information that goes onto each ballot in the state, each county and each jurisdiction. Each one has a list of candidates and questions that need to go on the ballots. The higher entities would define all the entries that are common to each small entity within it; for example, the state would define state-wide ballot entries, counties would add ones for the county, and so forth. While the ballots are designed, the schemas for the DBMS and Data Warehouse would also be defined. The exact names and descriptions that go onto each line of each ballot would be matched with corresponding data element definitions in the databases (for original recording) and data warehouses (for reporting, accumulating, and rolling up), along with ETL. This could all be tested prior to mail-in ballots and early voting. Moreover, everything about it could be made fully public – the ballot definitions, the schema definitions and the ETL. Multiple processes in multiple (for safety) clouds that implement it could be made visible.

There is no reason why the whole thing couldn’t be made public. The data warehouses could update and display publicly their totals with updates as frequently as desired: each second, every five seconds, whatever. The totals could be displayed simultaneously for each voting locations, precinct., town, county and state. Along with display in real time of the voters who arrive to vote and ballots that arrive in the mail for processing, you have a fully transparent system.

Can databases handle this? Amazon’s cloud database handles over 10,000 transactions per second. Multiple copies could be used, with full redundancy. Capacity is not an issue, nor is reliability or security.

This does not fully solve the problem of intense feelings in local officials and the desire to adjust results to suit their preferences. But it goes a long way to getting those people out of the loop and making the essential back-office operations of accumulating the votes counted by the ballot counting machines transparent, until the proprietary machines can finally be eliminated altogether.

A Key Improvement to Youngkin’s Virginia Voting Integrity Plan

Governor Glenn Youngkin issued Executive Order 35 to assure voting security in the state of Virginia. The summary and the Order itself are worth a read. While some of it talks about existing procedures, together it’s an excellent foundation for assuring election integrity. The procedures include effective voting list maintenance, ballot security and machine testing, using only paper ballots. If all states adhered to this standard, our elections would be more secure than they are.

The biggest vulnerability remains the numerous local voting machines (ballot counters), supporting computers and officials. In this post, I describe the intense feelings some local officials have and their motivation to alter the election results. I propose a solution that eliminates all the propriety hardware with its attendant vulnerability to administrative corruption. Such a system cannot be built in time for this year’s elections. However, there are measures that can be taken this year that will address the vulnerabilities for corruption by local officials that are unaddressed by Executive Order 35.

The issue is the ballot counting machines are proprietary computer-based devices produced in small quantities by specialized companies to meet widely varied state-by-state requirements. In addition, they need to be coordinated with the design of the ballots, which cover not only national and state-wide voting, but also county-wide and often local elections on a single ballot. Each design needs to be coordinated with the ballot machines that will be fed the filled-out ballots. A ballot machine’s administrator sets it up for reading a specific ballot design, with a specific list of candidates and questions. Having the machine certified in general by bureaucrats is useless. What matters is whether the machine has been set up to read the specific ballot types it will be fed, and its ability to do so accurately. Then it matters if the machine is able to recognize that it has been fed the wrong type of ballot and refuse to process it. How are the results stored and given out for totalling? Is it on a display that a human enters into a computer? If so, using what program? How are the numerous ballot types and thus vote totals handled. And how is this able to be displayed along with the voter registrations, as called for in the EO? What computer using what software coordinates with all the local voting place computers to track totals?

I hope that the total transformation of local voting I have proposed happens in time for the next election cycle. Gov Youngkin’s EO handles most of the key issues of voter list maintenance and paper ballot handling and tracking. Paying some attention to the remaining vulnerabilities in the administration of the ballot counting machines and associated data handling would cure the largest remaining vulnerability short of eliminating the proprietary machines altogether.

Voting Integrity: The Problem of Local Control

Suppose some adult who had never played baseball, even for fun as a kid, was quickly taught the rules and played for a day. How well do you think they would play? Suppose they tried again two years later and then again a few times, each with a two year gap between tries. Do you think they would know what the infield fly rule was, much less be able to hit a pitched ball?

That’s what it’s like for the people who administer the voting machines at the more than 170,000 voting precincts in the US on voting day.

You might say it doesn’t matter, because they’re all skilled computer administrators. Sure. That’s like saying they’re good golfers, so when they walk onto the baseball diamond and stand on the pitcher’s mound, it’s no problem.

The kids who practice hard in Little League play the same game of baseball as the New York Yankees, just at a different level. What if the rules varied dramatically from place to place? What if some places had balls, but the bats were flat and there were wickets, like in Cricket, because each league makes up its own rules?

Even worse, suppose that many leagues decided that physical bats and balls were too old-fashioned, and that they should play video games instead? But each county and state would decide the rules for the video game its administrators would control and make different decisions? This is just like the mess with paper ballots and varying levels of automated counting and voting.

What about the makers of the equipment that was supplied for the game that was played every two years? They would have to carefully understand the updated rules each jurisdiction (county, state) would require and update their sporting equipment and video games to follow the requirements – and get it right the first time so that on game day, it would actually work and the inexperienced administrators wouldn’t screw it up too badly. What small company could manage to address such a semi-custom, always-changing market like this and do it error-free?

You’ve got people without experience playing a game once every two years with a wide variety of radically different, changing equipment made by tiny suppliers. That’s not all!

Suppose the point of this voting equipment is to enable local people all over the country to vote on their favorite baseball players. Suppose that officials in each state, county and town (there are over 3,000 counties and over 89,000 local governments) decided what and who should be on each ballot, and exactly how they should be named. There are local Little League players, players on local adult leagues and several levels of professional teams. Officials at each level control what the names are. Suppose one official decides that the player Mr. Berra should be called “Lawrence Berra.” Another decides he should use a more recognizable name, “Larry Berra.” Another decides that a widely used nickname is best, “Yogi Berra.” When it comes time to add up the votes, there’s a reasonable chance that all three Berras could have vote totals. This may sound like making fun, but it happens often in real life!

People feel strongly about baseball. They root for their home team, and can throw vicious insults at players for rival teams. Suppose it’s voting time for the best players. Fans come into the jurisdictions and vote in whatever way it’s supported. The inexperienced administrator controlling the local version of widely varied, changing equipment is in charge of counting the votes correctly, and other officials at the thousands of higher levels are in charge of summing up the votes they’ve been given for people who aren’t named consistently. This is baseball – the stakes are huge.

To give an idea of the intensity of the feelings, there is a widely read book called “Red Sox Fans Are from Mars, Yankees Fans Are from Uranus: Why Red Sox Fans Are Smarter, Funnier, and Better Looking (In Language Even Yankee Fans Can Understand)” How careful and strictly rule-following do you think the voting administrators in the Bronx and in Boston’s South End are going to be working with their peculiar, rarely-used semi-custom machines to do the counting? Not to mention the county and state-level administrators?

Suppose Yogi Berra isn’t elected a favorite baseball player and fraud is suspected as the reason. What can be done? The natural response is to conduct an audit of the voting that is considered suspicious. The trouble is, none of the systems (or people) involved have complete, secure, unalterable audit trails! Someone could have changed the settings on a paper ballot scanning machine and no one would know. If you tried to audit for the whole state, you’d have to go to the many places scanned paper ballots were kept, set up machines for the different rules in thousands of different towns (who’s on the ballot, where they are and how they’re named). Given the problems with inexperienced people and varied equipment, there will be problems during the audit just as there were during the original tally. Auditing electronic votes is even harder, if it can be done at all. And the audit will certainly take a long time, when everyone reasonably wants accurate results soon after voting closes.

Today’s voting and auditing is a highly suspicious mess for many reasons. Decades of incremental changes haven’t made it better. Can anything be done to achieve true voting integrity?

Yes. High integrity voting with near-real-time results can be achieved, but it requires a revolution in the voting process – not much for voters, but a complete re-start of the voting systems and counting process.

The main points of the new system are:

• Eliminate electronic voting machines. These are costly, widely varying devices that require skilled, accurate, bias-free administration.

• Eliminate paper ballot scanning machines that also count the votes; they are also computer systems requiring setup and administration!
  

• If people vote at a voting location, they fill out a paper ballot in private. They take the ballot to a screened-off desk and feed it to a scanning machine connected to a simple off-the-shelf computer with screen. After scanning, they see the image on the screen and can verify it’s accurate, or re-scan. The image is then converted to the intended votes which are displayed for confirmation. Upon confirmation, the voter may optionally be given a print-out record of their votes, like a receipt. The voting totals are instantly updated in a multiple secure Clouds.

• If ballots are received in the mail, the process is much the same, except a poll worker does the work at a voting station that isn’t screened off.

• The voting location is continually monitored with security cameras, including all handling of paper ballots, before and after scanning, and when boxed in a storage room. The rule is that ballots can go into the room but they can't be removed.
  

This revolutionary process eliminates all the local custom equipment and human administrators, with the attendant risk of error and corruption. The new centralized system needs to be designed and administrated carefully to assure effective and secure results.

If the typical government or corporate bureaucracy were in charge of building such a system, we could expect the usual results. But for a small, entrepreneurial group using COTS equipment and software taking an open-source approach with full transparency of results, for example publicly showing vote totals as they were made within seconds, it could be done quickly and rolled out incrementally.

Of course government politicians and bureaucrats would have to support the approach and give up their iron-clad control of the process. But once it was proven in small-scale practice, the only reason to resist would be their support of the existing expensive and fraud-enabling process.

There is a great deal more to be said about how such a system could be implemented, some of which I have described here.

https://www.blackliszt.com/2020/12/how-to-build-a-secure-auditable-voting-system.html

This isn’t the whole story of how to achieve voting integrity. There are the important issues of accuracy of the voting rolls, voter ID and preventing duplicate votes. But a system of this kind would be a big step ahead.

How to build a secure, auditable voting system

 I am a computer and software guy with experience in building systems, networks and security in multiple industries. I know only what I’ve read about the voting systems in place in the US. The basic information that’s widely available is enough to make it clear that today’s voting systems were designed using circa 1990 principles without regard to the security methods that were used by the best systems even at that time. In the light of modern systems and security design they might as well have large, blinking neon signs on them reading “Cheat Me.” This has NOTHING to do with the 2020 election, just as building a secure bank vault has nothing to do with whose money it holds. We ALL care about having safe, secure and accurate voting.

A Modern Voting System

It’s possible to build a voting system that is transparent, fully auditable and extremely difficult to cheat. Each vote would not just be recorded but also logged simultaneously in multiple places, locally and in the cloud and available for public and private review within seconds. This by itself would eliminate huge amounts of potential fraud by getting humans and local physical devices out of the counting loop and above all by ending the secrecy of today’s proprietary closed systems.

Some elements of a secure voting system vary for in-person voting and mail-in voting but ballot counting is the same:

• Custom voting equipment of any kind should be eliminated and replaced with COTS (Commercial Off-The-Shelf) equipment.
• The only local operations that should be performed are voter identification and vote capture
• Paper ballots (and envelopes, if relevant) should be scanned and the encrypted images sent securely, in real time, to multiple locations in the Cloud.
• Assure that each paper has a unique ID and include full log information, things like date/time, source IP, etc.
• All processing possible should be done in the cloud for both in-person and mail voting:
• in multiple places and clouds, with results compared
• fully logged real time in multiple places
• Converting images to votes should be done distributed in parallel
• The knowledge of whether a vote was made and who it was for should be kept separate
• The cloud tallied vote made by a person should be shown to them seconds after they vote in person
• Tallies are updated in real time by voting location including rejections with reason codes, all public
• While some on-site software is unavoidable, it should be minimal and become open source
• All places where voting takes place or ballots are counted should be under total video surveillance, streamed to multiple places, using COTS equipment
• Each jurisdiction enters the voting data uniquely under their control. For example, a state would enter information about the candidates who qualified for the ballot for governor, the US senators from their state, etc. The same would be done at the County and local levels, for example a town would add town council candidates, school board candidates, etc. In each case they would enter all the information that would appear on an electronic or paper ballot and more.
• Each jurisdiction would enter additional voting requirements such as what kind of ID is required, whether signatures are required and/or checked for mail ballots, etc. All such parameters are public and changes are logged.
• The parameters made public actually control the software operation in addition to being publicly readable documents, leaving no room for secretly interfering with the operation of the software.

Scanning and Processing a Ballot Overview

The process of scanning and processing a ballot would be nearly identical whether voting in person or by mail. This by itself is a big step ahead in efficiency and security. This means that a person fills out the same ballot in the same way whether at an in-person center or voting by mail. They see how they voted by looking at where they marked the paper. The only thing done locally by electronics during either in-person voting or mailed ballot processing is scanning the ballot (like taking a picture of it) and sending the unprocessed, encrypted image to multiple secure storage locations in the cloud, along with logging this in the cloud. Once ballot images are sent, stored and logged they can’t be altered or discarded. This reduces to a bare minimum the exposure to error or fraud at the processing location.

The fact that images are converted into votes and tallied within seconds by distributed software working in multiple clouds comparing their results and then showing them to the voter further reduces fraud – each voter can see whether their vote has been tallied correctly.

Various jurisdictions have methods to prevent someone voting more than once today. This is a big subject. Briefly, the key is assigning each qualified voter an ID. The vast majority of places already do this in some way, which is how they can mail ballots to registered voters and make sure no one votes in duplicate. A uniform system of voter ID’s would be created, each linked to whatever local method is in use to minimize disruption. The ID’s would be printed on all ballots mailed and affixed to ballots made available to people voting in-person. The ID’s would be scanned as part of the ballot and processed by software in the cloud immediately. Because this is done within seconds of ballot presentation, attempts at duplicate voting in any way would be caught and prevented from being included in vote totals.

Ballot Processing Center Setup

Most of a ballot processing center would be the same for in-person voting or for processing mail-in votes. In fact, centers could be used for both purposes, even at the same time.

Each center would have a unique name displayed near its entrance. There would an appropriate number of workstations, perhaps tables with chairs, for processing ballots. Each workstation would have a unique name prominently displayed on a sign, for example the name of the location and the number of the workstation. Each workstation would be equipped with a modern high resolution color scanner, preferably duplex (scans front and back in one pass). If for any reason duplex scanning is not practical or available, the operator would scan in two steps, front and back. But duplex is preferable because it eliminates a step and reduces the chance of operator error. There are a variety of COTS scanners with the quality and resolution to handle the job. The scanner would be connected to a simple off-the-shelf laptop computer with a tiny amount of custom software whose only job would be to control the scanner and copy the scanned images to multiple locations in the cloud, with each step logged multiple times. The laptop’s camera would be live and also streamed to the cloud.

In order to minimize the possibility of hacking, the computers used would be recent, bare, off-the-shelf models. Since all the leading laptop operating systems are huge, incredibly complicated bodies of software that have security holes and need regular patches, the software used on it would a small amount of custom compiled C code that would be made open source to enable the software community to find and correct errors and security issues. During room setup the physical machine would be connected to the internet; the custom control code would be downloaded, wipe the machine clean and make itself the only software on the machine. An operator would point the machine’s camera to the ID sign on the workstation, which the software would read, register itself to the cloud and display on its screen. It would then communicate with the cloud control software that would cause other tests to be made. There’s lots of detail here that is beyond the scope of this brief description, but the point is that the opportunities for local shenanigans or errors are brought to near zero.

Each room used for handling ballots would be equipped with multiple modern security video cameras streaming to the cloud to assure that nothing improper was done during ballot processing. Old-style proprietary video security systems would not be used. The video stream would be made available at least to appointed observers and perhaps also to the public.

Voting in Person

A voter checks in much like today, presenting ID as required. If they are deemed qualified to vote they are given a paper ballot with their unique voter ID affixed to it. They go to a private desk area to fill out the ballot. They take the ballot to an available scanning workstation and feed their ballot to the machine. After the ballot is read it is put in a discard bin.

If the voter chooses they can pause at the workstation for a couple seconds and wait for the results of processing their vote to appear on the screen of the laptop at the workstation. When the image of their ballot has been securely stored in the cloud and their votes extracted from the image, validated and tallied, the candidates and issues they voted for are displayed on the screen. The voter knows that not only have they voted but that their votes have been recorded and correctly tallied without delay, and with no further steps required. They can then leave the workstation, pick up their “I Voted” sticker if they like and leave the place of voting.

Additional security would be provided by asking each voter to not just look at the things they voted for displayed on the screen as recorded and tallied, but also to validate that the votes recorded for them do indeed match the votes they think they made. This would be done by giving the user two screen choices: "votes are correct" and "votes aren't correct."

Exactly how re-voting is handled must be done carefully, because it's an opportunity to introduce fraud.  One way to handle it would be for the user to touch the votes on the screen they want changed and once the voter is satisfied to click to submit the corrected vote. I suggest a paper-based method be used to assure that this potential door for hacking to enter remains closed. For example, the voter would take one of a stack of blank "vote correction" sheets, sign it, copy a number onto it that is displayed on the screen and scan the sheet to prove their intention to make a correction. The frequency and distribution of corrections should be monitored closely in real-time to detect bad things happening.

Voting by Mail

Mail-in ballots that arrive by any method would be immediately processed in the same kind of room as used for in-person voting – it could be the same room! There is no reason why the two kinds of voting couldn’t be done at the same time.

The mail-in ballot processing operator would:

• pick up the top envelope from the stack of unprocessed ballots, the in-box
• show the envelope to the camera being sure to avoid obscuring the unique code on the envelope
• This image provides a double-check that the ballot was processed
• scan the envelope (duplex, front and back, in one step)
• remove the contents of the envelope and place the envelope into a processed box, the out-box
• if the contents is a secrecy envelope, scan it (duplex, front and back), remove the ballot and place the envelope in the out-box
  
• unfold the ballot if required and scan it
• place the scanned ballot in the out-box.
• Repeat until the in-box is empty.

Each log entry would contain a date/time stamp, location GPS and name and workstation ID, links to the corresponding images that were scanned and to the place in the video log that captured the scanning process. There would be software running in multiple cloud locations that would process each log entry as it was written and make counts and summaries publicly available via API and web pages. The same software would produce real-time roll-ups so that anyone could follow the progress of ballot registration.

Many states have systems to enable mail-in voters to see whether and when their ballots have been received and then whether they’ve been accepted. Each ballot was printed with a code uniquely assigned to a voter. As soon as the log entry was written for the images the fact that the ballot with the voter’s ID was received would be sent by the cloud software to the state system for updating that voter’s ballot status to “received.” After complete processing, which would normally just be seconds later, the status would be updated as appropriate to “accepted” or “rejected.” The system would provide the state the reason the ballot was rejected, which could include duplicate ballot, lack of signature, ballot not in the right envelope (ID mis-match), etc.

Turning the Ballot Images into Votes and Totaling Them

Ballot images are captured and stored securely in the cloud along with detailed logs in the same way for in-person and mail-in ballots. The only differences are that additional images are stored for mail-ins – the envelopes, both outer and security.

The crucial process of “reading” the images would be performed by software running in the cloud. Multiple copies would be run on different servers controlled by different entities. The output of each major result from the software would be sent by that software to its siblings, all of whom would have to declare that the results matched in order to proceed. They would then “report” their results to multiple queues and  log files. This method of operating copies in parallel and comparing results is an established method of assuring against malicious actors and assuring fault tolerance.

In addition, the software would not be in a single block. The images would be broken up and the different parts processed by different pieces of software. This is a modern approach to building distributed software that is usually called micro-services. It is used to build highly scalable, secure and fault-tolerant systems out of components (called “services”), each of which has a small, isolated job to do. Using this method, the software service that decides whether a signature is present or whether a box or circle has been filled in to indicate a vote has no way of knowing who or what the vote is for, and therefore no way to slant the results. In the unlikely event of a successful hack of one or two pieces of software, it wouldn’t be able to hack the results.

To process the images the software uses modern OCR (Optical Character Recognition) algorithms. OCR is used on both the preprinted text and whatever the voter entered on the ballot. OCR is a mature technology, with software libraries widely available and deployed in production today. OCR is used by bank apps to enable customers to electronically deposit checks using just an image of the check taken by smartphone. Higher quality versions are in broad use today that enable people to scan and submit bank statements, pay stubs, ID’s, and many other document types in order to apply for a loan on-line, become an Uber driver and a host of other purposes. OCR software is no less accurate than human readers who type what they read, and arguably more accurate.

It’s important that the image processing be done in small steps so that no one piece of software processes a whole ballot. This serves multiple purposes including keeping a voter’s votes secret and maximizing the defense against hacking. Once an image was scanned, there are proven techniques for accomplishing this that are faster, more accurate and secure than the processing that is done today for both in-person and mail-in ballots by humans and local machines. Here are the highlights:

• Every image is stored with strong encryption in deep cloud storage with multiple backups.
• Paper ballots today do not normally have the voter’s name on them. The name appears only on the containing envelope. This is  a good practice and should be maintained.
• The image of the ballot itself would be processed by program that would use OCR to “read” all the printed text much like a person would.
• The OCR would pick out each candidate name and issue description and identify the area on the image in which a voter is supposed to fill in a circle in order to vote.
• The same program would create a unique ID for each snippet of the ballot image that the voter could have filled in and write each little image to a new file along with the identifying information, put it on a queue and create a log entry.
• Multiple copies of separate “vote recognition” programs would  be constantly reading the queues and reading the vote snippets. They would evaluate each snippet for whether it had been filled in or not according to uniform standards – without having any information about where the vote was made, which candidate the image was associated with or who the voter was. Each program would then write its results to a queue and log file itself. This file would contain the vote recognition program’s unique ID, the unique ID of the snippet and its judgment of whether or not it had been filled in.
• Separate “vote collector” programs would read the queues of the “vote recognition” programs to gather all the votes in a single ballot together. These would be written to a queue and log of their own.
• The first ballot-reading program would read the collected vote queue, use its data to see which vote was for which candidate as read by it from the ballot and write the final vote tally into a multi-copy log file. The most important data in each log entry is the list of candidates who received a vote. The unique ID of the image would also be in the log entry, linking them to make a completely transparent audit.
• It is essential that this step be performed in parallel by multiple copies of the software running in completely separate clouds and the results only released when all the copies reach consensus.
• If there are enough software copies, say a dozen, then if all but one report the same results, the exception is logged and discarded.
• If something goes wrong with a cloud location or service, so long as most of the services copies are alive the process would be unimpeded.
• Finally, vote tally programs would read the vote logs in real time and update vote totals in real time for anyone to see.
• Each individual in-person vote would in addition be immediately returned to the voting site and specific workstation for display to the person who voted.

The steps described above provide the general idea and specifics for ballots. Some fraction of the votes will be mail-in. They will be marked as being mail-in during the scanning process and will consist of more images; for a typical vote there would be six images, two for each of two envelopes and one ballot. Depending on the requirements set the system would:

• OCR and check the envelope.
• Record the ID on it and assure that the person hasn’t already voted.
• OCR and check the inner envelope.
• Apply the same ID check and assuring that it matches the containing envelope.
• Apply any signature requirements (see below)
• Process the ballot as already described, checking the ID for match.

Using modern technology the entire process just described for either in-person or mail-in should take place in seconds.

Suppose 200 million people voted and all the votes arrived in a 10 hour period, which they wouldn’t. This is 555 votes per second. Suppose that just a hundred machines were used; many times that are available in each of the major cloud services. This would mean that each of the 100 machines would need to handle roughly 5 votes per second. Even with all the parallel and additional processing and checking, this is a laughably trivial load for modern machines and networks to handle. The system would be self-managing like any good micro-service mesh and automatically scale up the servers if and when needed.

This is not a software specification. I’ve written many such documents and am well aware that there are a number of conditions and details I have not addressed. This is an overview to give a sense of the overall approach.

Checking Signatures

I have purposely left the issue of signatures to the end. I don’t want to address the question of to what extent they should be required and checked.

These are the main elements of a solution that can be applied to whatever extent is decided. First seeing whether there is a signature:

• The program that handles the image (probably not the ballot) that can have a signature would perform OCR on the image to identify and extract the portion of the overall image that could contain a signature, much like the ballot processing program extracts the areas on the ballot that the voter could have filled in.
• Much like the process described for seeing if a voting circle has been filled in, a separate program receives the signature image and decides whether there is a signature. The several such programs that look at this image assure that they concur and the results are logged and sent back.
• This information is then read by an additional signed-vote program. It takes the input from the signature-page-reading program and the is-there-a-signature program, and combines it with the input from the ballot reading program, creating the log that the vote tally programs read. This enables them to create separate talles of valid and invalid votes.

If signature matching is also required, additional steps must be performed. In short they are:

• The voter rolls with signatures should be scanned in advance of voting.
• The same physical equipment should be used as for mail-in ballot processing
• The software should get the name and address of the voter, the ID of the voter as used by the relevant authority and an image of the signature.
• Unfortunately, the exact method of doing this may vary by jurisdiction. I don’t know enough about current practice to handle this issue with confidence.
• When ballots are mailed to voters, the ID’s placed on the mailed documents should be put into a secure online table to enable the signature images as scanned during registration to be matched with signatures made on voting documents.
• During vote counting the same process to extract signature images as described about should be followed. The process to determine where a non-blank signature exists should also be followed.
• If the signature doesn’t exist at all, the vote is invalid and should be handled as above.
• If the signature exists, there are two ways to handle it, which could be done in any combination.
• The automated method would be done entirely by software. Probably using a machine language method called convolutional deep learning, neural networks would be trained with huge samples of real-life signatures with multiple signatures from the same person. For example check archives could be used for this purpose.
• The widely used human-in-the-loop method would show workers pairs of signatures on the same screen with no other information. One would be the original signature and the other would be the signature on the mail-in ballot. The worker would enter one key for “match” and another key for “no match.” No other information would be provided. The system would assure that the humans who saw the signatures lived far away from the signers, but in any case the checkers would only see each pair of images for 5 seconds or less.
• Each pair of signatures could be presented to multiple human and/or automated checkers and the results compared.
• This is a huge subject, but elaborations of these basic procedures would produce results that were no worse than today’s physical methods, with very low probability of bias entering into the process.

The methods I’ve described here can be applied to other things the voter may be required to write on an envelope, including for example a date.

Software Controls and Parameters

All software has controls and parameters to adapt to local requirements and conditions. In primitive systems like today’s proprietary machines, each machine is set up by a local systems administrator, who can set and change the machine’s parameters and controls at any time.

In this system, all controls and parameters for all software are contained in a centralized system of tightly controlled and logged editable metadata to control all operations of the system instead of typical administrative control and parameters. This is a key aspect of making a diverse set of micro-services fully controlled and coordinated, while conforming to the requirements and conditions of each jurisdiction. The metadata would be organized in a hierarchy with inheritance, so that rules set at a state level would automatically inherit down and control the relevant aspect of domains within its jurisdiction. The hierarchy would establish a parent/child relationship between blocks of metadata so that counters such as voter and candidate vote totals would automatically roll up. There could be multiple hierarchies enabling for example a voting location to belong to just one town, but the town to belong separately to a county and a congressional district.

The metadata would control exactly which images constituted a mail-in vote, the tests to be applied for validity, the reason codes used for rejection, etc. This is an important aspect of making the system operation fully transparent – the metadata could be used to generate a human-readable document on the web anyone could read.

The controls for creating and editing the metadata are crucial. There would be a CRUD (Create Read Update Delete) matrix between each permission group and each metadata block instance, for example a state. A person who belonged to the permission group for a particular state with C permission would be able to enter and edit candidates and issues to vote for. Since this is done only once per election and the data is so small, it’s likely that such high-level permissions would be restricted to a couple of centralized people with security similar to that for launching attack rockets. Local security would be for creating voting locations and stations. Things like whether signatures are required would be made at the appropriate controlling jurisdiction level. In any case all changes would be made in collaboration with a central group including verbal interaction with multiple people to prevent hacking of any kind.

In all cases setting and changing parameters is highly infrequent but dangerous, which is why gaining access is made burdensome and the results fully public. Changes would be halted at an agreed time prior to an election and before early voting if any.

Because all control parameters and their settings are handled in this way with public viewing of the settings, there is no need to do any software administration and update for any reason, which makes it possible for the software source code itself to be made available for public inspection.

Building the System

The system described here can be built quickly and inexpensively if the appropriate skills and entrepreneurial methods are used. Disaster, delays and ballooning expense would result from a typical corporate/governmental RFP process with endless meetings, reviews and input from “experts.”

Separate teams of a couple people each with appropriate skills could write each of the components/services described here. The toughest skills to get are the currently rare knowledge of bare-machine programming; therefore a preliminary step of running the software on a standard operating system could be used to get a working prototype. There are a few infrastructure things that would need to be agreed to, for example the exact methods for making calls among the mesh of services and otherwise coordinating their activities. It would be best if common tools like redis were used for reliable, super-fast queuing were agreed to and used when appropriate. The metadata control system would need to be built by a single team, but there would not be a much code involved. Its API would be accessed by all software services, probably just once at the start of operation.

The system could first be deployed in small scale with purely local elections for things like home-school associations. Cooperating government entities could make boxes of ballots from past elections available to try out the software.

Conclusion

One of the key benefits of the modern method of voting I’ve described is that it eliminates nearly all of the human setup and administration that is currently performed by thousands of people at vote-processing locations. It also eliminates the many thousands of error-prone human steps that are required to process votes, including things like USB drives that are currently moved by administrators from voting machines to web-connected laptop computers.

While there are lots of details I haven’t filled in, nothing I’ve described here should be foreign to software people on the front lines. Systems much like it are in production at scale in multiple industries. The wide-scale logging, parallel processing and comparison of results are standard methods for assuring that a fault of any kind, malicious or just random, doesn’t cause problems. While everyone, including me, has concerns about hacking, it’s well-known that the worst hacks are typically inside jobs, and taking people out of most of the processing goes a long way to increasing security. The chances of success would be greatly increased by making the software be a kind of open source so that anyone can point to vulnerabilities. For example, open-source Linux software runs something like 90% of the world’s web servers; it’s the gold standard for open and auditable while also being more secure than anything produced by a closed software group.

If a system like this were in use, everyone would be able to be confident that insiders of any variety weren’t using their power over the process to skew the results; except for the identity of the people voting, every step of every vote would be open to inspection by anyone in near-real-time.

Everyone should be able to support bringing voting up to modern standards.