We now have a federally certified voting system product that has completed the required testing by a federally certified independent test lab. That's a milestone in itself, as is the public disclosure of some of the results of testing process. Thanks to that disclosure, though, we now know that the test lab did practically zero security testing of the Premier product, because of a gross mis-understanding of prior security review. For a complete, accurate, and brief explanation of the whole situation, I urge you to read this letter to the EAC. The letter is from a group of absolutely top-notch voting technology and/or computer security experts, who were involved in California's Top to Bottom Review (TTBR) of voting systems, which included the Premier system that was recently certified.

At the risk of over-simplifying, the story goes like this.

1. The TTBR found loads of technical security problems and concluded that
   • the security problems were so severe that its technological security mechanisms were unable to protect the system; and
   • the problems could be addressed only with strict procedural security - chain of custody, tamper-evident seals, and the like.
2. Next, the test lab mis-interpreted these conclusions, assuming that the  system's vulnerabilities depended only on effective procedural controls; therefore, no need to test technical security mechanisms!
3. The test plan included no additional security tests, and hence the Premier system passed testing despite the many security flaws found in the TTBR.

That's the gist, but do read the letter to the EAC. It's a fine piece of writing in which Joe Hall and Aaron Burstein set out everything fair and square, chapter and verse. I have to say it's astonishing.

Now, maybe this seems exceptionally geeky, with cavils over test plans and test lab results, and so on. Or maybe it seems critical of the EAC/NIST testing program. But in fact that test program is incredibly important as a gate through which computing technology must pass before being used to count votes. In a very real sense, the current testing program is just getting started, so perhaps it's not surprising that there are many lessons to learn. And my thanks go to all these TTBR verterans for speaking out to remind us how much there is to learn on the road to excellence both of voting systems and of the program to test them.

-- EJS