Towards an AI Research Agenda for Elections and Beyond (Part 2)

Courtesy Getty Images

I recently posted my first part of this longer commentary about the AI research agenda that’s necessary for elections specifically, and a lot of government usage generally. It appears (statistically at least 🤓), that there is interest in the content. Great — keep sending those comments here or to me directly. Last time, I focused on a couple of prerequisite points relevant to the general idea. This time I focus more acutely on AI-driven “domain specific” natural language agents (NLAs), starting with usage in elections. Then, I generalize to make the case that exactly the same research is required for a broad range of government applications — and not in my opinion, but that of a recent Federal government memo to all agencies laying out 2024 priorities for research and development, and led by “Advancing trustworthy artificial intelligence (AI) technology that protects people’s rights and safety, and harness it to accelerate the Nation’s progress.”

Authoritative Election Information, On Demand

Before getting into the research needs, let’s be specific about a couple of ways that a natural language agent could help people with information about elections, so long as the information was true, and also authoritative. Many voters (and others) want and deserve accurate information about a host of election issues:

• How elections are administered and operated in their state;

• What their options are as a voter; how to find if they are eligible;

• If they have a problem then how to find a solution, and

• Much more. 

How does that work today? For a few specific issues, some states have useful online services, like online voter registration. But for the broad range of information desired or required, it’s available on web sites, or via phone call or eMail to the voter’s local elections office. Local election officials (“LEOs”) are outnumbered by voters by several thousands to one (or more in the 50+ large counties that comprise nearly half of US voters). There just aren’t enough hours in the day for LEO staff to do their job as it is, much less if they undertook to operate an “ask-me-anything hotline” — which is why they don’t. 😏 And if there were web pages and documents that contained everything anyone wanted to know, few people would find them and read them (today, readers despise trudging through menus and link-after-link or poor search interfaces nearly as much as they enjoy automated phone attendant trees; “press # to repeat” 🙄). 

Finally, let’s face it, the web content that’s there, and search engines to help locate relevant information, might provide some of the information to some people, some of the time. But if that were sufficient, today we wouldn’t have so many well-meaning voters confused about some of the basics. And then, of course, it’s not reliable. Government websites age out of date — sometimes quickly (its a budget thing), and often are not optimized for search engines. With search, people are just as likely to find other sources of information, less accurate, and in some cases, intentionally inaccurate.

This is a sad truth, 🙁 but one that gets us to misinformation and disinformation. 

Election Lies, and Confusion

The first sad point: election lies and confusions aplenty are part of the large base large-language models (LLMs) currently available, along with the underlying information that enables NL agents to lie, confuse, and just plain make stuff up.

So, if we’re ever going to have a truly useful and nontoxic voter-assistance NLA, first there is plenty of research to be done on a much better underlying technology base than we have today. We need a new LLM that are not polluted and not hallucinatory, but still adequately able to converse with people, even without the facility, engaging wit, and human-like personal expression of today’s NLAs based on today’s LLM.  

The second sad point (but with an optimistic afterburner): even if we had such a safe-and-sane, effective voter-assistance NLA, we would need a 2nd one. That’s right: The second domain-specific NLA would be trained on all the election falsehoods and confusions, trained on them specifically as being false or incorrect or inaccurate. The purpose of this NLA would be specifically to speak to prompts like: “Is <allegation> really true about this election?” or “I read that <allegation> has been part of elections for years.” where “<allegation>” is any kind of currently existing crazy, false, or misguided claim about U.S. elections — and updated over time as new ones come out.

The Research Agenda

So, what is the research agenda for AI used for goodness in elections? I’ve already answered that question by imagineering two AI-based NL agents, just above. To repeat: research on LLMs that are:

• Far more limited in scope, as a result of not being based on a huge collection of text from all over the Internet with all the good, bad, and ugly of human nature recorded therein.

• Not prone to the lies and hallucinations which, at present are the focus of much practical work on constraining existing LLMs with “guardrails” (and so far, easy to bypass) and “Reinforcement Learning From Human Feedback.”

In short, I’m describing less capable, but far less risky LLMs that would be suitable for domain-specific NLAs that are safety-critical and with very low risk tolerance. Some refer to these hypothetical future better LLMs as “Lightweight Large (but not huge) Language Models,” or L3Ms for short.

To be more specific about “domain-specific NLA,” these are natural language agents, that:

• Are developed specifically to be limited to one domain of human knowledge that is largely represented as textual information;

• Are developed with the sole purpose of helping people navigate a specific knowledge base, that:

  • Is too large for a single person to master without a major time investment; 

  • Cannot be effectively served to people via text search and/or custom developed “Mother of All FAQ” content; and

  • Has a well-defined base of authoritative data that can be marshaled as training datasets, and feasible to update over time for retraining.

• Have the capability to be non-responsive to prompts that are not in the scope of the specific domain (as opposed to responsive to anything by default, except where contra-indicated by fallible guardrails);

• Lack the creativity and personability of current LLM-based NLAs, instead responding to prompts in a more factual manner, with predictable and testable consistency of responses to similar prompts.

• Have the capability to provide “evidence” (i.e., references to the authoritative training material that a response was based on).

And if you want to wander into the weeds of responsible use of AI for elections, or for that matter any kind of domain-specific service for public benefit, there’s more on this.

In short, this is a substantial set of goals and requirements, requiring substantial work. As far as anyone I know can tell, this agenda is not being pursued at present. Next time, I explain some possible reasons for that, buttressed by what the venture investors are saying about “tech-titan AI land grabs,” and what the US government is saying about its own wish list.