Who Should Make a Voter AI Chatbot? (Part 3)
After two previous installments on the question: “Who should make a Chatbot for voters?” — we’ve refined the question considerably based on the following 3 observations:
Elections generally, and voter support specifically, is an area of very low tolerance for inaccuracies, hallucinations, and repeating falsehoods.
It’s a terrible idea to build a so-called “lightweight Chatbot” app on top of the existing services powered by current large language models (LLMs) from the AI tech-titans. They’re all polluted with the bad and the ugly of human content that they were trained on. A Chatbot based on any of them is doomed to fail, lie, hallucinate, etc.
It’s also not a good idea for any of those tech-titans to use their considerable expertise and resources to tinker with their own LLM to be an info service to people on a specific topic. We’ve already seen cases where exactly such an attempt failed embarrassingly. The currently available LLMs are too polluted. Yes, there are many techniques being invented and used to constrain these LLMs and mitigate safety risks, but no combination of them will get you anything close to a hallucination-free system.
An Intelligent Digital Service Agent
Based on these observations, the question really shouldn’t be about a “Chatbot” but something conceptually similar to (but so far, nothing like) an “intelligent digital service agent.” Specifically:
An informational natural language agent (NLA);
An NLA that is “domain specific;” that is, with a mission limited to being an information service about one particular topic area (e.g., elections);
An NLA that is built to use an authoritative information base that has been carefully collected and curated by experts in the domain.
For example, incorporating authoritative content on everything about elections in a given state, but only from election authorities: state election code, county election websites, poll worker training manuals, …anything that describes how elections work, and anything that is a useful resource for voters, but only what is available from state and local elections offices, and other government resources.
Built on a base model that is not a general-purpose LLM, but selected for basic natural language processing. However, what’s not desirable, nor a priority (for now) includes human-like conversations incorporating witty, clever, creative responses; rather, stick to boring but trustworthy “just the facts” responses.
Note: we’ll say more about this in the next installment on how this can be done; for the moment we recognize the persuasive power, deep engagement, and perceived authority (along with trust and believability) that conversational-AI is enabling today. And that capability is the epitome of a double-edged sword so, let’s set aside the UX/UI elements and issues for now.
An NLA that uses existing techniques to identify and reject out-of-domain prompts or questions.
And an NLA that Includes some method to limit responses to only what is in the authoritative information base.
A Case of Vujà Dé ?
Does anything like this exist? Well, after reading all of that, it does seem to be the anthesis of Déjà Vu (and even more so as you read-on) because, no, not that I know of (although there is more on this in Part 4.) We think it will be a considerable task to create a very high-accuracy, zero hallucination, domain specific NLA by combining several techniques in a new way.
Given that, and the characteristics of the necessary NLA above, we can begin to see an answer to the “who” question. The “who” likely needs to be comprised of:
A collaboration of several types of experts…
This must include actual experts in the domain; in the case of elections, experts in election administration, election technology, and election education and training.
This also must include broad efforts to gather and curate nearly all, and certainly only authoritative information.
This must also include AI-conversant technical staff to assemble the components into a testable system, and develop detailed test plans.
This must also include skilled user experience and interface (UX/UI) designers and really customer engagement (CE) experts, because a simple fill-in-the-blank, ask-me-anything, interface provides no context or assistance to help the user craft an in-scope question.
We’ll go into detail later on this, but there must be a plurality of ways and means for the voter-user to interact with the Agent in an easy, convenient, non-threatening, and even delightful manner. (And as I mentioned earlier, there will come a time when the “conversational” characteristics of the Agent will factor in.)
And given the scale, nature, and lack of sufficient commercial gain, this is likely to also include efforts from non-profit organizations (or potentially pro-bono contributions or in-kind services from commercial organizations).
Finally, without a doubt, this must also include a diverse set of stakeholder organizations — including, but not limited to, state and local elections organizations (who may be first among equals here) and election NGOs — to assemble a sufficiently broad audience for feedback and testing, as well as review and vetting of the incorporated authoritative information base.
In other words, the “who” is a broad coalition of organizations, for which just based on the above may not even be complete. Readers will surely (and quickly) identify who I am missing here. Yet, even with what we have posited here, it’s safe to say, “No, we have not seen anything like this ever before.”
Is That It?
For public service and policy folks, perhaps. For election technology and policy wonks, not so much. For a reasonably skeptical wonk — and please remember everyone should be skeptical of anything with “AI” — there are several issues that I’ve glossed over, both in terms of functional requirements (including trustworthiness) and technical questions about LLM usage, and of course, technical operations.
To be continued …
