How Authoritarians Will Use AI
And how we can use it against them.
CLEARVIEW, A BIOMETRIC DATA COMPANY, launched its facial recognition systems in 2017, offering a free trial to police departments. By 2020, the New Jersey Attorney General’s office, the Los Angeles and Chicago police departments, and many Canadian police departments all refused to use Clearview technology within their jurisdictions, and with good reason. They saw the risk—to their constituents, their reputations, and their own ability to withstand the temptations of power.
“For practical applications, Clearview AI was all the rage,” said Peter Sloly, former chief of the Ottawa Police Service, in an interview for The AI Dilemma: 7 Principles of Responsible Technology, which I coauthored with Juliette Powell. “But police chiefs actually held themselves in check and voluntarily put it back on the shelf. The police decided they shouldn’t have it, let alone use it.”
Stories like this represent rays of hope amid the convergence of two great fears: anti-democratic governments and artificial intelligence. The Economist counts 76 elections with nationwide votes taking place in 2024, in countries ranging from Iceland (“full democracy”) to the United States (“flawed democracy”) to North Korea (“full authoritarian”). When candidates in any of those elections talk openly about retribution, it’s only natural to wonder how they might use AI to identify, track, and persecute their opponents.
A worst-case scenario would see more governments routinely and ruthlessly weaponizing the technology until it becomes standard practice, even in democracies. A more optimistic scenario suggests that AI will, in its chaotic and competitive way, lay the groundwork for a future of more transparent and civil politics, where activities of all institutions, including government and big tech, are more accountable to the public at large.
There are good reasons to think the optimistic scenario will prevail. However, even at best, it will bring with it a heavy burden of responsibility for tech companies, engineers, governments, and the rest of us. A dystopian political future of AI isn’t predetermined, but it’s all too plausible. To avoid this, there need to be solutions at many levels of society, with standards of responsible practice for business and government, and also tools that make it easier for the rest of us to resist algorithmic manipulation.
TO BE CLEAR, AI ITSELF IS NOT THE THREAT. Artificial intelligence is the group of autonomous, automated, algorithmic systems increasingly in use today, not intelligent in a human sense, but growing rapidly in capability. AI has already enabled many powerful, positive developments for humanity. But it also comes with immense risks, especially for vulnerable people. The track record of big tech companies in self-regulation so far has been inadequate at best. Governments, even sometimes in full democracies like the Netherlands, have also been dismal.
Facial recognition is a good example. It is a rapid, efficient, and inexpensive way of identifying people—matching an image against others taken from social media and closed-circuit TV. It’s an invaluable tool, for example, for identifying victims in large-scale disasters or investigating human traffickers. But governments also use it routinely to track, arrest, and penalize suspected dissidents and members of targeted groups. It’s consistently prone to errors and false identifications, especially for women, darker-skinned people, children, and the elderly, who are generally underrepresented in the facial images used to train the software.
These errors help explain the urgency of the backlash against facial recognition. For example, the American Civil Liberties Union lawsuit forced Clearview to stop selling to private companies, and the 2021 Canadian Privacy Commission ruling outlawed Clearview software in Canada. Seminal “gender shades” research by Joy Buolamwini and Timnit Gebru, and the outrage it provoked, have convinced some generative AI (GenAI) companies to alter their data sets to reduce the ingrained bias favoring white men. Yet there’s still a lot of inherent bias in the data, much of which was gathered from social media: A recent test of the image generator MidJourney found that requests for a picture of a Mexican person consistently yield pictures of a man in a sombrero. Ask for an Indian person and you’ll get a white-bearded man in a turban. Facial recognition and biometric data are likely to be explicitly prohibited by the European Union’s AI Act, when the EU Parliament ratifies it, as expected, this year.
Governments and political campaigns also misuse other types of AI. A recent Freedom House report listed dozens of examples. The Philippines, Cambodia, Iran, Vietnam, and India have restricted or blocked access to independent news or social media channels. Myanmar, China, Belarus, Nicaragua, Saudi Arabia, and 50 other countries put people in prison for their online posts. Forty-seven countries, including the United States, had active disinformation campaigns using GenAI hallucinations, including fake images of Donald Trump embracing Anthony Fauci and fake video of Joe Biden making transphobic comments. We saw another example before New Hampshire’s presidential primary, when deepfake robocalls used ElevenLabs’s AI voice generation software to simulate Biden telling people not to vote.
Just about every one of these activities could be (and in the past, has been) accomplished without AI, of course. As Rory Mir, a spokesperson for the Electronic Frontier Foundation, put it, “There are already coordinated misinformation campaigns and bots putting out messages against vaccines or to affect the climate change discussion. AI just makes all this more unique, harder to detect, and more widely present.”
GenAI media tools are particularly powerful tools for disruption. They offer dictators what they offer the rest of us: broader reach, more productivity, easier targeting of specific people, and less immediate human oversight. Most of them, like most of us, probably haven’t tapped into its power because the technology is still so young and there hasn’t been time to experiment with it.
“It’s not about writing one blog post with ChatGPT and you doing that manually,” says marketing agent Mike Taylor in his prompt engineering course on Udemy. “It’s about generating 1,000 blog posts at once and publishing them all. It’s not about creating one nice image for the website. It’s about scaling it up so you have a good image for every single page on the website automatically when you publish it.”
WE’RE JUST BEGINNING TO SEE the possibilities of AI threats at a larger scale. Many of them have to do with gaining and misusing personal and institutional data, including images and videos. Deepfake scammers stole $25.6 million from a Hong Kong bank in early February, using GenAI to fake a videoconference with the company’s CFO. They could do this because they had enough data about the CFO and the bank’s practices, apparently from internal video conferences, to make the simulation convincing.
For a private scammer, getting that kind of data presumably requires a lot of effort. Governments can just commandeer it—or any personal data—directly from big tech companies. Even in the United States, where the Fourth Amendment prohibits unreasonable searches and seizures, there are few legal constraints on public-sector collection, purchase, and use of personal data—including browsing and purchase histories—that have been gathered by commercial entities like Google, Facebook, and Netflix.
According to Jonathan Askin, a Brooklyn Law School professor who is also founding director of the school’s Law Incubator and Policy Clinic and the Brooklyn Justice Lab, this is a failing of the “third-party doctrine” exception to the Fourth Amendment. That doctrine, established by the Supreme Court in the 1970s, enables the government to gather information willingly disclosed to “third-party intermediaries,” including corporations, without a warrant. “We need a better way to participate in social media environments,” he said. “There should be trusted intermediaries to protect your data, through which you will not have made a demonstration that you've given up your privacy rights.”
With that kind of data in its AI systems, a government could exert control over people by manipulating even minor aspects of daily life. The most widespread chilling effect probably wouldn’t come from the risk of imprisonment, but from minor cues and snags in the world around us. Travel restrictions, credit checks, and zoning regulations could be applied with more fine-grained detail and subtlety using an individuals’ weaknesses and vulnerabilities—a debt, a secret dalliance, a troublesome browsing history, or a child needing a costly medical treatment.
Steven Feldstein, an expert on democracy and technology at the Carnegie Endowment for International Peace observed that using AI for this kind of micro-repression “requires considerably fewer human actors than conventional repression, entails less physical harassment, and comes at a lower cost.” The government wouldn’t even need a human decision-maker to choose the targets. An AI system can pick targets and execute the harassment, while those in power deny that it is even happening.
AI can also be used to target neighborhoods and groups more efficiently. Already there are systems like ShotSpotter, a sound recognition system that collects noise from microphones in public places, alerting law enforcement when gunfire erupts. “When they hear a car backfire,” Mir said, “most people would look out the window and use their judgment about whether to call. Now [ShotSpotter] increases the presence of law enforcement when no one has requested them, which is particularly problematic when it’s people at risk of police violence.” Sound recognition systems, like other forms of digital surveillance, are also prone to misidentification, and false alarms also draw first responders away from real emergencies. An unscrupulous politician could raise the crime rate statistics for a neighborhood simply by placing the microphones there.
If you think you’d be protected because of your own status or income, consider a story recounted by legal scholar Brett Frischmann and philosophy professor Evan Selinger in their book Re-Engineering Humanity: A young boy came home from an upper-middle-class, suburban New Jersey elementary school, ecstatic because he had been selected to wear a free “activity watch” that tracked his heart rate, body temperature, and movement. He was not supposed to take it off, even when showering or sleeping. The school enclosed a letter explaining that the activity watch program was funded by a grant from the U.S. Department of Education designed to help students make progress in physical education and weight loss. The school would compile the data from the watches to help judge the effectiveness of its program.
The program’s intent was benign, but nonetheless manipulative. “They present the idea that you could choose otherwise than to wear the watch,” says Frischmann. “But in the majority of circumstances, you’ll never choose otherwise.”
The illusion of choice is one of the most problematic aspects of AI systems even in democratic societies. Digital tools almost always represent the path of least resistance because they are designed to be frictionless—to make it easy to respond without paying attention. People also tend to regard digital deductions as authoritative. Studies have found, for instance, that many people trust life advice from a GenAI therapy bot more than the guidance of a human therapist. The speed and convenience of an AI tool makes it seem like we are in control when using it. However, like the child wearing the activity watch, we’re not really in control of what gets measured or how the data are used. In a more authoritarian society, the watch could be distributed as a signal of trust, and then used to track movement or record conversation.
FOR ALL THE STRENGTH OF THE AI WAVE, there is a democratic undertow. The uncontrollable nature of unsupervised data on which large language models are trained leads to unexpected results. University of California at San Diego political scientists Eddie Yang and Margaret E. Roberts call this the “authoritarian data problem.” They point out that the best GenAI systems are typically trained on diverse data from many sources. In tightly controlled regimes, censorship reduces the breadth of the data stream. That makes them less effective at providing answers to challenging problems.
Some tightly controlled governments solve this problem by incorporating data from other countries. However, that opens up access to censored information. This may be why ChatGPT is banned in China, Russia, Iran, North Korea, and other authoritarian states. Even their officially sanctioned GenAI systems leak. “ChatYuan, a chatbot created by Chinese AI companyYuan Yu,” writes Feldstein, “generated numerous problematic responses, such as listing China’s economic problems and naming Russia as an aggressor in the Russia-Ukraine war. Unsurprisingly, Chinese authorities shut down ChatYuan within days of its release.”
AI can also be used to counter the influence of misinformation, including the misinformation generated by other AI. Tools are now on the market claiming to detect GenAI text with 99 percent accuracy. The rapidly growing field of visual deep-fake detection uses techniques such as frame-by-frame video analysis—for example, an eye that blinks at a different speed than a human eye is a signal of deception. Meta just announced a technical standard for photorealistic AI images, audio, and video, requiring Instagram, Facebook, and Threads users to identify the provenance of what they post. Standards like this could help counter misinformation.
Humans may get more sophisticated over time at detecting fakes as well. Studies at the University of Pennsylvania and Germany’s University of Passau have found that people can learn to distinguish AI-generated text from human-generated text. The skill appears to increase with exposure to the apps like ChatGPT. Image detection may be trickier, and of course if the technology improves, our detection abilities may not keep up. Nonetheless, some people do seem attuned to catch deepfakes. The fake Biden robocalls in New Hampshire came to light because they raised red flags with people who received them—and who then verified the issue by calling the campaign.
All these AI abuses have triggered responses from governments that seem ready to adopt a more stringent standard—at least rhetorically. In November, representatives of 28 governments signed the declaration of the Bletchley AI Safety Summit, saying that AI should be “human-centric, trustworthy, and responsible.” Signatories included the UK (which hosted the meeting), the United States, China, the EU, Saudi Arabia, and Israel. This was yet another sign that government leaders recognize they will be held accountable in some way for this technology. It should not be surprising that an unprecedented number of AI-regulating laws are proposed in democratic systems like the EU, the United States, and Canada, even as tech pundits dispute the ability of politicians to understand what they’re regulating.
WE NEED BETTER LAWS AND STANDARDS, not just to constrain companies, but governments as well. We need prosecutable laws against AI-generated misinformation used to stalk, defame, or harm people (a popular issue after the Taylor Swift deepfakes). We need more transparency in AI systems, including more established ways to query companies and governments about their intent. We need mechanisms like the “trusted intermediaries” that Jonathan Askin proposes to restrict the ever-growing loophole of the third-party exception to the Fourth Amendment’s Warrant Requirement.
Most of all, we need more independent auditing of AI systems, including a structure like financial oversight, where the auditors are required to maintain independence from the companies they oversee. This feature is included in the current version of the EU’s AI Act for all high-risk systems.
Regulation alone will only go so far, especially in monitoring the regulators themselves. Most proposed AI-related regulatory measures, including the EU’s AI Act, make exceptions for governments and the military. They also tend to favor large companies at the expense of small ones. This is a serious threat to entrepreneurial innovation. Large companies can more easily bear the expenses of auditing and lobbying; there need to be ways for small, scrappy companies to compete.
Artificial intelligence systems might seem anti-democratic, but so far, paradoxically, they have generated tools that shed light on corruption and malfeasance. In the end, the rise of AI and authoritarianism have one thing in common: They’re prodding many of us to react—and thus to think differently about our responsibility to society. Hannah Arendt, in her writing on totalitarianism, distinguished between the authority of a dictator and true power with others—the impact of people acting together in a thoughtful and considered way. That’s the type of human learning that will help us keep up with machine learning in the years to come.
 | A guest post by
|