What are the dangers of Intelligence Agencies using AI tools for surveillance?

A year ago, we started asking three questions: 

• How are intelligence agencies using AI?
• What effect does this use have on our fundamental rights? 
• How does UK law regulate intelligence agencies use of AI? 

Our new report "Artificial Intelligence Agencies: The Use and Regulation of AI in UK Intelligence Agencies" collates our answers, and offers recommendations about what we can do to address the challenges that AI poses. 

The first question was difficult to answer. In the UK, you cannot seek access to information from intelligence agencies through FOIA . We know - based on statements from the directors of GCHQ and MI6 - that intelligence agencies are using AI to analyse huge amounts of data. We know that the government is investing heavily in data centres, and views AI as critical infrastructre. We have more detail about how police forces are using AI, and can infer that intelligence agencies might be doing something similar, albeit on a larger scale. We looked at how intelligence agencies in other countries are using AI, and extrapolated.

A murky picture began to form. Intelligence agencies are likely using AI to analyse huge amounts of personal data from different public and private sources. They are also likely working with third party companies to provide some of this AI software. The AI can likely analyse sentiment, map relationships, track and predict movement, and identify intelligence targets. And it can do this all faster and across much more data than humans can, albeit with serious potential to introduce errors and bias.

The second question was a bit easier to answer. The way intelligence agencies use AI is troubling for three reasons. First, the advent of AI incentivises intelligence agencies to collect even more personal data about us. This data can reveal relationships, political beliefs, health conditions, hobbies, sexual preferences, financial status, and more. By analysing data from different sources together, what may seem like an innocuous piece of information can become a piece in a puzzle that reveals a complete picture of someone’s life. Second, AI is not infallible - some AI systems have been proven to reinforce racial and gender biases; and others have been caught lying, hallucinating, or ignoring programmed instructions. In intelligence agency contexts, the stakes for these mistakes are high, as AI analysis plays a role in targeting individuals or identifying suspects. Finally, AI has likely vastly expanded the amount of data that intelligence agencies are able to analyse, opening the door to mass surveillance.

These points taken together suggest that AI use may fundamentally shift how intelligence agencies function. Instead of using data to verify or investigate known threats, data is used to predict them. This change is akin to a shift from targeted spearfishing to trawling. And it likely involves a lot more data, and a lot more people.

Investigating the third question revealed a complex and fragmented regulatory regime. Broad national security exemptions enable non-compliance with a number of existing safeguards including those under data protection law. Additionally, the nature of AI systems, and particularly, their ability to “learn” and retain information vast amount of information and the opaque nature of their reasoning, means that existing regulation is simply insufficient, and existing safeguards are undermined. This is evident in the application of the Investigatory Powers Act 2016 where we identified significant gaps. The result is a legal framework that falls far short of adequately regulating intelligence agencies in the modern world, seriously threatening our privacy and other rights which rely on that foundation.