Since 2016, Avon and Somerset Police teamed with Bristol City Council to develop a comprehensive predictive analytics program using sensitive data to assess risks related to crime and social harm. However, internal reviews and new disclosures reveal some models performed poorly and transparency has been lacking, raising critical questions as the UK scales AI use in policing.
- Avon and Somerset Police developed at least 23 predictive crime and risk models starting in 2016.
- Two risk-scoring models were abandoned after Bristol officials lost confidence in their reliability.
- Lack of public awareness and transparency has sparked accountability demands and legal actions.
What happened
Avon and Somerset Police, in collaboration with Bristol City Council, created a vast data repository combining sensitive personal information such as police intelligence, mental health, housing, and educational data. Using this data, multiple machine-learning models were built to score thousands of individuals on their potential risks related to various criminal and social outcomes, such as burglary, missing persons, and domestic abuse.
Despite the ambitious scope intended to help law enforcement anticipate and prevent crimes, internal evaluations, supported by independent analysis, revealed actual performance issues. Two risk assessment models were quietly discontinued after concerns about their accuracy and trustworthiness arose. The program’s opaque operation led to public criticism, with affected individuals unaware of their inclusion or how scores might impact them.
Why it matters
The use of AI-driven predictive analytics in policing raises profound ethical, legal, and societal questions. Without clear transparency, oversight, and accountability, these tools risk undermining public confidence and potentially resulting in unfair treatment or discrimination against individuals flagged by imperfect models.
The findings are particularly significant as the UK accelerates the integration of AI technologies into national policing strategies. Leaders like Andy Marsh, former Avon and Somerset chief constable and current CEO of the College of Policing, advocate widespread adoption of AI tools. However, these pilot projects demonstrate that robust evaluation and transparency must accompany such expansion to avoid eroding trust and effectiveness.
What to watch next
Stakeholders will be closely monitoring how UK police forces implement recommendations for transparency and independent review of predictive policing tools. Legal challenges, such as those initiated by Bristol residents seeking clarity on their data use, may set precedents influencing national policies on AI accountability and data rights.
Additionally, the College of Policing’s evaluations of about 100 AI tools currently in use will be critical in determining standards for deployment. Regulators and civil rights groups will likely push for frameworks ensuring that AI applications in criminal justice meet high reliability, fairness, and transparency benchmarks before being broadly adopted.