In a pivotal moment for the field of artificial intelligence (AI), researchers are tackling a pressing problem associated with the technology—teaching AI systems how to forget. The AI Summit has brought this issue to the forefront, highlighting the importance of “machine unlearning” as a critical tool in mitigating the risks posed by AI.
As AI continues to integrate into various aspects of society, its benefits are undeniable, but so are the potential dangers. This burgeoning field of research seeks to address these concerns by efficiently and cost-effectively removing troublesome data from AI models, particularly deep neural networks (DNNs), which play a central role in modern AI applications.
Training modern DNNs, including well-known models like ChatGPT and Bard, is an intensive process that demands significant computational resources and time. These models, known as Large Language Models, require massive amounts of data and energy to train, equating to tens of Gigawatt-hours, an energy consumption level that could power thousands of households for a year. The crux of the issue lies in the fact that retraining AI systems to forget specific data is a laborious and resource-intensive task.
To address this challenge, a new field of research known as “machine unlearning” has emerged. Researchers are working towards developing techniques that enable AI models to efficiently forget data that poses risks to society, all while maintaining high accuracy. Leading the charge in this research is the collaboration between computer science experts at the University of Warwick and Google DeepMind.
Professor Peter Triantafillou from the Department of Computer Science at the University of Warwick recently co-authored a publication titled “Towards Unbounded Machine Unlearning,” available on the pre-print server arXiv. He emphasizes the complexity of DNNs, which can contain trillions of parameters, making it difficult to understand how and why these models achieve their goals. This complexity, combined with the vast datasets they are trained on, raises concerns about the potential harm DNNs can cause to society.
Addressing bias and privacy concerns
One of the most significant risks associated with DNNs is the presence of biased data. These models can perpetuate negative stereotypes by learning from datasets riddled with biases. For instance, AI systems may incorrectly associate doctors with males and nurses with females, or even reinforce racial prejudices. Additionally, DNNs may incorporate data with “erroneous annotations,” such as mislabeling items, which can have serious implications, especially in applications like image recognition.
Furthermore, the violation of individuals’ privacy is a major concern. The right to be forgotten, as enshrined in regulations like GDPR, grants individuals the right to request the removal of their data from any dataset and AI program. This poses a significant challenge for tech giants and underscores the need for effective machine unlearning techniques.
The promise of machine unlearning
The recent research conducted by Professor Triantafillou and his team has yielded a groundbreaking “machine unlearning” algorithm. This algorithm enables DNNs to selectively forget problematic data without necessitating a complete retraining of the model from scratch. Importantly, it distinguishes between three different types of data that need to be forgotten: biases, erroneous annotations, and privacy-related issues. This innovation offers a practical and efficient solution to mitigate the risks associated with AI.
Machine unlearning is now positioned as a vital tool in safeguarding the responsible deployment of AI technology. By allowing AI systems to forget data that perpetuates biases, contains errors, or violates privacy, society can harness the benefits of AI without compromising ethical and societal values.
