OpenAI, the organization behind the popular ChatGPT, is mobilizing a team to prevent potential catastrophic threats posed by what they term as “frontier AI models”. These models could surpass the capabilities of current leading AI models, presenting both unparalleled opportunities and severe risks.
*”Frontier AI models… have the potential to benefit all of humanity. But they also pose increasingly severe risks,”* the company elaborated in a recent blog post.
Leading the “preparedness” team
Spearheading the team, which has been named “Preparedness”, is Aleksander Madry. The team’s primary mission is to mitigate risks that emerge from these rapidly evolving AI models.
The threats that the company is preparing for aren’t trivial. These so-called “catastrophic” risks span across a wide spectrum, including individualized persuasion, threats in cybersecurity, and even chemical, biological, radiological, and nuclear threats. There’s also the danger of AI systems autonomously replicating and adapting.
Future-proofing against advanced AI threats
OpenAI foresees its new team playing a pivotal role in connecting capability assessment, evaluations, and internal red teaming for frontier models. This includes not only the models they plan to roll out in the immediate future but also those that reach the level of Artificial General Intelligence (AGI).
Furthermore, the Preparedness team will shoulder the responsibility of formulating a risk-informed development policy (RDP). The company elaborated, *”Our RDP will detail our approach to… establishing a governance structure for accountability and oversight.”*
This newly established RDP will not only provide guidelines for developing frontier model capabilities but also extend and complement OpenAI’s ongoing risk mitigation endeavors.
Seeking exceptional talent
The company isn’t only focusing on policy and procedure. OpenAI is actively scouting for top-tier talent from a diverse pool of technical backgrounds to strengthen the Preparedness team.
Launch of the AI preparedness challenge
In a strategic move to foster innovation and tap into external expertise, OpenAI has also rolled out the AI Preparedness Challenge. The main goal of this competition is to identify methods to prevent catastrophic misuse of AI technologies.
As an incentive, the organization is offering $25,000 in API credits for standout submissions. They’re also keen on scouting potential team members among the frontrunners of this challenge.
OpenAI’s commitment to safe AI
Earlier in July, OpenAI took the initiative to open a new alignment research division. The division’s focus is on conceptualizing training methods that prevent superintelligent AI systems – those that could potentially surpass human intelligence and act against human values – from inflicting significant damage.
Simultaneously, OpenAI, along with other eminent AI labs, voluntarily pledged to bolster safety, security, and trust in the realm of AI.
These moves come after a significant wake-up call in May, where numerous tech industry magnates, scholars, and other notable figures signed an open letter. The message was clear: the unchecked evolution of AI might trigger an extinction-level event, and reigning in this technology should be an utmost global priority.
