In a notable change of stance, the United Kingdom is actively taking measures to confront the potential threats tied to Artificial Intelligence (AI). Prime Minister Rishi Sunak has unveiled plans for an international AI Safety Summit, underlining the urgency of addressing these threats. This shift follows an earlier AI white paper, which, in the spring, characterized the existential risks linked to AI as having a “high impact, low probability.”
The change in perspective is partly attributed to the influence of the Effective Altruism (EA) movement, a group of thought leaders advocating for AI safety, often supported by prominent figures in the tech industry.
Effective Altruism: Advocating for a future-centric approach
The Effective Altruism movement has been instrumental in shaping the UK’s stance on AI safety. Rooted in Oxford University and fueled by backing from Silicon Valley heavyweights. EA adherents believe that super-intelligent AI, if not appropriately managed, could pose an existential threat to humanity.
Their approach emphasizes long-term considerations over immediate concerns, asserting that the pursuit of super-intelligent AI should proceed with unwavering caution. For them, the potential outcomes of AI development are binary: either utopia or annihilation. This perspective has gained traction in the UK, with government advisers aligned with EA concerns and Sunak’s close ties to AI labs connected to the movement.
Ian Hogarth’s warning spurs action
The pivotal moment in this shift came when tech investor Ian Hogarth penned a viral Financial Times article in April, sounding the alarm about the race towards “God-like AI” and its potential to bring about the obsolescence or destruction of humanity. His article echoed the sentiments of the influential “AI pause” letter, which called for a moratorium on large-scale AI experiments.
In conjunction with another letter suggesting that AI posed an extinction risk, this triggered widespread discussions and prompted Prime Minister Sunak to express his concern about these risks.
Effective Altruism’s influence on the UK’s AI safety taskforce
Under Ian Hogarth’s leadership, the UK’s Foundation Model Taskforce announced new partnerships, a significant number of which are associated with the Effective Altruism movement. Notably, the Centre for AI Safety, known for its “AI extinction risk” letter, receives primary funding from Open Philanthropy, a major EA donor organization. Arc Evals, which has potentially catastrophic risks of AI systems, is another partner EA donors support.
The Community Intelligence Project, focusing on governance models for transformative technology, is also connected to the EA community. The taskforce’s research team includes Cambridge professor David Krueger, who has received a substantial grant from Open Philanthropy to reduce the risk of human extinction due to AI systems.
AI safety and frontier models
The taskforce, now rebranded as the “Frontier AI Taskforce,” is broadening its focus to address emerging AI risks beyond the immediate horizon. It aims to manage risks and harness the technology’s benefits for society. However, this change in perspective has garnered criticism from some quarters.
Researchers and AI ethics experts argue that the focus on existential risks has overshadowed pressing concerns such as bias, data privacy, and copyright issues in today’s AI models. They express concern that AI safety discussions have become alarmist and lack empirical evidence.
The intersection of Effective Altruism, tech giants, and policy
The EA movement’s strong ties to Silicon Valley raise questions about its objectivity. Major AI labs like OpenAI, DeepMind, and Anthropic have connections to EA, with its ideology influencing their ethos and decisions.
Open Philanthropy, founded by Facebook co-founder Dustin Moskovitz, provided OpenAI with substantial initial funding. Skype founder Jaan Tallinn was an early investor and former director at DeepMind. Elon Musk, a proponent of “longterm” ideology related to EA, has hired Dan Hendrycks, director of the Center for AI Safety, as an adviser to his new startup, xAI.
EA’s financial support for AI safety
To counter the perceived threats, the EA movement is channeling significant resources into AI safety. Harold Karnofsky, Head of Open Philanthropy, has temporarily stepped away to focus on AI safety. The EA career advice center, 80,000 hours, recommends AI safety technical research and shaping the future governance of AI as the top career choices for EAs. This support is coupled with a unique vocabulary, including terms like “existential risks” and “probability of doom,” which sets EA apart in AI safety discussions.
While the UK’s AI safety agenda is gaining momentum, some experts caution against neglecting immediate AI-related challenges such as bias, data privacy, and ethical considerations. They argue that the focus on existential risks has overshadowed these pressing issues. Critics also express concerns that the movement’s closeness to AI companies may lead to regulatory capture, where the industry exerts undue influence on policymaking.
A complex ecosystem: EA’s role in shaping AI policy
The Effective Altruism movement’s influence in shaping the UK’s AI safety agenda is undeniable. Rooted in academia and supported by Silicon Valley, it has successfully propelled existential risks into the limelight. However, as the UK navigates this complex landscape, policymakers must balance long-term AI safety concerns with the immediate ethical, legal, and societal challenges that today’s AI technologies pose.
The United Kingdom’s proactive approach to AI safety reflects the evolving landscape of AI development. While Effective Altruism and its long-term perspective have played a pivotal role in this shift, the UK faces the challenge of striking a balance between addressing existential risks and managing AI’s ethical and societal implications in the here and now. As the international AI Safety Summit approaches, it remains to be seen how the UK will navigate these intricate waters and contribute to shaping the future of AI on a global scale.
