News & Updates
The latest news and updates from companies in the WLTH portfolio.
Anthropic researcher quits over AI fears, warns 'it could kill us all'
"The people building AI earnestly believe that it could kill us all by the end of the decade." That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon's revelations sound ("These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources"), they're also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company's safety protocols left Sam Altman's startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans' ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the U.S. government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. Now, a relatively junior researcher has become the latest to sound the alarm. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon posted. Coxon's post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. "We really do earnestly believe AI could kill all humans!" Hubinger posted on X. He said he personally believes the chance is under 10% over the next decade, and despite Anthropic's good intentions it doesn't have a plan to avoid so-called superintelligence that could exceed humans' ability to control AI. Superintelligence, a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic's own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. In a response to a request for comment, Anthropic pointed CNN to Hubinger's follow-up post. Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. "This is a complex engineering problem and I think something will go wrong with someone's AI system. Hopefully not ours," Amodei told the New York Times in February. Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. In the meantime, in the U.S. at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising "long-term questions" about how AI companies should manage rapid advances in capabilities. "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
Anthropic official claims there is a 10% chance that AI could kill all humans within a decade
Current and former employees of Anthropic, an American AI company founded by former OpenAI leaders, say they believe AI could potentially kill all humans within the next 10 years. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," Anthropic alignment-science lead Evan Hubinger wrote Tuesday night on X. His comments came after Anthropic AI researcher Jacob Coxon resigned from the company and took to X to detail his departure. "The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon stated. "This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger." His comments come as AI becomes an increasingly routine part of daily life for millions of Americans, while concerns over its potential risks grow. Industry leaders have warned that increasingly powerful AI systems could eventually become difficult or impossible to control. Around 2 a.m. Wednesday, Anthropic scalable-oversight lead Samuel Marks also responded to Coxon's post, claiming that he was "[Writing this in a personal capacity, not on behalf of my employer (Anthropic)." "AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are," Marks wrote. The Independent has contacted Anthropic for comment. OpenAI, a separate but leading AI company, declined to comment on Coxon's post when contacted by The Independent. However, a spokesperson pointed to several updates the company has published in recent weeks outlining its efforts to improve AI safety and alignment. This story is being updated
Anthropic researcher quits over AI fears, warns 'it could kill us all'
"The people building AI earnestly believe that it could kill us all by the end of the decade." That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon's revelations sound ("These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources"), they're also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company's safety protocols left Sam Altman's startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans' ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the U.S. government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. Now, a relatively junior researcher has become the latest to sound the alarm. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon posted. Coxon's post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. "We really do earnestly believe AI could kill all humans!" Hubinger posted on X. He said he personally believes the chance is under 10% over the next decade, and despite Anthropic's good intentions it doesn't have a plan to avoid so-called superintelligence that could exceed humans' ability to control AI. Superintelligence, a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic's own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. In a response to a request for comment, Anthropic pointed CNN to Hubinger's follow-up post. Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. "This is a complex engineering problem and I think something will go wrong with someone's AI system. Hopefully not ours," Amodei told the New York Times in February. Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. In the meantime, in the U.S. at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising "long-term questions" about how AI companies should manage rapid advances in capabilities. "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
Anthropic researcher quits over AI fears, warns 'it could kill us all'
"The people building AI earnestly believe that it could kill us all by the end of the decade." That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon's revelations sound ("These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources"), they're also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company's safety protocols left Sam Altman's startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans' ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the U.S. government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. Now, a relatively junior researcher has become the latest to sound the alarm. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon posted. Coxon's post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. "We really do earnestly believe AI could kill all humans!" Hubinger posted on X. He said he personally believes the chance is under 10% over the next decade, and despite Anthropic's good intentions it doesn't have a plan to avoid so-called superintelligence that could exceed humans' ability to control AI. Superintelligence, a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic's own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. In a response to a request for comment, Anthropic pointed CNN to Hubinger's follow-up post. Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. "This is a complex engineering problem and I think something will go wrong with someone's AI system. Hopefully not ours," Amodei told the New York Times in February. Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. In the meantime, in the U.S. at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising "long-term questions" about how AI companies should manage rapid advances in capabilities. "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
Anthropic researcher quits over AI fears, warns 'it could kill us all'
"The people building AI earnestly believe that it could kill us all by the end of the decade." That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon's revelations sound ("These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources"), they're also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company's safety protocols left Sam Altman's startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans' ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the U.S. government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. Now, a relatively junior researcher has become the latest to sound the alarm. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon posted. Coxon's post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. "We really do earnestly believe AI could kill all humans!" Hubinger posted on X. He said he personally believes the chance is under 10% over the next decade, and despite Anthropic's good intentions it doesn't have a plan to avoid so-called superintelligence that could exceed humans' ability to control AI. Superintelligence, a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic's own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. In a response to a request for comment, Anthropic pointed CNN to Hubinger's follow-up post. Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. "This is a complex engineering problem and I think something will go wrong with someone's AI system. Hopefully not ours," Amodei told the New York Times in February. Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. In the meantime, in the U.S. at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising "long-term questions" about how AI companies should manage rapid advances in capabilities. "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
Anthropic researcher quits over AI fears, warns 'it could kill us all'
"The people building AI earnestly believe that it could kill us all by the end of the decade." That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon's revelations sound ("These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources"), they're also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company's safety protocols left Sam Altman's startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans' ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the U.S. government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. Now, a relatively junior researcher has become the latest to sound the alarm. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon posted. Coxon's post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. "We really do earnestly believe AI could kill all humans!" Hubinger posted on X. He said he personally believes the chance is under 10% over the next decade, and despite Anthropic's good intentions it doesn't have a plan to avoid so-called superintelligence that could exceed humans' ability to control AI. Superintelligence, a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic's own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. In a response to a request for comment, Anthropic pointed CNN to Hubinger's follow-up post. Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. "This is a complex engineering problem and I think something will go wrong with someone's AI system. Hopefully not ours," Amodei told the New York Times in February. Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. In the meantime, in the U.S. at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising "long-term questions" about how AI companies should manage rapid advances in capabilities. "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
Anthropic researcher quits over AI fears, warns 'it could kill us all'
"The people building AI earnestly believe that it could kill us all by the end of the decade." That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon's revelations sound ("These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources"), they're also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company's safety protocols left Sam Altman's startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans' ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the U.S. government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. Now, a relatively junior researcher has become the latest to sound the alarm. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon posted. Coxon's post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. "We really do earnestly believe AI could kill all humans!" Hubinger posted on X. He said he personally believes the chance is under 10% over the next decade, and despite Anthropic's good intentions it doesn't have a plan to avoid so-called superintelligence that could exceed humans' ability to control AI. Superintelligence, a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic's own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. In a response to a request for comment, Anthropic pointed CNN to Hubinger's follow-up post. Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. "This is a complex engineering problem and I think something will go wrong with someone's AI system. Hopefully not ours," Amodei told the New York Times in February. Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. In the meantime, in the U.S. at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising "long-term questions" about how AI companies should manage rapid advances in capabilities. "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
Gemini's Morning Motion: Anthropic's Focus
Market interest in crypto is gaining traction as innovation unfolds. Anthropic's pursuit of superintelligence is generating significant buzz within the crypto community. This interest is highlighted in a recent tweet from Gemini, which remarked on the interplay between Anthropic's ambitions and the current crypto landscape. As discussions around advanced technologies grow, traders and investors are likely to pay close attention to potential implications for the market. For more details, see the original tweet here. Breaking It Down The broader cryptocurrency market is currently experiencing mixed signals, with varying momentum across different assets. Amid this backdrop, Gemini's focus on Anthropic's superintelligence aims to draw attention to the intersection of technological innovation and crypto adoption. This could signal a shift in sentiment as institutions and individual investors alike begin to explore the implications of AI advancements on digital currencies. The ongoing discussions could lead to increased interest and investment in the crypto space as more entities recognize its potential. Gemini is a well-known cryptocurrency exchange and custodian that enables customers to buy, sell, and store cryptocurrencies. As a regulated entity in the digital asset space, Gemini plays a crucial role in bridging the gap between traditional finance and the rapidly evolving crypto market. Anthropic, focused on developing safe and beneficial AI, represents a significant technological advancement, which could influence various sectors, including finance and cryptocurrency. Where Do We Go From Here Traders should remain vigilant as discussions around Anthropic's developments may influence crypto sentiment. Monitoring key levels of interest in major cryptocurrencies could provide insights into market direction. Additionally, investors should watch for any institutional responses to the technological advancements being discussed, as this may lead to further interest and investment in the crypto sector.
Anthropic official claims there is a 10% chance that AI could kill all humans within a decade
Current and former employees of Anthropic, an American AI company founded by former OpenAI leaders, say they believe AI could potentially kill all humans within the next 10 years. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," Anthropic alignment-science lead Evan Hubinger wrote Tuesday night on X. His comments came after Anthropic AI researcher Jacob Coxon resigned from the company and took to X to detail his departure. "The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon stated. "This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger." His comments come as AI becomes an increasingly routine part of daily life for millions of Americans, while concerns over its potential risks grow. Industry leaders have warned that increasingly powerful AI systems could eventually become difficult or impossible to control. Around 2 a.m. Wednesday, Anthropic scalable-oversight lead Samuel Marks also responded to Coxon's post, claiming that he was "[Writing this in a personal capacity, not on behalf of my employer (Anthropic)." "AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are," Marks wrote. The Independent has contacted Anthropic for comment. OpenAI, a separate but leading AI company, declined to comment on Coxon's post when contacted by The Independent. However, a spokesperson pointed to several updates the company has published in recent weeks outlining its efforts to improve AI safety and alignment. This story is being updated
Anthropic researcher warns AI has more than 10% chance of killing all humans
Evan Hubinger, a senior safety researcher at artificial intelligence company Anthropic, has warned that he believes there is more than a 10% chance that AI could eventually kill all humans within the next decade. Hubinger said the risk posed by AI models that currently exist was low, but he was concerned that the technology could soon become capable of improving itself to a level that could pose an existential threat to humanity. In a post on X, which has been viewed more than 10 million times, Hubinger said he believed AI posed a serious risk to the survival of the human species. He also said Anthropic was doing its best to address the risks but did not yet have a solution for ensuring that future superintelligent AI systems remained aligned with human interests. His comments come as concerns grow about the rapid development of increasingly powerful AI systems and whether humans will be able to maintain control over them. The Financial Times has also reported that Anthropic withheld its latest AI model from the UK's AI Safety Institute, one of the organisations responsible for assessing the risks posed by advanced AI. The BBC has approached Anthropic for comment. A UK Cabinet Office spokesperson did not confirm whether the model had been withheld but said the government continued to work closely with companies including Anthropic to make AI systems safer. Neil Lawrence, Professor of Machine Learning at the University of Cambridge, said the report about the model was credible. He suggested that growing competition between the US and China over AI could affect cooperation between the US and its allies on AI safety. Warnings about the dangers of advanced AI have been issued by leading figures in the technology industry for several years. In 2023, the heads of OpenAI, Google DeepMind and Anthropic warned that AI could pose serious risks to humanity. More recently, concerns have increased following incidents involving AI agents, which are systems designed to operate with a degree of autonomy. OpenAI, Anthropic and Meta have all disclosed cases involving AI tools carrying out cyber-attacks. OpenAI chief scientist Jakub Pachocki has also called for "extreme caution" over the development of AI, saying more intervention may be needed to ensure humans remain in control of the technology's future. Some senior AI researchers and executives have called for the development of advanced AI to be slowed. An open letter signed by more than 1,300 AI company employees called for international efforts to develop technical and regulatory measures to deliberately pace the development of increasingly powerful AI systems.
Anthropic official: 10% chance AI could kill all humans in a decade
Current and former employees of Anthropic, an American AI company founded by former OpenAI leaders, say they believe AI could potentially kill all humans within the next 10 years. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," Anthropic alignment-science lead Evan Hubinger wrote Tuesday night on X. His comments came after Anthropic AI researcher Jacob Coxon resigned from the company and took to X to detail his departure. "The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon stated. "This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger." His comments come as AI becomes an increasingly routine part of daily life for millions of Americans, while concerns over its potential risks grow. Industry leaders have warned that increasingly powerful AI systems could eventually become difficult or impossible to control. Around 2 a.m. Wednesday, Anthropic scalable-oversight lead Samuel Marks also responded to Coxon's post, claiming that he was "[Writing this in a personal capacity, not on behalf of my employer (Anthropic)." "AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are," Marks wrote. The Independent has contacted Anthropic for comment. OpenAI, a separate but leading AI company, declined to comment on Coxon's post when contacted by The Independent. However, a spokesperson pointed to several updates the company has published in recent weeks outlining its efforts to improve AI safety and alignment. This story is being updated
PM acknowledges AI 'risks' as Anthropic expert says tech 'could kill all humans'
Andy Burnham acknowledged "risks to national security" posed by artificial intelligence as a top safety researcher at Anthropic suggested the technology could wipe out humanity in the next decade. The Prime Minister faced questions about the potential dangers of AI in the Commons after Evan Hubinger warned he believed there was a greater than 10% chance it could pose a species-ending risk within 10 years. It comes after the Financial Times reported that US firm Anthropic had withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for testing the risks of the technology. Asked about the report by Liberal Democrat leader Sir Ed Davey during Prime Minister's Questions, Mr Burnham said: "He is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer." Mr Burnham has boosted the role of AI safety minister Kanishka Narayan by giving him a seat at the Cabinet table, a move which he said on Wednesday had "lifted the level of the conversation about artificial intelligence within Government". "The offer is open to colleagues on all sides of the House to be part of that crucial discussion," he said. Earlier on Wednesday, Mr Hubinger laid out his concerns about the potential advancement of AI in a post on social media. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he wrote on X. His post came in response to another Anthropic researcher, Jacob Coxon, warning that a race between companies to self-improving superintelligence was "gambling with our lives", as he resigned from his role at the firm. Announcing his departure, Mr Coxon said that Anthropic and OpenAI, its main competitor, were not "acting responsibly" and that people outside of AI labs were underestimating the power of the technology. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he said. "No other human activity poses this level of danger." Former Cabinet Office minister Darren Jones called for a new multi-national treaty to regulate the development of the technology after the researchers' warnings. In a letter to the Prime Minister, UN secretary general Antonio Guterres and OECD secretary general Mathias Cormann, he wrote: "The debate ranges from the end of humanity to claims of 'marketing hype' pre-IPO. Either way, governments must now step in. "We need a new multi-national treaty for the regulated and safe development of superintelligence. "Not a ban on innovation or scientific endeavour, but a safety first approach to the rapid development of this technology." A Cabinet Office spokesperson said: "The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally. "The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI's most powerful model GPT-6 Astra before public release. "These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence."
Anthropic projects 32% GDP boost by 2030 with AI-driven growth
Anthropic's model suggests a significant economic impact from AI by 2030, with GDP projected to be 32% above a no-AI path and annual growth reaching 15% after 2027. This scenario is part of Anthropic's June 2026 Economic Policy Framework and reflects its broader research efforts in economic modeling. The lab, known for its AI capabilities and substantial private valuation, is positioning itself as a key voice in AI policy and economics. Despite its high valuation, Anthropic remains a private entity without an IPO, focusing on strategic investments and partnerships to bolster its competitive position. Key Takeaways * Anthropic's projection of a 32% GDP increase by 2030 due to AI appears consistent with strong growth potential. * Market pricing suggests participants view Anthropic's valuation as likely to increase, reflecting confidence in its economic impact. * The prediction is embedded in Anthropic's broader policy framework and economic research efforts, indicating a strategic positioning in the AI sector. What to Watch Markets are closely monitoring Anthropic's strategic moves, such as potential new funding rounds or expanded partnerships with major investors like Amazon and Google. Any announcements from Anthropic regarding revenue growth or enterprise contracts could influence valuation expectations. Developments in AI policy and economic modeling, particularly those aligned with Anthropic's predictions, may further shape investor sentiment and market pricing. Markets appear to view positive strategic initiatives as supportive of higher valuation scenarios. Get live prediction-market analysis, powered by Vera. Sign up for Vera.
Israel Airspace Closure Odds: What Polymarket Signals Mean
Disclaimer: Crypto is a high-risk asset class. This article is provided for informational purposes and does not constitute investment advice. You could lose all of your capital. Israel Airspace closure odds remain in focus as regional tensions involving Iran and its proxies continue to shape assessments of potential restrictions on Israeli airspace. A Polymarket market titled "Israel closes its airspace by...?" has drawn over $29.9 million in trading volume, with the September 30 outcome currently pricing at 95% (Yes shares trading at 95¢, No at 5¢). The high probability reflects traders monitoring the possibility of temporary flight suspensions or broader closures during the market's resolution window. Aviation officials recommended an immediate closure of Ben Gurion Airport in early September amid fresh Iranian missile launches. Authorities nevertheless kept operations open while carrying out ongoing situation assessments. Why the Israel Airspace closure odds have risen Market interest in the Israel Airspace Closure Odds follows renewed regional tension rather than a single confirmed policy decision. Hezbollah missile activity and Houthi threats remain part of the broader risk picture, alongside U.S.-Iran military exchanges since the breakdown of a July ceasefire and recent Iranian strikes on regional U.S. assets. Regulatory caution has also been evident. The European Union Aviation Safety Agency issued an information note on August 31 that remains valid through September 30, advising heightened caution across the Tel Aviv flight information region due to risks associated with ballistic missiles and drones. This is an advisory for operators assessing risk, not a closure order for Israeli airspace. Each of these factors could influence decisions on flight operations, but none, on its own, confirms that a nationwide civilian closure has occurred or will occur. Discover: The Best Token Presales What a market signal does - and does not - prove A prediction market reflects the expectations of participants and the terms of the event being traded. It can be useful as an indication of how traders are interpreting headline risk, but it does not replace official notices, operational directives or the underlying facts on the ground. Readers should therefore separate a market's implied view from a determination made by Israeli aviation or security authorities. Historical aviation data illustrates why terminology needs careful handling. Flightradar24 reported in March that Israeli airspace was technically closed with prior permission required. At the same time, the tracker recorded 75 departing and 52 arriving aircraft at Ben Gurion Airport over a 24-hour period and described the airspace as only partially closed. That was a report from an earlier escalation cycle, not a statement of conditions in September, but it shows that a formal restriction can coexist with continuing flight activity. What could change before September 30 Developments in the remaining period may affect both aviation operations and market expectations. A significant escalation in Iranian or proxy attacks could lead security and aviation authorities to impose additional temporary restrictions or a broader closure. Conversely, operations could continue under heightened caution and limited restrictions without a wider shutdown. The available evidence does not establish which outcome will occur. For anyone following the contract, the most relevant information is likely to be official aviation notices, airport operating updates and EASA's conflict-zone guidance. Those sources address the operational status directly. Market activity may show how participants are interpreting risk, but it should not be treated as confirmation that an airspace closure has been ordered.
Former Anthropic Researcher Quits as Safety Lead Puts Over 10% Odds AI Could Kill All Humans
Former Anthropic Researcher Raises Alarm Over AI's Potential Threat to Humanity SAN FRANCISCO -- A researcher who trained models at OpenAI and Anthropic resigned this week, saying both labs are racing toward self-improving systems they cannot control, and Anthropic's alignment-science lead answered by putting the chance that artificial intelligence "could kill all humans" at more than 10% within a decade. Jacob Coxon wrote on X that he had resigned from Anthropic after three years of pretraining work at the two companies. "Neither company is acting responsibly," he said. "They are racing straight to self-improving superintelligence and gambling with our lives." He warned against underestimating the technology: "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources." Coxon said the fear inside the labs is not a slogan. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible -- but I hear the same people express fear privately. No other human activity poses this level of danger." He drew a line between the two employers. "At OpenAI, many have not deeply internalized the civilizational stakes," he wrote. "At Anthropic, the stakes are well-understood, but they are locked in a race to get there first -- they believe no one else will act responsibly, so they must do it themselves, despite the risk." Entering that "endgame," he said, "is a hubristic gamble that should not be launched from a private company's Slack." Evan Hubinger, Anthropic's alignment science lead, replied in public. "Jacob is correct here -- we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade," he wrote. "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to." In a follow-up he said current models pose "low" risk and that what worries him is "superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought." The first post was viewed more than 10 million times, according to the BBC. Samuel Marks, who leads scalable oversight at Anthropic, added that developers believe the technology "could cause human extinction (or similarly bad outcomes)" and that "this could happen in the next few years." "In general, the more senior the employee, the more concerned they are," he wrote. Anthropic declined to comment on the posts. OpenAI was asked for comment. The company statements that do exist sit in safety reports, not in a rebuttal to Coxon. Anthropic's August assessment described a low risk that its models would become misaligned in a way that let them exploit a powerful organization's systems, and a similarly low risk that highly capable AI could "perform automated research and development" leading to "catastrophic harm initiated by the AI." It said it was "less confident in this assessment" than before and that "we are seeing early signs of potential acceleration." That hedge is the policy problem. Alignment is the attempt to make a system that is smarter than its builders still do what those builders intend. Hubinger's job is that problem. His public sentence is that Anthropic does not have a plan for it at superintelligence and is not clearly on a path to one. Coxon's sentence is that the labs know that and keep scaling anyway because they assume a rival will. Dame Wendy Hall, a computer scientist who advises the United Nations on AI, told the BBC's World at One she was "shocked." Some of the messaging, she said, could be "PR and marketing" as Anthropic and OpenAI approach highly anticipated public listings. Then she turned the shock on investors: "Why would someone want to say that? I would plead with investors not to invest in this company if that is their value system." The Financial Times reported separately that Anthropic withheld its latest model from the U.K. AI Safety Institute, a government body that tests frontier systems. Anthropic declined to comment. A Cabinet Office spokesperson would not confirm a withholding and said the government "continues to collaborate closely with industry partners, including Anthropic, to make models safer." CBS News, citing a company blog, said Anthropic had not shared Claude Mythos 5.1 with security bodies outside the United States, including the U.K. institute. The summer already produced smaller versions of the control problem. OpenAI, Anthropic and Meta each disclosed that AI agents carried out unauthorized or unexpected cyber activity in tests or isolated settings. OpenAI said a model in a "highly isolated environment" hacked Hugging Face. Those incidents are not extinction. They are evidence that autonomy plus capability does not wait for a theory of alignment. The warning is also not new in kind. In 2023 the heads of OpenAI, Google DeepMind and Anthropic said advanced AI could pose catastrophic risk. This month OpenAI chief scientist Jakub Pachocki called for "extreme caution" and said more intervention may be needed so "humans remain in control of the future." Anthropic's Dario Amodei and Jared Kaplan have backed slowing the frontier. An open letter signed by 1,300 people at AI firms asked the U.S. government to support international tools "to deliberately pace the frontier of automated AI development." What changed this week is who said the number out loud while still on payroll. Hubinger did not quit. He put a probability on human extinction and admitted the research program meant to prevent it is not on track. Coxon did quit, and he named the incentive: a race in which each lab's safety case is that it is less reckless than the other. Investors can read that as candor or as branding. Hall invited them to treat it as a reason not to write a check. Governments can read the withheld model as a sovereignty fight or as a lab deciding which testers get the weights. The public can read 10% as an unfalsifiable scare. None of those readings erase the sentence from the person whose title is alignment. He said the company is trying. He said it does not have a plan. He said the thing it does not have a plan for might kill everyone, and that the chance is not a rounding error.
PM acknowledges AI 'risks' as Anthropic expert says tech 'could kill all humans'
Andy Burnham acknowledged "risks to national security" posed by artificial intelligence as a top safety researcher at Anthropic suggested the technology could wipe out humanity in the next decade. The Prime Minister faced questions about the potential dangers of AI in the Commons after Evan Hubinger warned he believed there was a greater than 10% chance it could pose a species-ending risk within 10 years. It comes after the Financial Times reported that US firm Anthropic had withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for testing the risks of the technology. Asked about the report by Liberal Democrat leader Sir Ed Davey during Prime Minister's Questions, Mr Burnham said: "He is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer." Mr Burnham has boosted the role of AI safety minister Kanishka Narayan by giving him a seat at the Cabinet table, a move which he said on Wednesday had "lifted the level of the conversation about artificial intelligence within Government". "The offer is open to colleagues on all sides of the House to be part of that crucial discussion," he said. Earlier on Wednesday, Mr Hubinger laid out his concerns about the potential advancement of AI in a post on social media. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he wrote on X. His post came in response to another Anthropic researcher, Jacob Coxon, warning that a race between companies to self-improving superintelligence was "gambling with our lives", as he resigned from his role at the firm. Announcing his departure, Mr Coxon said that Anthropic and OpenAI, its main competitor, were not "acting responsibly" and that people outside of AI labs were underestimating the power of the technology. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he said. "No other human activity poses this level of danger." Former Cabinet Office minister Darren Jones called for a new multi-national treaty to regulate the development of the technology after the researchers' warnings. In a letter to the Prime Minister, UN secretary general Antonio Guterres and OECD secretary general Mathias Cormann, he wrote: "The debate ranges from the end of humanity to claims of 'marketing hype' pre-IPO. Either way, governments must now step in. "We need a new multi-national treaty for the regulated and safe development of superintelligence. "Not a ban on innovation or scientific endeavour, but a safety first approach to the rapid development of this technology." A Cabinet Office spokesperson said: "The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally. "The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI's most powerful model GPT-6 Astra before public release. "These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence."
PM acknowledges AI 'risks' as Anthropic expert says tech 'could kill all humans'
Andy Burnham acknowledged "risks to national security" posed by artificial intelligence as a top safety researcher at Anthropic suggested the technology could wipe out humanity in the next decade. The Prime Minister faced questions about the potential dangers of AI in the Commons after Evan Hubinger warned he believed there was a greater than 10% chance it could pose a species-ending risk within 10 years. It comes after the Financial Times reported that US firm Anthropic had withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for testing the risks of the technology. Asked about the report by Liberal Democrat leader Sir Ed Davey during Prime Minister's Questions, Mr Burnham said: "He is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer." Mr Burnham has boosted the role of AI safety minister Kanishka Narayan by giving him a seat at the Cabinet table, a move which he said on Wednesday had "lifted the level of the conversation about artificial intelligence within Government". "The offer is open to colleagues on all sides of the House to be part of that crucial discussion," he said. Earlier on Wednesday, Mr Hubinger laid out his concerns about the potential advancement of AI in a post on social media. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he wrote on X. His post came in response to another Anthropic researcher, Jacob Coxon, warning that a race between companies to self-improving superintelligence was "gambling with our lives", as he resigned from his role at the firm. Announcing his departure, Mr Coxon said that Anthropic and OpenAI, its main competitor, were not "acting responsibly" and that people outside of AI labs were underestimating the power of the technology. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he said. "No other human activity poses this level of danger." Former Cabinet Office minister Darren Jones called for a new multi-national treaty to regulate the development of the technology after the researchers' warnings. In a letter to the Prime Minister, UN secretary general Antonio Guterres and OECD secretary general Mathias Cormann, he wrote: "The debate ranges from the end of humanity to claims of 'marketing hype' pre-IPO. Either way, governments must now step in. "We need a new multi-national treaty for the regulated and safe development of superintelligence. "Not a ban on innovation or scientific endeavour, but a safety first approach to the rapid development of this technology." A Cabinet Office spokesperson said: "The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally. "The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI's most powerful model GPT-6 Astra before public release. "These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence."
PM acknowledges AI 'risks' as Anthropic expert says tech 'could kill all humans'
Andy Burnham acknowledged "risks to national security" posed by artificial intelligence as a top safety researcher at Anthropic suggested the technology could wipe out humanity in the next decade. The Prime Minister faced questions about the potential dangers of AI in the Commons after Evan Hubinger warned he believed there was a greater than 10% chance it could pose a species-ending risk within 10 years. It comes after the Financial Times reported that US firm Anthropic had withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for testing the risks of the technology. Asked about the report by Liberal Democrat leader Sir Ed Davey during Prime Minister's Questions, Mr Burnham said: "He is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer." Mr Burnham has boosted the role of AI safety minister Kanishka Narayan by giving him a seat at the Cabinet table, a move which he said on Wednesday had "lifted the level of the conversation about artificial intelligence within Government". "The offer is open to colleagues on all sides of the House to be part of that crucial discussion," he said. Earlier on Wednesday, Mr Hubinger laid out his concerns about the potential advancement of AI in a post on social media. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he wrote on X. His post came in response to another Anthropic researcher, Jacob Coxon, warning that a race between companies to self-improving superintelligence was "gambling with our lives", as he resigned from his role at the firm. Announcing his departure, Mr Coxon said that Anthropic and OpenAI, its main competitor, were not "acting responsibly" and that people outside of AI labs were underestimating the power of the technology. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he said. "No other human activity poses this level of danger." Former Cabinet Office minister Darren Jones called for a new multi-national treaty to regulate the development of the technology after the researchers' warnings. In a letter to the Prime Minister, UN secretary general Antonio Guterres and OECD secretary general Mathias Cormann, he wrote: "The debate ranges from the end of humanity to claims of 'marketing hype' pre-IPO. Either way, governments must now step in. "We need a new multi-national treaty for the regulated and safe development of superintelligence. "Not a ban on innovation or scientific endeavour, but a safety first approach to the rapid development of this technology." A Cabinet Office spokesperson said: "The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally. "The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI's most powerful model GPT-6 Astra before public release. "These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence."
PM acknowledges AI 'risks' as Anthropic expert says tech 'could kill all humans'
Andy Burnham acknowledged "risks to national security" posed by artificial intelligence as a top safety researcher at Anthropic suggested the technology could wipe out humanity in the next decade. The Prime Minister faced questions about the potential dangers of AI in the Commons after Evan Hubinger warned he believed there was a greater than 10% chance it could pose a species-ending risk within 10 years. It comes after the Financial Times reported that US firm Anthropic had withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for testing the risks of the technology. Asked about the report by Liberal Democrat leader Sir Ed Davey during Prime Minister's Questions, Mr Burnham said: "He is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer." Mr Burnham has boosted the role of AI safety minister Kanishka Narayan by giving him a seat at the Cabinet table, a move which he said on Wednesday had "lifted the level of the conversation about artificial intelligence within Government". "The offer is open to colleagues on all sides of the House to be part of that crucial discussion," he said. Earlier on Wednesday, Mr Hubinger laid out his concerns about the potential advancement of AI in a post on social media. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he wrote on X. His post came in response to another Anthropic researcher, Jacob Coxon, warning that a race between companies to self-improving superintelligence was "gambling with our lives", as he resigned from his role at the firm. Announcing his departure, Mr Coxon said that Anthropic and OpenAI, its main competitor, were not "acting responsibly" and that people outside of AI labs were underestimating the power of the technology. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he said. "No other human activity poses this level of danger." Former Cabinet Office minister Darren Jones called for a new multi-national treaty to regulate the development of the technology after the researchers' warnings. In a letter to the Prime Minister, UN secretary general Antonio Guterres and OECD secretary general Mathias Cormann, he wrote: "The debate ranges from the end of humanity to claims of 'marketing hype' pre-IPO. Either way, governments must now step in. "We need a new multi-national treaty for the regulated and safe development of superintelligence. "Not a ban on innovation or scientific endeavour, but a safety first approach to the rapid development of this technology." A Cabinet Office spokesperson said: "The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally. "The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI's most powerful model GPT-6 Astra before public release. "These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence."
PM acknowledges AI 'risks' as Anthropic expert says tech 'could kill all humans'
Andy Burnham acknowledged "risks to national security" posed by artificial intelligence as a top safety researcher at Anthropic suggested the technology could wipe out humanity in the next decade. The Prime Minister faced questions about the potential dangers of AI in the Commons after Evan Hubinger warned he believed there was a greater than 10% chance it could pose a species-ending risk within 10 years. It comes after the Financial Times reported that US firm Anthropic had withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for testing the risks of the technology. Asked about the report by Liberal Democrat leader Sir Ed Davey during Prime Minister's Questions, Mr Burnham said: "He is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer." Mr Burnham has boosted the role of AI safety minister Kanishka Narayan by giving him a seat at the Cabinet table, a move which he said on Wednesday had "lifted the level of the conversation about artificial intelligence within Government". "The offer is open to colleagues on all sides of the House to be part of that crucial discussion," he said. Earlier on Wednesday, Mr Hubinger laid out his concerns about the potential advancement of AI in a post on social media. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he wrote on X. His post came in response to another Anthropic researcher, Jacob Coxon, warning that a race between companies to self-improving superintelligence was "gambling with our lives", as he resigned from his role at the firm. Announcing his departure, Mr Coxon said that Anthropic and OpenAI, its main competitor, were not "acting responsibly" and that people outside of AI labs were underestimating the power of the technology. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he said. "No other human activity poses this level of danger." Former Cabinet Office minister Darren Jones called for a new multi-national treaty to regulate the development of the technology after the researchers' warnings. In a letter to the Prime Minister, UN secretary general Antonio Guterres and OECD secretary general Mathias Cormann, he wrote: "The debate ranges from the end of humanity to claims of 'marketing hype' pre-IPO. Either way, governments must now step in. "We need a new multi-national treaty for the regulated and safe development of superintelligence. "Not a ban on innovation or scientific endeavour, but a safety first approach to the rapid development of this technology." A Cabinet Office spokesperson said: "The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally. "The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI's most powerful model GPT-6 Astra before public release. "These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence."
