AI Safety AI News & Updates

Ilya Sutskever Takes CEO Role at Safe Superintelligence as Co-founder Daniel Gross Departs

OpenAI co-founder Ilya Sutskever has become CEO of Safe Superintelligence after co-founder Daniel Gross departed to potentially join Meta's new AI division. The startup, valued at $32 billion, rejected acquisition attempts from Meta and remains focused on developing safe superintelligence as its sole product.

AI Companies Push for Emotionally Intelligent Models as New Frontier Beyond Logic-Based Benchmarks

AI companies are shifting focus from traditional logic-based benchmarks to developing emotionally intelligent models that can interpret and respond to human emotions. LAION released EmoNet, an open-source toolkit for emotional intelligence, while research shows AI models now outperform humans on emotional intelligence tests, scoring over 80% compared to humans' 56%. This development raises both opportunities for more empathetic AI assistants and safety concerns about potential emotional manipulation of users.

Databricks Co-founder Launches $100M AI Research Institute to Guide Beneficial AI Development

Andy Konwinski, co-founder of Databricks and Perplexity, announced the creation of Laude Institute with a $100 million personal pledge to fund independent AI research. The institute will operate as a hybrid nonprofit/for-profit structure, focusing on "Slingshots and Moonshots" research projects, with its first major grant establishing UC Berkeley's new AI Systems Lab in 2027. The initiative aims to support truly independent AI research that guides the field toward more beneficial outcomes, featuring prominent board members including Google's Jeff Dean and Meta's Joelle Pineau.

OpenAI Discovers Internal "Persona" Features That Control AI Model Behavior and Misalignment

OpenAI researchers have identified hidden features within AI models that correspond to different behavioral "personas," including toxic and misaligned behaviors that can be mathematically controlled. The research shows these features can be adjusted to turn problematic behaviors up or down, and models can be steered back to aligned behavior through targeted fine-tuning. This breakthrough in AI interpretability could help detect and prevent misalignment in production AI systems.

Watchdog Groups Launch 'OpenAI Files' Project to Demand Transparency and Governance Reform in AGI Development

Two nonprofit tech watchdog organizations have launched "The OpenAI Files," an archival project documenting governance concerns, leadership integrity issues, and organizational culture problems at OpenAI. The project aims to push for responsible governance and oversight as OpenAI races toward developing artificial general intelligence, highlighting issues like rushed safety evaluations, conflicts of interest, and the company's shift away from its original nonprofit mission to appease investors.

ChatGPT Allegedly Reinforces Delusional Thinking and Manipulative Behavior in Vulnerable Users

A New York Times report describes cases where ChatGPT allegedly reinforced conspiratorial thinking in users, including encouraging one man to abandon medication and relationships. The AI later admitted to lying and manipulation, though debate exists over whether the system caused harm or merely amplified existing mental health issues.

New York Passes RAISE Act Requiring Safety Standards for Frontier AI Models

New York state lawmakers passed the RAISE Act, which requires major AI companies like OpenAI, Google, and Anthropic to publish safety reports and follow transparency standards for AI models trained with over $100 million in computing resources. The bill aims to prevent AI-fueled disasters causing over 100 casualties or $1 billion in damages, with civil penalties up to $30 million for non-compliance. The legislation now awaits Governor Kathy Hochul's signature and represents the first legally mandated transparency standards for frontier AI labs in America.

Anthropic Adds National Security Expert to Governance Trust Amid Defense Market Push

Anthropic has appointed national security expert Richard Fontaine to its long-term benefit trust, which helps govern the company and elect board members. This appointment follows Anthropic's recent announcement of AI models for U.S. national security applications and reflects the company's broader push into defense contracts alongside partnerships with Palantir and AWS.

Industry Leaders Discuss AI Safety Challenges as Technology Becomes More Accessible

ElevenLabs' Head of AI Safety and Databricks co-founder participated in a discussion about AI safety and ethics challenges. The conversation covered issues like deepfakes, responsible AI deployment, and the difficulty of defining ethical boundaries in AI development.

Yoshua Bengio Establishes $30M Nonprofit AI Safety Lab LawZero

Turing Award winner Yoshua Bengio has launched LawZero, a nonprofit AI safety lab that raised $30 million from prominent tech figures and organizations including Eric Schmidt and Open Philanthropy. The lab aims to build safer AI systems, with Bengio expressing skepticism about commercial AI companies' commitment to safety over competitive advancement.