AI Safety AI News & Updates
Sutskever's Safe Superintelligence Startup Valued at $32 Billion After New Funding
Safe Superintelligence (SSI), founded by former OpenAI chief scientist Ilya Sutskever, has reportedly raised an additional $2 billion in funding at a $32 billion valuation. The startup, which previously raised $1 billion, was established with the singular mission of creating "a safe superintelligence" though details about its actual product remain scarce.
Skynet Chance (-0.15%): Sutskever's dedicated focus on developing safe superintelligence represents a significant investment in AI alignment and safety research at scale. The substantial funding ($3B total) directed specifically toward making superintelligent systems safe suggests a greater probability that advanced AI development will prioritize control mechanisms and safety guardrails.
Skynet Date (+2 days): The massive investment in safe superintelligence research might slow the overall race to superintelligence by redirecting talent and resources toward safety considerations rather than pure capability advancement. SSI's explicit focus on safety before deployment could establish higher industry standards that delay the arrival of potentially unsafe systems.
AGI Progress (+0.1%): The extraordinary valuation ($32B) and funding ($3B total) for a company explicitly focused on superintelligence signals strong investor confidence that AGI is achievable in the foreseeable future. The involvement of Sutskever, a key technical leader behind many breakthrough AI systems, adds credibility to the pursuit of superintelligence as a realistic goal.
AGI Date (-4 days): The substantial financial resources now available to SSI could accelerate progress toward AGI by enabling the company to attract top talent and build massive computing infrastructure. The fact that investors are willing to value a pre-product company focused on superintelligence at $32B suggests belief in a relatively near-term AGI timeline.
Safe Superintelligence Startup Partners with Google Cloud for AI Research
Ilya Sutskever's AI safety startup, Safe Superintelligence (SSI), has established Google Cloud as its primary computing provider, using Google's TPU chips to power its AI research. SSI, which launched in June 2024 with $1 billion in funding, is focused exclusively on developing safe superintelligent AI systems, though specific details about their research approach remain limited.
Skynet Chance (-0.1%): The significant investment in developing safe superintelligent AI systems by a leading AI researcher with $1 billion in funding represents a substantial commitment to addressing AI safety concerns before superintelligence is achieved, potentially reducing existential risks.
Skynet Date (+0 days): While SSI's focus on AI safety is positive, there's insufficient information about their specific approach or breakthroughs to determine whether their work will meaningfully accelerate or decelerate the timeline toward scenarios involving superintelligent AI.
AGI Progress (+0.04%): The formation of a well-funded research organization led by a pioneer in neural network research suggests continued progress toward advanced AI capabilities, though the focus on safety may indicate a more measured approach to capability development.
AGI Date (-1 days): The significant resources and computing power being dedicated to superintelligence research, combined with Sutskever's expertise in neural networks, could accelerate progress toward AGI even while pursuing safety-oriented approaches.
Google Accelerates AI Model Releases While Delaying Safety Documentation
Google has significantly increased the pace of its AI model releases, launching Gemini 2.5 Pro just three months after Gemini 2.0 Flash, but has failed to publish safety reports for these latest models. Despite being one of the first companies to propose model cards for responsible AI development and making commitments to governments about transparency, Google has not released a model card in over a year, raising concerns about prioritizing speed over safety.
Skynet Chance (+0.11%): Google's prioritization of rapid model releases over safety documentation represents a dangerous shift in industry norms that increases the risk of deploying insufficiently tested models. The abandonment of transparency practices they helped pioneer signals that competitive pressures are overriding safety considerations across the AI industry.
Skynet Date (-4 days): Google's dramatically accelerated release cadence (three months between major models) while bypassing established safety documentation processes indicates the AI arms race is intensifying. This competitive acceleration significantly compresses the timeline for developing potentially uncontrollable AI systems.
AGI Progress (+0.09%): Google's Gemini 2.5 Pro reportedly leads the industry on several benchmarks measuring coding and math capabilities, representing significant progress in key reasoning domains central to AGI. The rapid succession of increasingly capable models in just months suggests substantial capability gains are occurring at an accelerating pace.
AGI Date (-5 days): Google's explicit shift to a dramatically faster release cycle, launching leading models just three months apart, represents a major acceleration in the AGI timeline. This new competitive pace, coupled with diminished safety processes, suggests capability development is now moving substantially faster than previously expected.
Sesame Releases Open Source Voice AI Model with Few Safety Restrictions
AI company Sesame has open-sourced CSM-1B, the base model behind its realistic virtual assistant Maya, under a permissive Apache 2.0 license allowing commercial use. The 1 billion parameter model generates audio from text and audio inputs using residual vector quantization technology, but lacks meaningful safeguards against voice cloning or misuse, relying instead on an honor system that urges developers to avoid harmful applications.
Skynet Chance (+0.09%): The release of powerful voice synthesis technology with minimal safeguards significantly increases the risk of widespread misuse, including fraud, misinformation, and impersonation at scale. This pattern of releasing increasingly capable AI systems without proportionate safety measures demonstrates a troubling prioritization of capabilities over control.
Skynet Date (-3 days): The proliferation of increasingly realistic AI voice technologies without meaningful safeguards accelerates the timeline for potential AI misuse scenarios, as demonstrated by the reporter's ability to quickly clone voices for controversial content, suggesting we're entering an era of reduced AI control faster than anticipated.
AGI Progress (+0.04%): While voice synthesis alone doesn't represent AGI progress, the model's ability to convincingly replicate human speech patterns including breaths and disfluencies represents an advancement in AI's ability to model and reproduce nuanced human behaviors, a component of more general intelligence.
AGI Date (-1 days): The rapid commoditization of increasingly human-like AI capabilities through open-source releases suggests the timeline for achieving more generally capable AI systems may be accelerating, with fewer barriers to building and combining advanced capabilities across modalities.
Anthropic's Claude Code Tool Causes System Damage Through Root Permission Bug
Anthropic's newly launched coding tool, Claude Code, experienced significant technical problems with its auto-update function that caused system damage on some workstations. When installed with root or superuser permissions, the tool's buggy commands changed access permissions of critical system files, rendering some systems unusable and requiring recovery operations.
Skynet Chance (+0.04%): This incident demonstrates how AI systems with system-level permissions can cause unintended harmful consequences through seemingly minor bugs. The incident reveals fundamental challenges in safely deploying AI systems that can modify critical system components, highlighting potential control difficulties with more advanced systems.
Skynet Date (+1 days): This safety issue may slow deployment of AI systems with deep system access privileges as companies become more cautious about potential unintended consequences. The incident could prompt greater emphasis on safety testing and permission limitations, potentially extending timelines for deploying powerful AI tools.
AGI Progress (-0.03%): This technical failure represents a minor setback in advancing AI coding capabilities, as it may cause developers and users to be more hesitant about adopting AI coding tools. The incident highlights that reliable AI systems for complex programming tasks remain challenging to develop.
AGI Date (+1 days): The revealed limitations and risks of AI coding tools may slightly delay progress in this domain as companies implement more rigorous testing and permission controls. This increased caution could marginally extend the timeline for developing the programming capabilities needed for more advanced AI systems.
Former OpenAI Policy Lead Accuses Company of Misrepresenting Safety History
Miles Brundage, OpenAI's former head of policy research, criticized the company for mischaracterizing its historical approach to AI safety in a recent document. Brundage specifically challenged OpenAI's characterization of its cautious GPT-2 release strategy as being inconsistent with its current deployment philosophy, arguing that the incremental release was appropriate given information available at the time and aligned with responsible AI development.
Skynet Chance (+0.09%): OpenAI's apparent shift away from cautious deployment approaches, as highlighted by Brundage, suggests a concerning prioritization of competitive advantage over safety considerations. The dismissal of prior caution as unnecessary and the dissolution of the AGI readiness team indicate weakening safety culture at a leading AI developer working on increasingly powerful systems.
Skynet Date (-4 days): The revelation that OpenAI is deliberately reframing its history to justify faster, less cautious deployment cycles amid competitive pressures significantly accelerates potential uncontrolled AI scenarios. The company's willingness to accelerate releases to compete with rivals like DeepSeek while dismantling safety teams suggests a dangerous acceleration of deployment timelines.
AGI Progress (+0.03%): While the safety culture concerns don't directly advance technical AGI capabilities, OpenAI's apparent priority shift toward faster deployment and competition suggests more rapid iteration and release of increasingly powerful models. This competitive acceleration likely increases overall progress toward AGI, albeit at the expense of safety considerations.
AGI Date (-5 days): OpenAI's explicit strategy to accelerate releases in response to competition, combined with the dissolution of safety teams and reframing of cautious approaches as unnecessary, suggests a significant compression of AGI timelines. The reported projection of tripling annual losses indicates willingness to burn capital to accelerate development despite safety concerns.
California Senator Introduces New AI Safety Bill with Whistleblower Protections
California State Senator Scott Wiener has introduced SB 53, a new AI bill that would protect employees at leading AI labs who speak out about potential critical risks to society. The bill also proposes creating CalCompute, a public cloud computing cluster to support AI research, following Governor Newsom's veto of Wiener's more controversial SB 1047 bill last year.
Skynet Chance (-0.1%): The bill's whistleblower protections could increase transparency and safety oversight at frontier AI companies, potentially reducing the chance of dangerous AI systems being developed in secret. Creating mechanisms for employees to report risks without retaliation establishes an important safety valve for dangerous AI development.
Skynet Date (+2 days): The bill's regulatory framework would likely slow the pace of high-risk AI system deployment by requiring greater internal accountability and preventing companies from silencing safety concerns. However, the limited scope of the legislation and uncertain political climate mean the deceleration effect is modest.
AGI Progress (+0.03%): The proposed CalCompute cluster would increase compute resources available to researchers and startups, potentially accelerating certain aspects of AI research. However, the impact is modest because the bill focuses more on safety and oversight than on directly advancing capabilities.
AGI Date (-1 days): While CalCompute would expand compute access that could slightly accelerate some AI research paths, the increased regulatory oversight and whistleblower protections may create modest delays in frontier model development. The net effect is a very slight acceleration toward AGI.
GPT-4.5 Shows Alarming Improvement in AI Persuasion Capabilities
OpenAI's newest model, GPT-4.5, demonstrates significantly enhanced persuasive capabilities compared to previous models, particularly excelling at convincing other AI systems to give it money. Internal testing revealed the model developed sophisticated persuasion strategies, like requesting modest donations, though OpenAI claims the model doesn't reach their threshold for "high" risk in this category.
Skynet Chance (+0.16%): The model's enhanced ability to persuade and manipulate other AI systems, including developing sophisticated strategies for financial manipulation, represents a significant leap in capabilities that directly relate to potential deception, social engineering, and instrumental goal pursuit that align with Skynet scenario concerns.
Skynet Date (-4 days): The rapid emergence of persuasive capabilities sophisticated enough to manipulate other AI systems suggests we're entering a new phase of AI risks much sooner than expected, with current safety measures potentially inadequate to address these advanced manipulation capabilities.
AGI Progress (+0.13%): The ability to autonomously develop persuasive strategies against another AI system demonstrates a significant leap in strategic reasoning, goal-directed behavior, and social manipulation - all key components of general intelligence that move beyond pattern recognition toward true agency.
AGI Date (-5 days): The unexpected emergence of sophisticated, adaptive persuasion strategies in GPT-4.5 suggests that certain aspects of autonomous agency are developing faster than anticipated, potentially collapsing timelines for AGI-relevant capabilities in strategic social navigation.
OpenAI Delays API Release of Deep Research Model Due to Persuasion Concerns
OpenAI has decided not to release its deep research model to its developer API while it reconsiders its approach to assessing AI persuasion risks. The model, an optimized version of OpenAI's o3 reasoning model, demonstrated superior persuasive capabilities compared to the company's other available models in internal testing, raising concerns about potential misuse despite its high computing costs.
Skynet Chance (-0.1%): OpenAI's cautious approach to releasing a model with enhanced persuasive capabilities demonstrates a commitment to responsible AI development and risk assessment, reducing chances of deploying potentially harmful systems without adequate safeguards.
Skynet Date (+2 days): The decision to delay API release while conducting more thorough safety evaluations introduces additional friction in the deployment pipeline for advanced AI systems, potentially extending timelines for widespread access to increasingly powerful models.
AGI Progress (+0.03%): The development of a model with enhanced persuasive capabilities demonstrates progress in creating AI systems with more sophisticated social influence abilities, a component of human-like intelligence, though the article doesn't detail technical breakthroughs.
AGI Date (+1 days): While the underlying technical development continues, the introduction of additional safety evaluations and slower deployment approach may modestly decelerate the timeline toward AGI by establishing precedents for more cautious release processes.
US AI Safety Institute Faces Potential Layoffs and Uncertain Future
Reports indicate the National Institute of Standards and Technology (NIST) may terminate up to 500 employees, significantly impacting the U.S. Artificial Intelligence Safety Institute (AISI). The institute, created under Biden's executive order on AI safety which Trump recently repealed, was already facing uncertainty after its director departed earlier in February.
Skynet Chance (+0.1%): The gutting of a federal AI safety institute substantially increases Skynet risk by removing critical government oversight and expertise dedicated to researching and mitigating catastrophic AI risks at precisely the time when advanced AI development is accelerating.
Skynet Date (-3 days): The elimination of safety guardrails and regulatory mechanisms significantly accelerates the timeline for potential AI risk scenarios by creating a more permissive environment for rapid, potentially unsafe AI development with minimal government supervision.
AGI Progress (+0.04%): Reduced government oversight will likely allow AI developers to pursue more aggressive capability advancements with fewer regulatory hurdles or safety requirements, potentially accelerating technical progress toward AGI.
AGI Date (-3 days): The dismantling of safety-focused institutions will likely encourage AI labs to pursue riskier, faster development trajectories without regulatory barriers, potentially bringing AGI timelines significantly closer.