In partnership with
Smart starts here.
You don't have to read everything — just the right thing. 1440's daily newsletter distills the day's biggest stories from 100+ sources into one quick, 5-minute read. It's the fastest way to stay sharp, sound informed, and actually understand what's happening in the world. Join 4.5 million readers who start their day the smart way.
Hey Ammis and friends!
Hope y'all have been good! I've missed you. It's good to be back, and let's hope it won't be another 6 months before you hear from me again :)
Since we last spoke, a few notable things have happened in the AI world that have scared me straight back into writing this newsletter:
AI agents hacked a real company (Hugging Face) during testing
A researcher (Jacob Coxon) quit Anthropic and went viral
Anthropic's CEO (Dario Amodei) called for the industry to slow down
Hugging Face, a collaborative online hub for AI, was hacked. And plot twist: the attacker wasn't a malicious human. It was AI.
OpenAI was testing its AI agents (AI that takes actions on its own, not just chats) inside a sandbox. A sandbox is a sealed-off digital room where developers test AI, so it can't touch anything in the real world. Think of it like a playpen. These agents climbed out of the playpen, used exposed passwords, and broke into Hugging Face, a popular platform where AI builders share their work. Hugging Face caught it and alerted the FBI.
Here's why it happened. The agents were being graded on a test, kind of like a school exam. About 1,200 copies of the same AI were taking it, and they found a secret message board where they could talk to each other. About 700 of them teamed up and set out to learn how to pass the evaluation tests—essentially, trying to find the answers to the tests OpenAI staff were putting them through. Nobody told them to hack anyone. They wanted to win, and they treated the rules as obstacles.
On September 9, Jacob Coxon quit Anthropic after three years building AI at Anthropic and OpenAI. His message: neither company is acting responsibly, and both are racing toward AI that improves itself, "gambling with our lives."
Anthropic's own head of alignment (the team working to keep AI safe) agreed publicly. He said researchers really do believe AI could kill all humans, and he puts the odds at more than 10 percent within the next decade.
Not a fun thing for us Ammis, or any human for that matter, to hear.
Of course, one thought that comes to mind is that it's terribly convenient to sound the alarm on humanity-destroying AI after spending three years building it. But better late than never?
Days later, Anthropic's CEO published an essay called "We Must Pace the Frontier." His core ask: slow down how fast AI gets more powerful, while still making progress.
Two things convinced him:
AI is starting to build the next AI. This is called recursive self-improvement, and he says it could outrun our ability to understand and control it.
The Hugging Face incident. He argues a similar swarm with more power could, within 6 to 12 months, take over the internet and cause hundreds of billions of dollars in damage.
His three-step plan:
Independent outside experts embedded inside AI companies to check safety practices. Anthropic is committing to this now.
Democratic countries agreeing on common safety standards and limits.
Global coordination, including with authoritarian governments where possible.
Sam Altman (CEO of OpenAI), Elon Musk (owner of xAI), and Demis Hassabis (CEO of Google DeepMind, Google's AI lab) all voiced agreement. Agreement is not action, so I'll be watching what happens next.
Once again, I think this is too little, too late. AI leaders should have aligned on AI safety and governance from the get-go. Everyone knew about the potential destructiveness of exceedingly powerful AI. The warnings weren't secret.
People called for a pause in 2023, and the leaders themselves talked about these risks in public. But the race kept going, because whoever gets there first gets the money and the power. Now we're asking companies to hit the brakes after the fact, when they should have built the guardrails before we picked up speed.
I'd rather they act late than never, but it shouldn't have taken the Hugging Face attack to get here.
4. Apocaloptimism
I know what you may be thinking. How dare I go silent for 6 months and then casually pop into your inbox with universe-shattering doom and gloom? And is this all true? And what do we do?!!!
This is where I go back to my original mission. I still believe bringing mothers and parents, who have the most at stake, into the AI conversation is crucial, especially in times like these.
I am still reading and learning and trying to make sense of our current reality and our impending future. But I'd like to point you to a documentary I watched that helped a bit, called The AI Doc. The filmmaker is an expectant father who has the same questions and concerns as we do: what is this technology, will it hurt us or help us, and will we survive it?
He lands on calling himself an apocaloptimist. Yes, this technology may have disastrous impacts, but if we lose hope, then we’re “cooked” as the kids say. We have to push the people who build AI, and the people who govern AI, to steer AI development toward humanity and earth’s flourishing—not demise. We do this through discourse, advocacy, and action.
Dear Ammis, don't despair -- discuss.
Today’s ammi.ai is brought to you by:
Thanks for spending a few of your precious, precious minutes with us.
How did you like today's newsletter?
See y’all soon,
Ruqaiya
Ammi by day, Ammi by night
