Study Hacks Blog

Over the last several months, the two leading frontier AI labs have shown some brazen behavior.

It started with a series of​ carefully planned announcements​ and reports from OpenAI that try to establish how unnerving and powerful (not to mention ​felonious​) their LLM-powered agent systems have become.

They then handed the baton to Anthropic, whose employees began ​publicly debating​ the exact probability that these technologies would lead to human extinction. They delivered this all with an eerie calmness that conveyed a nihilistic inevitability about their work.

The ground suitably softened, the campaign crescendoed with Anthropic CEO Dario Amodi releasing a letter, titled ​“We Must Pace the Frontier,”​ in which he enumerates all the harms his own company’s research might cause, and then, instead of apologizing and promising to stop, concludes that catastrophe can only be avoided if we “build the technology in the right way,” which involves – surprise, surprise – the government slowing down potential competitors and allowing the labs to take the lead in advancing the relevant tech. OpenAI CEO Sam Altman quickly ​tweeted his support​ for this brave plan.

Internally, these labs have long embraced this style of messianic thinking, casting themselves as humanity’s only hope against the defied powers of superintelligent AI. This summer, in some sense, was their attempt to induct the rest of us into this long-held ideology.

It didn’t work.

Read more

Yesterday, Jacob Coxon, an Anthropic engineer, resigned. As he ​explained on X​: “[OpenAI and Anthropic] are racing straight to self-improving superintelligence and gambling with our lives.”

A former AI lab employee making these types of claims isn’t unprecedented. (I remember hearing back in 2023 about an ex-OpenAI engineer who was certain the company was just months away from runaway recursive self-improvement.) What proved more shocking were the responses that soon followed.

Evan Hubinger, the head of “Alignment Science” at Anthropic, ​tweeted the followin​g:

“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

And he wasn’t alone. Another Anthropic engineer, Samuel Marks, ​joined the conversation​:

“AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.”

This is stunning.

Multiple representatives of an American company are publicly claiming that they’re building what they believe to be a weapon of mass destruction that they likely can’t prevent from accidentally deploying. They then declared that they have no interest in stopping.

Recently, I’ve been trying to move away from combating hype in AI discourse, as it’s exhausting, and it distracts me from my core work of helping humans flourish in our technological world. But this rhetoric has become so brazen, anti-humanist, and, quite frankly, deranged, that I felt I needed to join the chorus of voices that have started to push back today.

Read more

Wendell Berry died last week at 92 at his home in Port Royal, Kentucky, where he farmed his land using traditional techniques and wrote with pencil and paper in a cabin lacking electricity.

Berry was celebrated for his polemics against industrial agriculture and his elegiac tributes to family farms and small-town living. But for those with no direct connection to food production or ability to move to a rural enclave, Berry’s example points toward a broader lesson: the great satisfactions of living one’s life on purpose.

As a young man, Berry set out on a path toward an elite academic career. Soon after earning a bachelor’s and master’s degree at the University of Kentucky, he headed to Stanford to study creative writing, followed by a stint in Italy and France (funded by a Guggenheim fellowship). He later took a professorship at New York University’s Heights campus in the Bronx. Along the way, he got married and had two kids.

This was the situation he found himself in when, in 1964, at the age of 29, Berry made a radical decision: He would quit his job to move back to Henry County, Kentucky, where his family had lived and farmed for many generations.

Here’s how the writer David Demaree ​summarized​ what happened next:

Read more

Back in January, I received a note from a senior software engineer in Silicon Valley. He described himself as an AI skeptic who became converted after trying Claude Code for the first time. “Overnight, it changed the way I do my job,” he wrote. “It’s really, really good.”

As he explained, he no longer used a standard development environment. Instead, he “exclusively uses Claude Code” to get the job done, interacting with the tool in a terminal window and allowing it to program on his behalf.

“If I had to guess,” he concluded, “I’d say a task that would have taken me a week now takes me 2 days.”

This past winter, when I surveyed more than 300 software developers to learn how AI was transforming their jobs, the majority told a similar tale of shifting from writing their own code to instructing AI agents. The speed with which this new tool became ubiquitous in this industry was stunning.

This story matters for the rest of us because AI coding tools have emerged as the prime example of the power of AI—the first step of many more soon to come on this technology’s disruptive march through our work and our lives.

But what if the reality here is more complicated?

Last week, I received a new message from that same senior engineer who wanted to share an alarming addendum to his tale…

Read more

A couple of weeks ago, the AI company Hugging Face ​announced​ that they had discovered an intrusion into their production infrastructure. They didn’t know the source, but noted that large language models appeared to be involved. The following week, OpenAI ​admitted​ that the breach was the result of an AI system test that went awry.

The initial news coverage created the sense that something unnerving had just occurred:

• ​The Wall Street Journal​ called it “the stuff of cybersecurity nightmares.”
• ​The Hill​ said, “Washington and the technology industry are on high alert this week after OpenAI revealed that some of its AI agents went rogue.”
• ​The AP​ quipped that “to be fair, James Cameron did warn us” (a reference to The Terminator), before describing the event as a “told-you-so moment for researchers who had warned for years that the technology could pose an existential threat to humanity.”

Yikes! It’s perhaps not surprising, then, that I’ve received more emails about this incident than any other recent AI story I can remember.

So, what really happened here?

Read more