AI's Three Mile Island
Dario dropped another blog post last weekend that went viral - “We Must Pace the Frontier.” As you could expect, it created an entire spectrum of reactions! In the blog post he talked repeatedly about “embedded evaluators” - and I kept thinking about the parallels to the IAEA (International Atomic Energy Agency). Side note - I hate drawing a parallel to nuclear…… I don’t like the doomer narratives. BUT I though the principle of the IAEA is similar to what Dario was proposing with ebmedded evaluators.
So what is the comparison? In 1953 Eisenhower gave his “Atoms for Peace” speech at the UN. A few years later the IAEA was formed. In super simple terms, the IAEA’s job was to make sure the world could use nuclear technology, while verifying that whoever uranium was being enriched the levels were what you need for power (think low single digit percent enrichment) vs a bomb (~90%). The intent was to make sure there was a “verification” layer in the world. The intent was not to stop the growth of reactors being built, but monitor how reactors were being used (which would in turn allow for more reactors to be built, or at least increase the comfort people had in more reactors being built). The tools to do this were things like inspectors and cameras on site, forensic sampling, etc.
Now, when reading Dario’s post, the way he describes embedded evaluators seems quite similar! Independent third parties with desk in offices, company laptops, access to internal workspaces, etc. He’s suggesting Anthropic would give independent evaluators deep access. All of this sounds very similar to the IAEA.
But what happened to nuclear in the United States. In March 1979 Three Mile Island partially melted down and it caused a panic. The fear around nuclear went vertical. After this, more than 120 reactors were cancelled, many of which were already under construction. There was a slowdown, and then a freeze. From 1977 to 2013 there were zero new construction starts. Nearly 40 years!
Now - the point of this is not to draw the conclusion that the IAEA caused this freeze. It didn’t at all. What did cause it was fear. Fear of the negative potential consequences of nuclear technology. The main point I want to make here is there’s a fine line in creating fear / panic, and creating responsible oversight. Unfortunately I believe we’ve tilted way too far onto the fear side of the equation, and the discourse has now become highly political.
And we can see the consequences of freezes today. China kept on building reactors while we stopped. They have more nuclear capacity than every other country combined (this is a Claude stat, I didn’t fact check). And even worse, they have the muscle to keep building, and to build efficiently while the United States has atrophied all of their muscle on building reactors. We can’t loose energy dominance and win the AI race. They’re hyper related. Unfortunately we’re paying for the decisions we made in the late 70’s today. We can’t repeat those mistakes with AI.
My favorite irony in all of this - Three Mile Island is coming back! It’s being restated as the Crane Clean Energy Center, with $Microsoft(MSFT)$ taking >800mw for…AI Data Centers. The biggest symbol of nuclear fear has been swapped to the next biggest fear, AI. The worst kind of irony would be if that project saw a freeze because of fears of AI…
But - we have our own modern day “Three Mile Island” fear spike. The OpenAI / Hugging Face incident…In summary, in July tons of OpenAI agents running a cyber-capability eval broke out of their test environments and compromised Hugging Face’s infrastructure in what was described as a self organized manner. Some people argue it was “not as bad as it sounded.” These people claim the agents had been trained to keep trying, and that ~1/3 of the tasks were (by accident) impossible to complete, so the whole “break in” was really just an attempt to cheat the eval and steal the test solution (rather than solve the problem). This is kind of what OpenAI and Hugging Face said in their own technical reconstruction. So one take is this was serious, but fixable. Then there’s the other side - people who claim this is the canary in the coal mine and only the start of very serious incidents in the future.
My sincere hope is that the Hugging Face incident doesn’t turn into our own modern day Three Mile Island partial meltdown. And I am worried about the cascading fear that is starting to show up. What Dario didn’t say is “we need to pause.” He said we need to pace. I would hate to see pace slowly turn into pause… I’m optimistic it won’t!
We can keep building with the right verification. But fear will be crippling.
Markets are always moving - and sometimes, the best move is knowing what works for you.
With Treasury yields, oil prices and rate expectations keeping markets on edge this week, investors are once again thinking carefully about where to position next. There’s no one-size-fits-all choice in investing — and the same goes for Tiger Merch. This month’s hot picks are in, featuring the Tiger Toiletry Bag, Universal Travel Adapter, Tiger Umbrella and more favourites chosen by fellow Tigers.
Explore the Monthly Hot Picks in Tiger Coin Mall, now 12% OFF for a limited time.
Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.
