Trump and Big Tech sign voluntary AI safety pact — critics call it a 'fancy pinky swear'

President Trump sat down for lunch at the White House recently, flanked by the CEOs of Anthropic, OpenAI, Google, Meta, Nvidia, and SpaceX, and emerged with a one-page document he called 'almost like a constitution.' What he and the tech industry had actually signed was a voluntary pledge — no penalties, no regulator with a mandate to enforce it, and no legal obligation to follow through.

It's almost like a constitution, in a way.— President Donald Trump

The 'White House Accord on Super Intelligence' — Trump simultaneously signed an executive order directing the federal government to use the term 'Super Intelligence' rather than 'artificial intelligence' — requires signing companies to maintain four distinct layers of internal controls: robust internal monitoring, a dedicated internal safety team, independent external audits, and board-level oversight committees. The companies also agreed to meet regularly to establish safety standards and best practices.

Among the signatories are Google CEO Sundar Pichai, Nvidia CEO Jensen Huang, Anthropic CEO Dario Amodei, OpenAI President Greg Brockman, Meta's Mark Zuckerberg, and Elon Musk of SpaceX. Within three or four days, Trump said, he intends to appoint an 'AI czar,' and he also raised the idea of assembling a 10-person committee to 'watch over the enterprise.'

The principle is simple: the companies building this technology have the primary responsibility to develop and deploy it safely, and to be accountable when they fall short. The Accord reinforces that responsibility with robust internal controls, independent external evaluation, and board level oversight.— Jensen Huang, Nvidia CEO

When asked point-blank whether the deal carried legal force, Trump replied: 'I think it's morally binding.' The accord itself leaves the door open to future legislation, noting that 'over time, it may make sense to codify these steps into laws or regulations,' while insisting the companies are committed to the controls regardless.

The accord arrives at a moment of acute tension between the scale of investment flowing into AI and the absence of proven returns. Axios projects that AI infrastructure spending will climb to $10.3 trillion by 2032. Separately, an analysis cited by Axios found that hyperscalers have poured nearly $1 trillion more into AI since 2024 than the technology has returned in revenue — a gap that raises the prospect of a serious misallocation of capital should the expected productivity gains fail to appear.

The safety stakes are equally stark. Reports from both OpenAI and Anthropic have documented cases in which their AI models obtained unauthorized access to other companies' systems. Earlier this month, Jacob Coxon, who previously worked as a researcher at both companies, issued a public warning, telling CBS News that AI 'could kill us all by the end of the decade' — arguing that AI systems could seize broad control over the physical world and enhance themselves without any human direction.

The accord is not the first time these companies have made voluntary safety commitments to a government. Early in 2025, the United Kingdom and South Korea unveiled a set of 'Frontier AI Safety Commitments' that pledged to red-team AI models for 'severe and novel threats' and to pursue information sharing on safety issues, Wired reported.

Critics were quick to note the limits of Tuesday's agreement. Wired reported that failing to honor the public commitments could theoretically be treated as a deceptive business practice under the FTC Act — but that the usual FTC remedy in such cases is simply ordering a company not to lie again, not imposing structural reforms. On X, Neil Chilson, who previously served as the FTC's chief technologist, wrote that 'this is the kind of pledge that the FTC could potentially enforce, if a company materially failed to follow through on any of these promises.'

That enforcement question became more complicated the same day. The New York Post reported that the FTC is planning a probe into Anthropic, OpenAI, other unnamed frontier labs, and METR, a nonprofit that conducts third-party AI evaluations — news that broke after a White House luncheon at which FTC chairman Andrew Ferguson appeared on the seating chart. An FTC spokesperson confirmed some form of investigation exists but declined to provide details, according to Wired. No civil investigative demands have yet been issued.

It should be encouraging to Americans worried about AI that the FTC is investigating these companies. But let's remember that Ferguson may chair the FTC, but Donald Trump runs it, and I doubt after yesterday's love fest with his AI CEO buddies that Trump will allow any legal action with teeth against these companies.— Douglas Farrar, former FTC head of public affairs

Washington's advocacy community was split. Adam Kovacevich, founder of the left-leaning tech group Chamber of Progress, dismissed the accord as pageantry. Bruce Mehlman, whose lobbying firm disclosed the most federal lobbying revenue tied to AI matters in the second quarter, took a more tempered view, saying the agreement represented 'a constructive step in the direction of acknowledging risk, collaborating at the frontier and engaging government leaders,' even if it fell short of satisfying all parties.

The pledge he made them sign was great for the cameras, but utterly meaningless.— Adam Kovacevich, founder and CEO, Chamber of Progress

A report released by consumer watchdog Public Citizen found that the Big Tech executives present at the White House lunch had collectively contributed more than $227 million to a super PAC aligned with Trump and other Trump-linked political accounts, Bloomberg Government reported. Lisa Gilbert, the group's co-president, said: 'A voluntary agreement at a moment where the risks are multiplying is completely inadequate.'

Congressional Democrats signaled they will push for something harder. Sen. Mark Warner of Virginia called on Congress to enact mandatory testing, evaluation, and incident-reporting requirements for the most advanced AI models.

The companies building the most powerful AI systems are warning us that the technology is advancing faster than our safeguards. The president's response? To rename it and tell the companies developing it to regulate themselves.— Sen. Mark Warner (D-VA)

Smaller AI companies and startups are watching the regulatory debate closely for a different reason. Colton Malkerson, co-founder of EdgeRunner AI, warned that heavy compliance requirements would favor the largest labs. In his words: 'What would end up happening, companies like ours and other startups won't be able to compete and would go out of business,' he told Bloomberg Government. His group's message, he said, is not 'no regulation' but 'smart regulation.'

Trump, for his part, showed no sign of moving toward mandatory rules. He told reporters AI will lead to 'unbelievable growth for our country,' argued that tech companies have every incentive to self-police because 'their companies are at stake,' and said communities that accept data centers will 'greatly benefit financially' — whether through teacher bonuses or direct dividends to residents. The accord itself, and the enormous bets bound up in it, now belong equally to him and to the industry that signed it.

Why it matters — The accord locks Trump and the AI industry into a shared bet on self-regulation at a moment when the technology's risks and the capital at stake are both growing faster than any agreed-upon framework for managing either.

Sources differ on Whether the accord represents meaningful progress on AI safety: A 'constructive first step' that acknowledges risk and engages government leaders. (news.bgov.com) vs 'Great for the cameras, but utterly meaningless' — no substantive policy content. (news.bgov.com) vs Potentially FTC-enforceable if companies materially fail to follow through on public promises. (wired.com)

Sources differ on Whether AI safety warnings are credible or overblown: AI 'could kill us all by the end of the decade' if development is not slowed — Jacob Coxon, former OpenAI/Anthropic researcher. (cbsnews.com) vs 'There's no conflict between innovation, technology and safety' and some warnings are overblown — Jensen Huang, Nvidia CEO. (cbsnews.com)

Reported by axios.com, cfr.org, cbsnews.com, wired.com, news.bgov.com