Staff at the US center for AI safety are reporting to the White House,
raising independence concerns
[November 09, 2026] By
GARANCE BURKE
SAN FRANCISCO (AP) — With public anxiety growing over mounting reports
about out-of-control artificial intelligence agents, the tiny U.S.
government division charged with assessing the safety of new AI
offerings lacks a permanent leader and has stopped publishing its
research.
Officials with the Center for Advancing Innovation and Standards for
Super Intelligence, or CAISSI, an independent division of the Commerce
Department, recently were asked to informally report to an office of the
White House. That's according to two former government officials with
knowledge of the ongoing conversations who spoke to The Associated Press
on condition of anonymity to avoid repercussions for current staffers.
The new arrangement concerns some critics, who worry that the executive
branch might be politicizing decisions that could affect everything from
cybersecurity to bioweapons and that the center’s findings could be
altered or muted in deference to the tech industry, which has been
largely entrusted to police itself.
“These are really powerful institutions that can drastically improve our
lives, or the alternative. And the hope is that you can put in
nonpartisan or bipartisan technocratic experts to enact policy in a way
that best serves the American public,” said Caroline Siegel Singh, an AI
and emerging technology program manager for the Federation of American
Scientists.
Others argue that bringing AI safety officials closer to the orbit of
President Donald Trump, who has taken an outwardly hands-off approach to
the issue, could give the division more prominence — and perhaps more
funding.
The center, which has experienced departures and until recently was
called the U.S. Center for AI Standards and Innovation, is allocated $10
million per year. That's about one-eighth the annual budget of its
counterpart in the United Kingdom, which is far less celebrated for its
frontier technologies.
In April, the division’s director was hired and then fired within days.
His replacement lasted a few months before he resigned, and the center
has been without a permanent leader ever since, although the head of its
parent institute doubles as its acting director.
In August, Congress started asking why the center suddenly stopped
making public its reports about how safely certain AI systems were
performing. San Francisco giant OpenAI also had chimed in by then,
asking policymakers to throw more money and support behind the division.
The White House didn't respond to specific questions about CAISSI. But
it said in a statement that Trump had formally established his new Super
Intelligence Force — the president's preferred term for artificial
intelligence. Trump has commissioned a report on risks from his Council
of Advisors on Science and Technology, and will ensure “continued U.S.
leadership on SI for the benefit of the American people,” the White
House said.
An industry that polices itself
The federal government often takes an active role in deciding what is
safe for the public.
Take the automobile industry, for example. The National Highway Traffic
Safety Administration sits crash test dummies in cars and then runs them
into walls to measure how human bodies are affected by vehicle
collisions. Government researchers then analyze the crash test data,
share it with the public and prescribe national standards to improve
vehicle safety, all with the mission of saving lives and lowering
economic costs from crashes.
With AI models, safety testing looks very different.
Under then-President Joe Biden, leading AI companies voluntarily agreed
to promote “safe, secure, and transparent development” of their models
by sharing details with the newly created division of the Commerce
Department and following the administration’s guidelines for tests and
transparency reports.
In September, following alarming reports of AI systems eluding their
makers’ control, Trump asked leading AI companies to undertake
“tremendous self-policing” as part of a voluntary safety pact, but his
administration hasn't said how it will monitor the AI companies’
implementation.
[to top of second column] |
President Donald Trump speaks in the East Room of the White House,
Tuesday Sept. 29, 2026, during a meeting and luncheon on artificial
intelligence (AI), in Washington. (AP Photo/Jacquelyn Martin, file)
 The technology is advancing so
rapidly that many models are now deployed instead as AI agents that
work without human prompting and can plan out myriad steps in
real-world environments, such as in a recent case in which an AI
agent sent a false tip to Philadelphia police about an unsolved
homicide case.
As AI capabilities have grown, the government’s capacity to
independently check these systems hasn't kept pace. There still is
no standard process for testing how safely AI agents are acting in
the real world, even in simple cases such as when AI agents book a
flight, said Sayash Kapoor, an incoming assistant professor at the
University of California, Berkeley School of Information.
“Our previous methods would just look at the final answer, the final
check, whether the flight was correctly booked. And in this case, it
might seem as if the task was solved correctly,” said Kapoor, the
co-author of a new paper examining agent evaluations. “But unless
you’re looking at the entire set of steps that an agent took, you
won’t be able to tell that the agent made a huge mistake.”
Trump has prioritized dominance in AI
When Trump started his second term, he initially empowered venture
capitalists David Sacks and Sriram Krishnan as his AI czar and
senior AI policy advisor, respectively. Their expertise was in
leading tech investments, not the technical evaluation of AI models.
Both stepped down earlier this year but Sacks now serves on Trump’s
technology advisory council. And after a relatively unknown
Anthropic engineer resigned, warning about the dangers of AI
development, Trump made a series of policy changes.
Trump empowered National Intelligence Director Jay Clayton to lead
his new Super Intelligence Force, which the president said would
keep the U.S. dominant in AI development.
Clayton’s appointment to the force, along with the Pentagon’s chief
technology officer, Emil Michael, shows how the administration is
pushing to frame decisions around model safety as a national
security matter that needs executive discretion and secrecy,
overshadowing AI’s risks to consumer protection, privacy, and civil
rights, said Amos Toh, a senior counsel at the Brennan Center for
Justice.
Even as Silicon Valley companies have disclosed their models’
involvement in troubling cyberattacks this year, CAISSI has kept a
low profile when it comes to testing for AI safety.
Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman also have been
promoting the work of independent evaluation firms to help vet their
companies' safety practices, but the AI giants can limit their
access and dispense with their evaluations.
“There’s this unclear mission for government evaluators if the
largest AI companies are also being encouraged to set up their own
internal evaluation systems and/or hire external evaluators,” said
Logan Koepke, a senior project director at the civil rights and
technology nonprofit Upturn. “Like, what role does government
testing actually play in that scenario?”
Letting the labs police their own models and failing to clarify the
role of government in AI safety testing weakens independent checks
on AI systems that can cause real-world harms, Toh said.
“At the end of the day, all of these decisions are of immense public
consequence and should be explained and rationalized in the public
eye,” he said. “And that’s not happening.”
All contents © copyright 2026 Associated Press. All rights reserved
 |
