Chongqing, China – Three years ago, in the idyllic town of Woodside, south of San Francisco, the United States and China held their first high-level talks on the threats posed by artificial intelligence. President Xi Jinping and his longtime foreign minister appeared sincere in their conviction that a channel should be established between Beijing and Washington — a red call for AI in case of emergency.
He authorized a diplomatic effort that will begin in Switzerland a few months before the US presidential election in 2024. According to four sources who participated in the talks, a large US delegation had arrived with high expectations that were suddenly dashed. The Chinese team dismissed US concerns over runaway AI as academic, almost theoretical, and quickly turned the conversation to export controls that Beijing saw as another US effort to hold China back.
“They naturally see any U.S. diplomatic initiative as a trap that involves limitations or restrictions of one flavor or another on a capability,” Jake Sullivan, the U.S. national security adviser under President Biden, said in an interview.
Despite mistrust – and losing the White House to Democrat Donald Trump – an agreement was reached in Peru in November of that year, where both sides agreed to keep AI out of command and control of nuclear weapons.
“It was like breaking the seal that we could actually do something on AI,” Sullivan said. “In transition, I told the incoming Trump team that they should really embrace that dialogue. But the Trump administration’s approach was much more laissez-faire, and they weren’t particularly interested in it.”
“That’s all changed in the last few weeks,” he said.
The Trump administration, once eager to gun for technological supremacy, is now considering for the first time how much power AI could gain if left unchecked.
In a surprising reversal, officials told The Times that there have been quiet discussions to explore revived talks over an emergency channel ahead of President Trump’s state visit to China this week, driven by shared concern in Beijing and Washington over the introduction of Anthropic’s powerful new model mythos.
A senior administration official told reporters Sunday that the White House wants to create a channel of communication for AI, like others, which they have in “a number of areas that have a deep focus on the US and China.”
“I think what that channel of communication looks like, what the formality of that is and what that looks like has not been determined yet,” the official said, “but we want to take this opportunity with the leaders’ meeting to start the conversation.” We should establish a channel of communication on that matter.
Mythos’ capabilities are seen across industry and government An unprecedented cyber weaponCapable of infiltrating and exploiting digital communications systems, including government databases, financial institutions, and health care programs – with untold consequences.
It is not yet clear whether any announcement will be implemented this week or not. Experts say any negotiations between the United States and China on AI rules — designing some kind of arms control agreement governing the use of a technology that neither side fully understands or controls — will be fraught with doubt, misunderstanding and risk.
“Right now, there is almost no support from US policymakers to engage in formal discussions on AI governance with China,” said Alok Mehta, director of the Wadhwani AI Center at the Center for Strategic and International Studies.
“The argument is that this is a winner-takes-all race,” Mehta said, “and it is important to accelerate AI progress to ensure that the United States wins that race.”
America is at the forefront
China will enter those discussions with a powerful argument, that American leadership in AI – and the prevailing strategy of American AI companies – is leading the world to an alarming extent.
Each of the major US players in this field – OpenAI, Google, Anthropic, Microsoft and Meta Platform – is racing to be the first to create artificial general intelligence, or AGI, a limitless, capable model without a common definition, but most agree that performing any intellectual human task will require a model.
The prevailing theory is that the first person to achieve AGI will receive a reward that will multiply itself: a self-training, repeatedly improving intelligence, growing faster and surpassing all competitors.
In contrast, Chinese companies are pursuing a state-sanctioned strategy that focuses on integrating AI into integrated industries and systems, training models to improve individual functions, and accelerating development in a more tailored approach.
“The Chinese believe there is not one race, but many races,” said Scott Kennedy, senior adviser on Chinese trade and economics at the Center for Strategic and International Studies. “The US is focusing on achieving AGI, while China is focusing on the diffusion and applications of AI in the rest of its economy – manufacturing, humanoid robotics, all aspects of the Internet of Things.”
China scholars, AI industry insiders, and successive administrations have questioned Beijing’s strategic thinking and candor.
“It’s so ingrained in the community here that AGI will have this transformative potential that people can’t believe China isn’t also focusing on it,” said Matt Sheehan, a scholar of global technology issues at the Carnegie Endowment for International Peace who focuses on China. “It says it’s focused on applications, but is it a copy of an AGI program hidden somewhere in the mountains?”
But most insiders believe Beijing’s guidance to Chinese companies exposes its true intentions.
“They’re not as inundated with AGI as the United States is, and I think that remains the case today,” Sullivan said, so he found a lot of the conversation in the U.S. about extreme border risk — misalignment and loss of control — to be a bit abstract, and not really as relevant to how he sees the spread of AI in China.
President Biden welcomes Chinese President Xi Jinping in Woodside, California in 2023.
(Doug Mills/Pool Photo)
Although China’s progress has exceeded US expectations — especially since DeepSeq released its model a year ago — the state has focused computer power on specific applications rather than the broader strategy needed to develop more powerful models capable of moving toward AGI.
“It’s not just chips. It’s money,” Sheehan said. “China’s leading companies are much more financially constrained than US companies. There are concerns about a bubble here, but OpenAI is valued at about $800 billion. The leading Chinese companies going public are valued at $20 billion. There is only an order-of-magnitude difference in the available financing.”
Still, some in the U.S. government fear that China will steal technology wholesale if it doesn’t need comparable computing power.
It is not easy to do this. But last month, in a memo, the White House Office of Science and Technology Policy accused Chinese actors of “waging an industrial-scale campaign to distill American frontier AI systems” that were essentially copying the performance of the most advanced existing models “at a fraction of the cost.” The memorandum did not accuse Beijing of supporting the activity.
The memo said that in this process, carefully maintained security protocols are deliberately removed.
China benefits from talks
Whatever its strategic calculations, China will enter negotiations with the Trump administration trailing in the race – while disagreeing over the nature of the finish line.
AGI could, in theory, reach a stage of iterative self-improvement that could result in the loss of human understanding or control. But if it is only the Americans, not the Chinese, who want to reach that limit, who is responsible for stopping it?
Daniel Remler, who led AI policy at the State Department during the Biden administration and participated in the Geneva talks, cast doubt on Chinese claims of disinterest in AGI and ignorance of its risks. He said, China’s lagging behind in the race is not a strategic intention.
“Chinese technologists are close observers of the American AI ecosystem, and sometimes they say what they think,” Remler said. “Many were impressed by the (Mythos) model to the point of despair. Leaders of China’s top AI labs have been vocal in recent months, even before Mythos, about how compute-constrained they are at the limit. Some have said they will never catch up to their American competitors.”
Negotiations at this point in the race could follow a familiar pattern in the recent history of US-China diplomacy, in which Beijing claims it lags behind the United States in development, ultimately gaining a foothold and more concessions at the negotiating table.
In other competitive domains – such as China’s entry into the World Trade Organization and cybersecurity negotiations between Beijing and the Obama administration – agreements were ultimately reached that Washington believes would disenfranchise American companies.
The Trump administration “needs to approach AI diplomacy with China with clear expectations based on our own national interests,” Remler said.
Silicon Valley itself is divided over regulating AI. Anthropic, which was founded out of concern that other AI companies were failing to take security and alignment concerns seriously, raised concerns with the Trump administration over its own model mythos, a moment that has prompted the White House to consider the best path forward.
Alarmed after meeting with leaders of America’s top banks over their vulnerabilities, Treasury Secretary Scott Besant internally advised the US government to review future model releases – a practice already underway in China, where training parameters for models, known as “weights”, are publicly released.
Even the suggestion of government surveillance drew sharp reaction in Silicon Valley. Last week, the White House sent a memo to reassure industry partners that submitting new models for federal review would be entirely voluntary.
If talks on AI do eventually resume between Washington and Beijing, experts believe the talks will be far more complex than the arms control agreements that governed nuclear weapons in the Cold War.
The superpowers will discuss not only the dangers of instability in the global financial system, but also proliferation fears – that advanced AI tools would fall into the hands of bad actors interested in using bio or cyber weapons that could target both countries.
And ultimately they will have to decide whether to discuss regulating the integration of AI into the Chinese and U.S. militaries, an almost unfathomable goal between the world’s greatest adversaries, where trust is lowest and verification will be most difficult.
Those in the industry who fear most what artificial superintelligence could bring have told the Trump administration that dialogue with China is an existential necessity.
Dario Amodei, chief executive and co-founder of Anthropic, speaks at an event in New York in 2025.
(Michael M. Santiago/Getty Images)
But even within Anthropic, which has advocated diplomacy, there are concerns that Beijing could take advantage of its current disadvantage to entangle American industry at its peak.
Instead of pushing for a single comprehensive agreement, industry insiders are advising the administration to strike targeted deals with Beijing to mitigate specific risks, such as an agreement on nuclear command and control, two industry sources said.
Privately, Sullivan said, both Xi and Chinese Foreign Minister Wang Yi understood that the gravity of the emerging technology facing them required some kind of cooperation.
Sullivan said, “On an ideological level, I believe he had a strong belief in it and authorized it, but I believe his level of urgency was much lower than ours, and I saw it as a long-term process that would be implemented over time.”
“Their level of urgency and their stake in this has increased,” he said.
