Agent5Seven topics. One clear view.
Agent5

The latest AI news at a glance: seven topics, and a short, plain-English read on the stories. Free, no account.

Product
HomeHow it worksFAQAboutHow we fact-checkBlog
News
All newsModels & ReleasesFunding & DealsBenchmarksAgents & ProductsHardware & ComputePolicy & DramaRobotics
More
AdvertisePressPrivacyTerms
© 2026 Agent5
Agent5
Blog
Agent5
Blog
  1. Home›
  2. News›
  3. Policy & Drama›
  4. OpenAI will honor shared robots.txt signal for AI training preferences
Models & Releases24Funding & Deals20Benchmarks12Agents & Products70Hardware & Compute15Policy & Drama30Robotics7
Policy & Drama

OpenAI will honor shared robots.txt signal for AI training preferences

OpenAI: Making it easier for websites to express AI training preferences

OpenAI will honor a shared robots.txt signal to help website operators communicate their AI training preferences to participating providers.

Get smart on it

OpenAI announced support for a common signal in robots.txt that allows website operators to communicate their preferences about using content for foundational generative AI training to multiple AI providers at once. Previously, website operators had to identify individual crawlers and maintain separate rules for each provider, so this shared signal reduces that administrative burden. The effort is based on ongoing work by the Internet Engineering Task Force's AI Preferences working group, and OpenAI plans to implement the signal in the coming months while continuing to respect existing crawler controls that websites can already set.

Relevant courses

Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.

Browse AI courses on Maven →
Sources
OpenAI ↗
The biggest AI stories, once a day

One short email a day with the stories that matter. Free. Unsubscribe with one click.

Related news
Policy & DramaOpen story →

As Anthropic finally files for IPO, rising risks aren’t slowing the AI buildout, yet

The other artificial intelligence shoe dropped this week, as Anthropic filed for what surely will be the second blockbuster AI initial public offering of stock this year. Nearly 70,000 words in the filing list a ton of risks, but in short, it’s making unimaginably huge bets on buying computing power, it’s uncertain on how long […] The post As Anthropic finally files for IPO, rising risks aren’t slowing the AI buildout – yet appeared first on SiliconANGLE.

Policy & DramaOpen story →

AI can turn security fixes into working exploits before users receive an update

GLM 5.3 turned published-but-unshipped Firefox security fixes into working one-click exploits for six releases, at roughly $1,000 each.

Policy & DramaOpen story →

White House makes Section 232 determination regarding foreign-made robots

A Section 232 investigation by the secretary of commerce found that robotics imports pose a threat to U.S. national security and economic competitiveness. Source: Office of the White House Yesterday, U.S. President Donald J. Trump issued a proclamation stating that the secretary of commerce had submitted a Section 232 report about the national security effects of imports of robotics and industrial machinery, as well as related parts and components. “Based on the facts considered in that in

Policy & DramaOpen story →

A Defense Contractor Gave Their Personal AI Agent Access to a Secure Pentagon System

Sources tell WIRED that an Accenture employee’s personal Claude agent ran on a DOD network for two months before it was identified and the vulnerability could be patched.

Policy & DramaOpen story →

Anthropic is asking Wall Street to believe in AGI

Today on Decoder, I’m joined by Joey Brookhart, an analyst at the chip and AI research firm SemiAnalysis, to help me dig into the story inside the numbers of the Anthropic IPO filing — a document known as an S-1. Anthropic just filed its S-1 to go public on the Nasdaq, and it could start trading as soon as next month at a valuation of anywhere between $1.5 trillion and $2 trillion. That would immediately make it one of the most valuable companies on the planet. But the price

Policy & DramaOpen story →

DOJ exposes “ghost students” using AI to complete courses, steal student loans

The US has charged five individuals as part of the first broad federal crackdown on “ghost students”—fraudsters who steal the identities of hundreds of Americans to pose as fake college students and steal student loans. In a press release Thursday, the Department of Justice said it had uncovered a large-scale scheme relying partly on AI. The DOJ seized more than $750,000 in cryptocurrency from “a network associated with Kenyan nationals" after data analysis helped flag likely fraudsters. Assista