AI Search • Nov 2, 2026 • 5 min read

AI Training Control for Websites: What Exporters Should Do

Google, Bing and other AI providers are moving toward one shared way to say "do not train AI on my content": a policy token in robots.txt. For an exporter, the practical answer is simple. Decide whether your product pages should be used for AI training, write that choice into robots.txt once the support goes live, and do not confuse it with a block on AI search visibility.

01What was announced?

Documented

According to Search Engine Roundtable's report of Google's announcement, Google will support a shared robots.txt signal for AI training preferences. The signal is based on work from the IETF AIPREF working group. Google says it intends to support it in early 2027, and the report notes that Bing (Microsoft), OpenAI and Anthropic have taken similar steps.

Documented

The IETF draft (published October 30, 2026 and marked as work in progress) defines the signal. A site writes ordinary robots.txt groups keyed to the policy token __usage_ai-training. A Disallow expresses a preference against AI training for a path, and an Allow expresses a preference in favor. The draft says it "introduces no new directive names and no new grammar."

code

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

This is the shape of the draft's own example: block training everywhere except the press kit. Treat it as an illustration, not a recommended setting for your site.

02What does the token not do?

Documented

The draft states that the token is not a crawling directive. Usage preferences and crawl rules are independent, so you can still limit crawling with crawler-specific groups. A wildcard group (User-Agent: *) does not count as expressing the preference. If your file has no __usage_ai-training group, no preference is expressed. "Training" in the draft means using content to change the learned parameters of a generative AI model; classification, ranking and scoring models are excluded.

Google's own documentation of its site controls lists a separate site-wide setting in Search Console to manage inclusion in generative AI features on Google Search, including AI Overviews and AI Mode. The new token is a different control from that one.

03How does it compare with what Bing announced?

Documented

Per Search Engine Roundtable, Microsoft announced Bing controls that separate AI training from AI grounding (using your content to help generate answers). Publishers can choose training, grounding or both while remaining eligible for Bing search results. Reported dates: December 15, 2026 for temporary grounding controls in Bing Webmaster Tools, February 1, 2027 when Microsoft starts honoring the new preferences, and March 1, 2027 as the deadline for publishers using older directives to configure them.

Question	Training preference	Grounding / AI answers
What it controls	Whether content is used to train models	Whether content helps generate AI answers
Reported Bing option	AI Training	AI Grounding, Limited Grounding
Effect on normal rankings	Reported as none	Reported as none for traditional Bing rankings
Risk for exporters	Low for most B2B sites	Blocking can reduce visibility in AI experiences

04Should an exporter block AI training?

Analysis

There is no single right answer, but the trade-off is easy to frame. Your product specifications, certifications and application notes exist to be found and cited by buyers. Training and citation are different uses, yet the reporting suggests that blocking grounding can reduce AI visibility. A manufacturer that wants to appear in AI answers should be careful before blocking anything beyond training, and should treat a training opt-out as a business decision, not an SEO step.

Hypothesis

Many exporters will leave training open for public product pages and restrict only unique assets, such as private price lists, drawings or customer-only documents. We have not tested how each provider will treat mixed Allow and Disallow rules, so confirm behavior once the providers publish their documentation.

05A checklist for this week

Open your robots.txt and note which crawlers you already block or allow, including Google-Extended.
Decide per directory: public product pages, blog, downloads, private areas.
Do not add the new token yet. Support is planned for 2027, and the IETF draft may change.
Keep important pages crawlable and indexed. Our guide on llms.txt and AI-citation-friendly content explains why access comes before any AI-specific file.
If your site runs in several languages, make sure one robots.txt policy covers all language folders. Our website development service includes this kind of technical setup.

For the bigger picture of how AI answers choose sources, read GEO vs SEO: the differences that matter.

06Frequently asked questions

Will blocking AI training hurt my Google rankings?

Documented

Per the Roundtable report, sites that explicitly allow Google-Extended remain eligible to appear in Gemini Apps responses, and company-specific controls keep working alongside the shared signal. Microsoft reports that traditional Bing rankings are unaffected by these preferences. We found no source saying a training opt-out changes rankings, but check each provider's final documentation.