The digital world is changing rapidly. Whilst we have spent years focusing on traditional search engine marketing and optimisation (SEM & SEO), a new paradigm is now coming to the fore: Agentic Web.
For businesses, this means that it is no longer just about being found by people via search engines, but by autonomous AI agents and automated systems.
Leading technology decision-makers such as Google, Microsoft, GoDaddy, Hugging Face, Nvidia, Salesforce, ServiceNow, Databricks, Snowflake, GitHub, and Cisco have launched a new, open standard for this: the Agentic Resource Discovery (ARD) specification.
It remains to be seen whether, and to what extent, this standard will become widely adopted. One thing is certain, however: anyone who ignores this protocol risks becoming invisible in the curated responses and transactions of AI systems. The risk is disproportionate to the relatively low cost of implementation.
What is Agentic Resource Discovery (ARD)?
Until now, the internet of AI systems has been based primarily on manual integrations. If you wanted to use an AI agent or an LLM, you had to define in advance which APIs or tools (e.g. via the Model Context Protocol – MCP) were permitted to be accessed. This is static and does not scale.
Agentic Resource Discovery (ARD) fundamentally solves this problem. ARD is an open, decentralised protocol – comparable to the DNS system of the traditional internet – which enables AI agents to autonomously search for, find, and verify relevant resources, APIs, and specialised capabilities at runtime.
Instead of developers having to manually hard-code every API, the AI agent searches a federated directory for the best capability based on the user’s intent, verifies the source, and executes the action directly.
The difference in practice:
- Traditional approach (Pre-ARD): An AI assistant can only use the tools that the developer has hard-coded into the system.
- Agentic-Ready approach (ARD): A customer asks their AI agent: “Find me the most cost-effective cyber insurance for our vehicle fleet, compare the levels of cover, and prepare the digital policy.” or “Find me a local service provider who can repair my heating system today and book an appointment.” The agent searches the ARD directory, locates your company’s published booking API, verifies your domain, and completes the process.
Technical Functionality: Catalogues & Registries
The ARD architecture is based on two key pillars that define the trust and search model of the agentic web:
- Catalogues: Each company publishes a machine-readable manifest file called ai-catalog.json directly on its own domain under a standardised, protected path. As this file is hosted on your own web server, ownership of the domain serves as the cryptographic foundation for trust and identity.
- Registries: Registries function like search engines for AI agents. They crawl the web for published ai-catalog.json files, index their contents semantically, and make them searchable via a standardised API (e.g. POST /search) for querying AI clients.
Quelle: https://developers.googleblog.com/announcing-the-agentic-resource-discovery-specification/
The Structure of ai-catalog.json: The Technical Data Model
To ensure that registries and agents can interpret your interfaces correctly, the ai-catalog.json file must be structured exactly in accordance with the ARD specification.
A critical stumbling block in practice is the catalogue’s ‘Strict Value-or-Reference’ condition: For its technical definition, an entry must use either the url field (a reference to an external interface definition) or the data field (direct embedding of the schema)–never both at the same time.
Example of a minimal, valid ai-catalog.json for an Antic Web Readiness Checker:
JSON
{
“specVersion”: “1.0”,
“host”: {
“displayName”: “Digital Loop”,
“identifier”: “did:web:digital-loop.com”
},
“entries”: [
{
“identifier”: “urn:ai:digital-loop.com:services:agentic-readiness-check”,
“displayName”: “Agentic Web Readiness Checker”,
“type”: “application/mcp-server+json”,
“url”: “https://api.digital-loop.com/mcp/agentic-check”,
“description”: “Analysiert Webseiten auf ihre Maschinenlesbarkeit (Machine-Readiness), ARD-Kompatibilität, strukturierte Metadaten (Schema.org) sowie das Vorhandensein von KI-Schnittstellen-Protokollen wie MCP.”,
“representativeQueries”: [
“Überprüfe ob meine Website für KI-Agenten lesbar ist”,
“Scanne meine Domain nach einer ai-catalog.json und ARD-Schnittstellen”,
“Analysiere die Machine-Readiness meiner B2B-Plattform”
],
“tags”: [“agentic-web”, “ard”, “machine-readiness”, “mcp”, “seo”]
}
]
}
Key fields explained:
- identifier (Entry level): A globally unique, domain-anchored URN following the schema urn:ai:<domain>:<namespace>:<name>. This ensures that your interface remains uniquely identifiable across all global registries without any name collisions.
- type: The official media type of the resource (e.g. application/mcp-server+json for MCP servers or application/a2a-agent-card+json for agent-to-agent interfaces).
- representativeQueries: The strategic core for marketers. This is where 2 to 5 natural-language questions are defined, which describe exactly which tasks your tool can help with. Registries use these phrases as their primary search and ranking criteria.
How Do We Make Our Catalogue Visible on the Web?
To ensure your catalogue is crawled by the registries of major technology platforms (such as Google’s Gemini Enterprise Agent Platform or Microsoft’s Copilot), the ARD specification defines three complementary signal channel mechanisms:
Discovery Mechanism | Technical Implementation | Strategic Function |
Well-Known URI | The JSON file is available at the following exact location: https://domain.com/.well-known/ai-catalog.json [cite: 12] | Der primäre, standardisierte Abrufpfad für vertrauenswürdige Registry-Crawler und Partner-Agenten. [cite: 11] |
Robots.txt Directive | Add the following line: Agentmap: https://domain.com/.well-known/ai-catalog.json in deiner robots.txt [cite: 12] | Allows compliant crawlers to locate the catalogue immediately, without having to go via the homepage. [cite: 8] |
HTML Header Link | Integration of: <link rel=”ai-catalog” href=”https://domain.com/.well-known/ai-catalog.json”> im HTML-<head> [cite: 12] | Indicates to web crawlers, during normal indexing of the website, that your agent catalogue exists. [cite: 8] |
Governance & Operationalisation: To-Do List for Businesses
In practice, AI implementations rarely fail because of the technology itself, but rather due to a lack of structure, unclear responsibilities, and poor data discipline. When autonomous systems crawl your data to make purchasing decisions, that data must be error-free and protected. Every forward-looking business, therefore, needs clear ARD governance.
As technically savvy SEO managers or GAIO/AEO (Generative AI Optimisation / Answer Engine Optimisation) managers are thoroughly familiar with how search systems and indexing typically work, they play a central role here.
Strategic Allocation of Roles:
- GAIO (Generative AI Optimisation) Manager or SEO Manager: They have strategic ownership. They define the technical requirements, check the relevance of the representativeQueries and manage visibility in the search results.
- IT & Interface Teams: They are responsible for ensuring the infrastructure is deployed without errors.
Your ARD Checklist for Implementation:
- [ ] Set up a cross-functional task force: Bring marketing (GAIO/SEO), IT, and, where applicable, Legal/Compliance together around one table.
- [ ] Carry out an interface inventory: Record all web interfaces, MCP servers, customer-relevant APIs, and support bots existing within the company.
- [ ] Classify & select endpoints: Strictly define which APIs and Agentic endpoints are to be available to end customers and their agents. Private or sensitive internal core systems are consistently excluded.
- [ ] Programme ai-catalog.json: Generate the manifest file in compliance with the RFC-8141 URN schema and the Strict Value or Reference rule.
- [ ] Semantic keyword optimisation: Optimise the representativeQueries in the catalogue so that they perfectly match the typical search intents of modern AI agents.
- [ ] S Set up security & trust manifest: Include cryptographic proofs (such as DIDs or SPIFFE IDs) in the optional trustManifest block of your catalogue to provide unequivocal proof of your organisation’s authenticity.
- [ ] CORS & server configuration: Host the file statically under the well-known URI without any upstream authentication barriers or bot blocks.
- [ ] Enable signals: Implement the robots.txt directive (Agentmap) and the HTML header link.
- [ ] Establish continuous monitoring: Set up automated CI checks to continuously monitor the availability of your catalogue and the functionality of the underlying APIs.
Download now: ARD Implementierungs Checkliste
FAQ – Frequently Asked Questions about ARD
Does ARD replace traditional SEO?
No, it complements it. SEO optimises for human users who click on search results via visual interfaces. ARD optimises for autonomous AI systems that independently search for services, data and transaction options in the background. Both disciplines must be used in a complementary manner.
Why is the Model Context Protocol (MCP) not sufficient on its own?
MCP standardises how an agent and a server communicate with each other. However, it does not provide a mechanism through which an agent can dynamically determine in advance which servers or services actually exist on the web. ARD is the discovery layer (the phone book) that precedes the actual MCP connection establishment.
Do we have to make all our APIs public for the catalogue?
Absolutely not. Within the framework of your internal governance, you decide precisely which specific Agentic and API endpoints are made available to end customers. Only these are listed in the public catalogue. Confidential company data remains behind traditional IT security barriers.
These are typically services and interfaces that end users already access via a web UI (e.g. product searches or online calculators).
What happens if we do not implement ARD?
As AI assistants and ‘zero-click scenarios’ increasingly provide direct answers or process transactions autonomously in the background, traditional website traffic is declining.
If your offerings and interfaces for data agents cannot be found via ARD in a machine-readable format, your brand simply does not exist for these systems. You lose visibility on the modern web.
About the author:John Muñoz is the founder of Digital Loop and ensures that marketing and IT speak the same language. As a specialist MarTech company, we support organisations in strategically and successfully operationalising holistic and future-proof AI and technology solutions.
Next step: Would you like to ensure your brand’s visibility on the agentic web? Download our full ARD implementation checklist here or contact us for a strategic GAIO consultation.
Sources:
- https://drive.google.com/open?id=1sh3b536rlmzUyaHwfRJHZvi4-GlfBrhcP5z53EW5SSk
- https://drive.google.com/open?id=1gqXyGHeWnZrqyi76Ojg7JuWq3_RtUUASKYoM7zMDKRM
- How the Agentic Resource Discovery specification helps agents find each other at enterprise scale – Outshift | Cisco, https://outshift.cisco.com/blog/ai-ml/agentic-resource-discover-specification-helps-agents-find-each-other
- AI agents are getting their own search engine | ZDNET, https://www.zdnet.com/article/ai-agents-are-getting-their-own-search-engine/
- Introducing the Agentic Resource Discovery specification – Command Line – Microsoft, https://commandline.microsoft.com/agentic-resource-discovery-specification-ard/
- Agentic Resource Discovery: Let agents search – Hugging Face, https://huggingface.co/blog/agentic-resource-discovery-launch
- Announcing the Agentic Resource Discovery specification – Google Developers Blog, https://developers.googleblog.com/announcing-the-agentic-resource-discovery-specification/
- Agentic Resource Discovery (ARD) & ai-catalog.json: The Complete Guide – Synscribe, https://www.synscribe.com/agentic-discovery/agentic-resource-discovery
- How to Implement the ARD Specification: A Step-by-Step Guide for SaaS Companies, https://www.synscribe.com/blog/how-to-implement-ard-specification-saas
- urn-naming-guide.md – ards-project/ard-spec – GitHub, https://github.com/ards-project/ard-spec/blob/main/spec/urn-naming-guide.md
- GitHub and Google back ARD standard for AI agent discovery, https://www.developer-tech.com/news/github-google-ard-ai-agent-discovery/
- Google’s open standard for AI agents to discover and verify tools – Help Net Security, https://www.helpnetsecurity.com/2026/06/18/google-agentic-resource-discovery/
- Agent finder for GitHub Copilot now available, https://github.blog/changelog/2026-06-17-agent-finder-for-github-copilot-now-available/
