Enabling NLWeb for AI Agent Access
NLWeb (Natural Language Web) is an open-source framework from Microsoft that makes your website directly queryable by both people and AI agents. Instead of clicking through links and menus, your site visitors can ask AI agents, such as Gemini or ChatGPT, questions in natural language and get structured, context-aware answers from your own content.
After a one-time installation, NLWeb sets up an agent-based interaction layer on top of your public data. It lets AI agents have a conversation with your site, not just crawl it, by exposing an AI-friendly view of sources like your Schema.org markup, product feeds, and RSS. This helps agents and LLMs return more accurate conversational answers through the NLWeb WebApp connected to your site.
View an example for an e-commerce site
Customers can ask specific questions like, "show me waterproof hiking boots under $150 in a size 10" and receive instant, filtered results. AI shopping assistants can also compare products across specifications, check real-time inventory, and complete purchases on behalf of users.
Important:
NLWeb is separate from traditional search engine indexing and does not change how Google, Bing, or other search engines crawl or rank your site.
You must have an upgraded site and have a connected domain to use the NLWeb integration.
NLWeb currently only supports the main language of your site.
Installing NLWeb
Install NLWeb to give AI agent access to your site's structured content. After installation, the ASK protocol is switched on so AI agents that support NLWeb can query your public content.
To install NLWeb:
Go to SEO & GEO in your site's dashboard.
Scroll down to Tools and settings.
Click NLWeb.
Click Install NLWeb.
Note: Indexing may take a few minutes depending on the size of your site.
After installation:
NLWeb queries your site so it will be able to answer in a machine-readable format that AI agents can query.
The ASK protocol is automatically enabled, allowing supported AI agents to query your site's public content through natural language questions.
Managing AI Agent access
Use the ASK protocol setting in the NLWeb tool in the SEO dashboard to decide whether AI agents can query your indexed structured content. You can switch this access on or off at any time without changing your existing NLWeb connection.
To manage AI Agent access:
Go to SEO & GEO in your site's dashboard.
Scroll down to Tools and settings.
Click NLWeb.
Click the toggle next to Enable ASK protocol so AI tools can search and use your site's content:
Enabled: AI agents that support NLWeb can query your site's public structured content through natural language questions.
Disabled: Your site stays connected to NLWeb but is inactive for AI queries, so agents cannot use it to answer questions.
FAQs
Click a question below to learn more about NLWeb for AI Agent access.
What content is visible to NLWeb?
NLWeb can see any structured content on your site (such as content that has structured data markup) and treats it as a set of structured content. Learn more about adding structured data markup to your site's pages.
Can I remove my site from NLWeb after installation?
It is not currently possible to disconnect your site from NLWeb after installation, but you can disable the ASK protocol to prevent AI agents from querying your site.
Does NLWeb affect my site's SEO ranking?
No, NLWeb is separate from traditional search engine indexing with Google and Bing and does not change how your site is crawled, indexed, or ranked in search results.
What is the difference between NLWeb and search engine indexing?
Search engine indexing with Google, Bing, and others makes your site appear in traditional search results, while NLWeb makes your structured content accessible to AI agents for natural language queries as a separate system.
Will AI agents access my private content through NLWeb?
No, NLWeb respects your existing site visibility and access rules and only exposes content that is already publicly accessible, without sharing private data or user-specific information.
I have blocked AI crawlers in my robots.txt file, will NLWeb still work?
Blocking AI crawlers in your robots.txt file may limit NLWeb functionality, because AI agents need access to crawl and read your structured content. If you want AI agents to use NLWeb, make sure you have not blocked relevant AI crawlers in robots.txt.
I want to test how NLWeb works on my site. What do I do?
To test how NLWeb works on your site:
Make sure NLWeb is installed for your site.
Use the following MCP URL to connect to your favorite AI agent (e.g. Claude, ChatGPT or custom agents): https://www.[your-domain.com]/_api/nlweb/mcp.
Start querying your site.