> For the complete documentation index, see [llms.txt](https://support.limy.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://support.limy.ai/monitoring/bot-monitor.md).

# Bot Monitor

Before AI systems such as ChatGPT, Gemini, or Perplexity mention a brand in an answer, they rely on automated bots to browse, fetch, and interpret web content. These bots decide which pages are **relevant, which information is credible, and how frequently content should be revisited.**

Bot Monitor gives you the visibility into this otherwise invisible process by showing which AI bots are **crawling your site, which pages they read, and how often they return**. This insight is critical for understanding how your content is ingested and refreshed inside AI systems.

<figure><img src="/files/BLdeXoECA9tQUmH4KCSb" alt=""><figcaption></figcaption></figure>

### Once enabled, Bot Monitor allows you to:

* Identify which AI bots and crawlers are accessing your website
* See which pages are most frequently ingested by AI systems
* Monitor crawl frequency and patterns over time
* Understand how AI systems refresh their understanding of your content
* Identify gaps where important pages are not being crawled

This helps you to focus on the pages and content that actually influence AI-generated answers, rather than relying on assumptions.

### How to Set Up Bot Monitor

Step 1: Identify your CDN provider. This information is typically available from your DevOps or infrastructure team. Common providers include:

{% stepper %}
{% step %}

#### Identify your CDN Provider

This information is typically available from your DevOps or infrastructure team. Common providers include:

* Cloudflare
* Akamai
* Fastly
* Netlify
* CloudFront
  {% endstep %}

{% step %}

#### Follow the CDN integration [instructions](https://docs.getlimy.ai/limy-pixel/cdn-integration).&#x20;

{% endstep %}

{% step %}

#### Confirm bot data appears in Limy

{% endstep %}
{% endstepper %}

### Once the integration is completed

{% stepper %}
{% step %}

#### Go to your **Limy** account

{% endstep %}

{% step %}

#### Nativate to **Bot** **Monitor**, under **Monitoring**

{% endstep %}

{% step %}

#### AI bot activity will begin appearing as crawlers revisit your site

{% endstep %}
{% endstepper %}

### FAQs

<details>

<summary><strong>Does Bot Monitor track users or personal data?</strong></summary>

No. Bot Monitor only tracks automated crawlers and agents. No user sessions or personal identifiable information (PII) are collected.

</details>

<details>

<summary><strong>Does this integration affect site performance?</strong></summary>

No. The integration is read-only and does not impact site speed or availability.

</details>

<details>

<summary><strong>Do I need engineering support to set this up?</strong></summary>

Yes. CDN access is typically managed by DevOps or IT teams.

</details>

<details>

<summary><strong>How long does it take to see data?</strong></summary>

Data appears as AI crawlers revisit your site. Timing depends on crawl frequency and AI platform behavior.

</details>

<details>

<summary><strong>What if I don’t see important pages being crawled?</strong></summary>

This may indicate crawlability or accessibility issues, which can be addressed through Recommendations or technical adjustments.

</details>

***

### Related Content

{% content-ref url="/pages/4jHQks7BiAkIdaIqZJnh" %}
[AI Traffic Monitor](/monitoring/ai-traffic-monitor.md)
{% endcontent-ref %}

{% content-ref url="/pages/TqFqtwNJ5aHW2VPaSdr9" %}
[Monitoring](/monitoring/monitoring.md)
{% endcontent-ref %}

{% content-ref url="/pages/5MMZ2BXUk6D9mEVfmg0g" %}
[How Limy Generates Prompts](/intelligence/how-limy-generates-prompts.md)
{% endcontent-ref %}

{% content-ref url="/pages/va37M8uK82qZl0pbxr4O" %}
[Attribution](/monitoring/attribution.md)
{% endcontent-ref %}

Still have questions? We’re here to help. Contact us at **<answers@limy.ai>** - we’d be happy to assist.