Bot traffic tracking

Analytics scripts run in the browser. Crawlers such as GPTBot, ChatGPT-User, or Googlebot download the HTML and leave without running JavaScript, so a tracking pixel never fires for them. The only place their visits show up is the request itself: at your server, your middleware, or your CDN.

Bot traffic tracking reads that request (URL, user agent, IP, and status code) and sends it to Rankealo. Your page is never delayed: every integration reports in the background.

1. 1

   Open Agent Analytics Setup

   In the app, go to Bot traffic → Agent Analytics Setup. Your site token and endpoint are shown there.
2. 2

   Pick your provider

   Choose how your site is served (Vercel, Cloudflare, Netlify, a framework such as Astro or SvelteKit, WordPress, or any backend) and follow the steps for it. We preselect the provider we detect.
3. 3

   Confirm data is flowing

   Press Check now. The first hit can take a while, since it depends on when a crawler next visits. Most sites see one within 24 hours.

Every hit is grouped into one of three categories, based on why the crawler came:

Humans and unknown user agents are dropped before anything is stored. Query strings are removed from URLs, and crawler IPs are stored only as a hash.

A user agent is a claim, not an identity: any client can send "GPTBot". By default we label a hit as a claimed identity (user agent only). When your CDN tells us the request came from a verified bot (for example Cloudflare's verified-bot flag, sent as ai.verifiedBot), the hit is upgraded to verified by CDN metadata. We never mark a hit as verified from the user agent alone.

Not supported yet: Amazon CloudFront, Fastly, Akamai, Google Cloud CDN, Shopify, and Adobe Experience Manager. Hosted builders (Webflow, Squarespace, Framer, Shopify) can use the Cloudflare Worker when the domain is proxied through Cloudflare.

Every integration calls the same endpoint. You can call it directly from any backend.

POST https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest
Authorization: Bearer YOUR_SITE_TOKEN
Content-Type: application/json

{
  "websiteId": "YOUR_WEBSITE_ID",
  "domain": "www.your-site.com",
  "href": "https://www.your-site.com/pricing",
  "ai": {
    "userAgent": "Mozilla/5.0 (compatible; ChatGPT-User/1.0; +https://openai.com/bot)",
    "ip": "203.0.113.8",
    "statusCode": 200
  }
}

Authentication. Send your site token as Authorization: Bearer YOUR_SITE_TOKEN. If the sender cannot set headers (Cloudflare Logpush, Vercel Log Drain), pass it as ?token=YOUR_SITE_TOKEN instead, with ?websiteId=YOUR_WEBSITE_ID. The token is tied to one website: it only works with that site's websiteId.

Batches. Send up to 500 hits per request in a hits array. Each item takes href, user_agent, and optionally ip, statusCode, verified_bot, and occurred_at (ISO 8601; defaults to the time we receive it).

{
  "websiteId": "YOUR_WEBSITE_ID",
  "domain": "www.your-site.com",
  "hits": [
    { "href": "https://www.your-site.com/", "user_agent": "GPTBot/1.2", "ip": "203.0.113.8", "occurred_at": "2026-09-25T10:00:00Z" },
    { "href": "https://www.your-site.com/blog", "user_agent": "Googlebot/2.1", "statusCode": 200 }
  ]
}

Responses. skipped counts hits from another host or with an unknown user agent.

200 { "ingested": 2, "skipped": 0, "truncated": false }
400 { "error": "websiteId is required" }        // or invalid JSON, or more than 500 hits
401 { "error": "Unauthorized" }                 // missing or wrong token
404 { "error": "onboarding_not_found" }         // unknown websiteId
503 { "error": "AI crawler ingest is disabled" }

Open Bot traffic in the app (/bot-traffic). It shows hits over time by crawler, splits them into AI answers, indexing, and training, and lists the pages each crawler read and whether AI engines cited them.