Prerequisites#

An app that renders pages on a server you deploy (not a fully static export).
Your site token and website id from Bot traffic → Agent Analytics Setup in the app. The snippet there already has both filled in; the examples on this page use YOUR_SITE_TOKEN and YOUR_WEBSITE_ID. See the HTTP API reference for the full payload.

How the snippets work#

Every snippet does the same thing, in the idiom of its framework:

Only GET and HEAD requests whose user agent matches a known AI or search crawler are reported.
Static assets (scripts, styles, images, fonts) are skipped; robots.txt, sitemaps, and llms.txt are kept.
The report is a single event sent with your site token as a Bearer header, the same payload as the HTTP API.
The report is never awaited, so the page is never delayed and a Rankealo outage cannot break your site.

Serverless hosts

On serverless or edge runtimes, a request can be frozen as soon as the response is sent. Where the framework exposes waitUntil (Nuxt, and the Cloudflare adapters for Astro and SvelteKit), the snippet uses it. Elsewhere a few reports may be lost on cold, short-lived functions; if you host on Vercel or Netlify, the Vercel Log Drain or Netlify Edge Function counts every visit.

Astro#

1
Create the file
src/middleware.ts. If you already have middleware, chain both with sequence() from astro:middleware.
2
Paste the snippet
Use the snippet from Agent Analytics Setup (it has your token filled in), or the one below with your values.
3
Deploy
Deploy as usual, then confirm data is flowing.

ts

// src/middleware.ts
import { defineMiddleware } from "astro:middleware";

const CRAWLER_UA = /gptbot|chatgpt-user|oai-searchbot|oai-adsbot|claudebot|claude-searchbot|claude-user|anthropic-ai|perplexitybot|perplexity-user|google-extended|googlebot|googleother|bingbot|duckassistbot|applebot|meta-externalagent|amazonbot|bytespider|ccbot|mistralai-user/i;
const STATIC_ASSET = /\.(?:js|css|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf)$/i;

export const onRequest = defineMiddleware(async (context, next) => {
  const response = await next();
  const { request, url } = context;
  const ua = request.headers.get("user-agent") || "";
  if (
    (request.method === "GET" || request.method === "HEAD") &&
    !STATIC_ASSET.test(url.pathname) &&
    CRAWLER_UA.test(ua)
  ) {
    let ip: string | null = null;
    try {
      ip = context.clientAddress;
    } catch {
      // not available on every adapter
    }
    const report = fetch("https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest", {
      method: "POST",
      headers: { "content-type": "application/json", authorization: "Bearer YOUR_SITE_TOKEN" },
      body: JSON.stringify({
        websiteId: "YOUR_WEBSITE_ID",
        domain: url.hostname,
        href: url.origin + url.pathname,
        ai: {
          userAgent: ua,
          ip: ip,
          statusCode: response.status,
          source: "server_middleware",
        },
      }),
    }).catch(() => {});
    // Cloudflare adapter: keep the request alive until the report is sent.
    (context.locals as any)?.runtime?.ctx?.waitUntil?.(report);
  }
  return response;
});

Astro middleware only runs for on-demand (server-rendered) routes, so you need an adapter and output: 'server' or pages with export const prerender = false. Prerendered pages are served straight from the CDN and never reach the middleware. For a static Astro site, use Netlify, the Vercel Log Drain, or the Cloudflare Worker instead.

Remix / React Router#

1
Create the file
app/rankealo-bots.server.ts, plus one line at the top of handleRequest in app/entry.server.tsx. Run npx remix reveal (Remix) or npx react-router reveal (React Router 7) if you do not have an entry.server file yet.
2
Paste the snippet
Use the snippet from Agent Analytics Setup (it has your token filled in), or the one below with your values.
3
Deploy
Deploy as usual, then confirm data is flowing.

ts

// app/rankealo-bots.server.ts
const CRAWLER_UA = /gptbot|chatgpt-user|oai-searchbot|oai-adsbot|claudebot|claude-searchbot|claude-user|anthropic-ai|perplexitybot|perplexity-user|google-extended|googlebot|googleother|bingbot|duckassistbot|applebot|meta-externalagent|amazonbot|bytespider|ccbot|mistralai-user/i;
const STATIC_ASSET = /\.(?:js|css|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf)$/i;

/** Report a crawler request to Rankealo in the background. Never throws, never awaited. */
export function reportCrawler(request: Request, status: number) {
  const url = new URL(request.url);
  const ua = request.headers.get("user-agent") || "";
  if (request.method !== "GET" && request.method !== "HEAD") return;
  if (STATIC_ASSET.test(url.pathname) || !CRAWLER_UA.test(ua)) return;
  void fetch("https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest", {
    method: "POST",
    headers: { "content-type": "application/json", authorization: "Bearer YOUR_SITE_TOKEN" },
    body: JSON.stringify({
      websiteId: "YOUR_WEBSITE_ID",
      domain: url.hostname,
      href: url.origin + url.pathname,
      ai: {
        userAgent: ua,
        ip: request.headers.get("x-forwarded-for"),
        statusCode: status,
        source: "server_middleware",
      },
    }),
  }).catch(() => {});
}

// app/entry.server.tsx: add one line at the top of handleRequest
import { reportCrawler } from "./rankealo-bots.server";

export default function handleRequest(
  request: Request,
  responseStatusCode: number,
  // ...the rest of your existing parameters
) {
  reportCrawler(request, responseStatusCode);
  // ...your existing rendering code, unchanged
}

handleRequest runs for every document request, and it already knows the status code. Data requests from client-side navigation do not go through it, which is fine: crawlers load full documents. The IP comes from x-forwarded-for, which your host or proxy sets.

SvelteKit#

1
Create the file
src/hooks.server.ts. If you already export handle, combine both with sequence() from @sveltejs/kit/hooks.
2
Paste the snippet
Use the snippet from Agent Analytics Setup (it has your token filled in), or the one below with your values.
3
Deploy
Deploy as usual, then confirm data is flowing.

ts

// src/hooks.server.ts
import type { Handle } from "@sveltejs/kit";

const CRAWLER_UA = /gptbot|chatgpt-user|oai-searchbot|oai-adsbot|claudebot|claude-searchbot|claude-user|anthropic-ai|perplexitybot|perplexity-user|google-extended|googlebot|googleother|bingbot|duckassistbot|applebot|meta-externalagent|amazonbot|bytespider|ccbot|mistralai-user/i;
const STATIC_ASSET = /\.(?:js|css|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf)$/i;

export const handle: Handle = async ({ event, resolve }) => {
  const response = await resolve(event);
  const { request, url } = event;
  const ua = request.headers.get("user-agent") || "";
  if (
    (request.method === "GET" || request.method === "HEAD") &&
    !STATIC_ASSET.test(url.pathname) &&
    CRAWLER_UA.test(ua)
  ) {
    let ip: string | null = null;
    try {
      ip = event.getClientAddress();
    } catch {
      // not available on every adapter
    }
    const report = fetch("https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest", {
      method: "POST",
      headers: { "content-type": "application/json", authorization: "Bearer YOUR_SITE_TOKEN" },
      body: JSON.stringify({
        websiteId: "YOUR_WEBSITE_ID",
        domain: url.hostname,
        href: url.origin + url.pathname,
        ai: {
          userAgent: ua,
          ip: ip,
          statusCode: response.status,
          source: "server_middleware",
        },
      }),
    }).catch(() => {});
    // Cloudflare adapter: keep the request alive until the report is sent.
    (event.platform as any)?.context?.waitUntil?.(report);
  }
  return response;
};

The hook runs for server-rendered pages and endpoints. Pages prerendered at build time (export const prerender = true) are served as static files and are not seen; use your host's option (Netlify, Vercel Log Drain, Cloudflare) for those.

Nuxt#

1
Create the file
server/middleware/rankealo-bots.ts
2
Paste the snippet
Use the snippet from Agent Analytics Setup (it has your token filled in), or the one below with your values.
3
Deploy
Deploy as usual, then confirm data is flowing.

ts

// server/middleware/rankealo-bots.ts
const CRAWLER_UA = /gptbot|chatgpt-user|oai-searchbot|oai-adsbot|claudebot|claude-searchbot|claude-user|anthropic-ai|perplexitybot|perplexity-user|google-extended|googlebot|googleother|bingbot|duckassistbot|applebot|meta-externalagent|amazonbot|bytespider|ccbot|mistralai-user/i;
const STATIC_ASSET = /\.(?:js|css|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf)$/i;

export default defineEventHandler((event) => {
  if (event.method !== "GET" && event.method !== "HEAD") return;
  const url = getRequestURL(event);
  const ua = getRequestHeader(event, "user-agent") || "";
  if (STATIC_ASSET.test(url.pathname) || !CRAWLER_UA.test(ua)) return;

  const report = fetch("https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest", {
    method: "POST",
    headers: { "content-type": "application/json", authorization: "Bearer YOUR_SITE_TOKEN" },
    body: JSON.stringify({
      websiteId: "YOUR_WEBSITE_ID",
      domain: url.hostname,
      href: url.origin + url.pathname,
      ai: {
        userAgent: ua,
        ip: getRequestIP(event, { xForwardedFor: true }) || null,
        source: "server_middleware",
      },
    }),
  }).catch(() => {});
  // Background: keeps serverless/edge runtimes alive without delaying the page.
  if (typeof event.waitUntil === "function") event.waitUntil(report);
  // Return nothing so Nuxt keeps handling the request.
});

Nuxt runs server middleware before every request, so no status code is sent. The handler returns nothing, which lets Nuxt render the page as usual. Pages generated with nuxi generate or prerendered routes are served as static files and are not seen.

Express#

1
Create the file
rankealo-bots.js, registered with app.use() before your routes and express.static.
2
Paste the snippet
Use the snippet from Agent Analytics Setup (it has your token filled in), or the one below with your values.
3
Deploy
Deploy as usual, then confirm data is flowing.

js

// rankealo-bots.js: register before your routes and static middleware
const CRAWLER_UA = /gptbot|chatgpt-user|oai-searchbot|oai-adsbot|claudebot|claude-searchbot|claude-user|anthropic-ai|perplexitybot|perplexity-user|google-extended|googlebot|googleother|bingbot|duckassistbot|applebot|meta-externalagent|amazonbot|bytespider|ccbot|mistralai-user/i;
const STATIC_ASSET = /\.(?:js|css|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf)$/i;

function rankealoBots(req, res, next) {
  const ua = req.get("user-agent") || "";
  if (
    (req.method === "GET" || req.method === "HEAD") &&
    !STATIC_ASSET.test(req.path) &&
    CRAWLER_UA.test(ua)
  ) {
    // Report after the response is sent, so the crawler is never delayed.
    res.on("finish", () => {
      fetch("https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest", {
        method: "POST",
        headers: { "content-type": "application/json", authorization: "Bearer YOUR_SITE_TOKEN" },
        body: JSON.stringify({
          websiteId: "YOUR_WEBSITE_ID",
          domain: req.hostname,
          href: req.protocol + "://" + req.get("host") + req.path,
          ai: {
            userAgent: ua,
            ip: req.ip || null,
            statusCode: res.statusCode,
            source: "server_middleware",
          },
        }),
      }).catch(() => {});
    });
  }
  next();
}

module.exports = rankealoBots;

// app.js
// app.use(require("./rankealo-bots"));

The report is sent on the response's finish event, after the crawler already has its page, and includes the status code. Needs Node.js 18 or newer for the global fetch. Behind a proxy or load balancer, set app.set("trust proxy", true) so req.ipis the crawler's address.

What is sent#

URLOrigin and path. The query string is never sent.
User agentUsed to identify the crawler.
IPHashed on arrival, never stored raw.
Status codeWhen the framework knows it (all except Nuxt).
Website idTies the hit to your site and token.

Confirm data is flowing#

1
In Rankealo, open Bot traffic → Agent Analytics Setup.
2
Press Check now in step 3. Once a crawler has visited, it shows Receiving data with the time of the last hit.
3
Open the Bot traffic tab to see hits by crawler and by page.

The first hit depends on when a crawler next visits, which can take up to 24 hours. To test right away, request a page with a crawler user agent, for example curl -A "GPTBot/1.2" https://www.your-site.com/, then press Check now. A test like this is recorded as a claimed identity, like any hit without CDN verification.

Troubleshooting#

Nothing shows up after deploying+

Most often the page was prerendered and never reached the server. Request a server-rendered URL with curl -A "GPTBot/1.2" and press Check now.

Server-rendered pages still show nothing+

The snippet ignores ingest errors on purpose, so a wrong token fails silently. Compare the token and website id in the snippet with the ones in Agent Analytics Setup.

Hits are skipped+

The hostname must belong to the website you set up in Rankealo. Localhost and preview hostnames are ignored on purpose.

Frameworks