Cloudflare Logpush

• An Enterprise zone on Cloudflare.
• A Cloudflare API token with Logs: Editpermission on the zone, and the zone id (on the zone's Overview page).
• Your site token and website id from Bot traffic → Agent Analytics Setup in the app. The snippet there already has both filled in; the examples on this page use YOUR_SITE_TOKEN and YOUR_WEBSITE_ID. See the HTTP API reference for the full payload.

1. 1

   Copy the destination URL

   Logpush cannot always send an Authorization header, so the destination URL carries your website id and token as query parameters:

   url

   Copy

   https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest?websiteId=YOUR_WEBSITE_ID&token=YOUR_SITE_TOKEN

2. 2

   Create the Logpush job

   Run the command below with your zone id, API token, and hostname. It creates an http_requests job that sends NDJSON with only the fields Rankealo needs, filtered to your hostname. You can also create it in the dashboard under Analytics & Logs → Logpush → HTTP destination using the same URL and fields.
3. 3

   Wait for validation

   Cloudflare first sends a small test file. The endpoint accepts it and the job is created. Batches then arrive every few minutes.

curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{
  "name": "rankealo-bot-traffic",
  "dataset": "http_requests",
  "destination_conf": "https://app.rankealo.ai/api/ai-visibility/crawler-hits/ingest?websiteId=YOUR_WEBSITE_ID&token=YOUR_SITE_TOKEN",
  "output_options": {
    "output_type": "ndjson",
    "field_names": [
      "ClientRequestMethod",
      "ClientRequestHost",
      "ClientRequestURI",
      "ClientRequestUserAgent",
      "ClientIP",
      "EdgeResponseStatus",
      "EdgeStartTimestamp"
    ]
  },
  "filter": "{\"where\":{\"and\":[{\"key\":\"ClientRequestHost\",\"operator\":\"eq\",\"value\":\"www.your-site.com\"}]}}",
  "max_upload_records": 1000,
  "enabled": true
}'

• ClientRequestMethodOnly GET and HEAD are kept.
• ClientRequestHostMust belong to your site.
• ClientRequestURIPath; the query string is dropped.
• ClientRequestUserAgentIdentifies the crawler.
• ClientIPHashed on arrival, never stored raw.
• EdgeResponseStatusStatus code Cloudflare returned.
• EdgeStartTimestampWhen the request happened.

Batches may be gzip-compressed; Rankealo decompresses them. Humans and unknown user agents are dropped before anything is stored, and raw log lines are not kept.

• Enterprise zones only.
• Each batch keeps up to 500 crawler hits. The job above caps batches at 1,000 log lines (max_upload_records), which keeps normal traffic well inside that.
• The token sits in the destination URL, so treat the Logpush job config like a secret. If it leaks, contact support to rotate it.
• Hits are marked as claimed identities; Logpush does not forward Cloudflare's verified-bot flag in this setup.

1. 1
   In Rankealo, open Bot traffic → Agent Analytics Setup.
2. 2
   Press Check now in step 3. Once a crawler has visited, it shows Receiving data with the time of the last hit.
3. 3
   Open the Bot traffic tab to see hits by crawler and by page.

The first hit depends on when a crawler next visits, which can take up to 24 hours. To test right away, request a page with a crawler user agent, for example curl -A "GPTBot/1.2" https://www.your-site.com/, then press Check now. A test like this is recorded as a claimed identity, like any hit without CDN verification.