Skip to main content
Setting up AI Traffic means choosing the sensor that matches each hostname. This page covers the JavaScript snippet for AI referral traffic, server-side crawler sensors for Cloudflare and edge middleware deployments, and optional first-party proxy routing when your deployment or network setup requires it.

​
Where Setup Lives

AI Traffic setup now lives inside Project Settings > AI Traffic. To generate the snippet:
  1. Open your project
  2. Go to Project Settings
  3. Open the AI Traffic tab
  4. Click Generate Tracking Snippet
DevTune generates a unique snippet key for your project. The snippet code appears on the settings page, ready to copy. Cloudflare and edge middleware crawler sensors are configured in the same AI Traffic tab when those controls are available for your account.

​
Install the Snippet

Copy the snippet and add it to your website’s layout or template file so it loads on every page. The snippet should be placed before the closing </body> tag.

​
GitBook

If your documentation is hosted on GitBook, you can use the official DevTune integration instead of manually installing the snippet:
  1. Go to your GitBook space settings
  2. Navigate to Integrations
  3. Find and install the DevTune integration from the marketplace
  4. Enter your project snippet key when prompted
The integration will automatically add the tracking snippet to all your GitBook pages. Before enabling it, make sure your consent flow and any region-aware handling are configured so tracking only starts after the required opt-in. View the integration on GitBook.

​
Next.js

Add the snippet to your root layout file (app/layout.tsx or pages/_document.tsx): 
import Script from 'next/script';

export default function RootLayout({ children }) {
  return (
    <html>
      <body>
        {children}
        <Script
          id="devtune-tracking"
          strategy="afterInteractive"
          dangerouslySetInnerHTML={{
            __html: `PASTE_YOUR_SNIPPET_HERE`,
          }}
        />
      </body>
    </html>
  );
}

​
WordPress

Add the snippet to your theme’s footer.php file, or use a plugin such as “Insert Headers and Footers” to add it to the footer section.

​
Webflow / Squarespace

Go to Site Settings, then Custom Code, and paste the snippet in the Footer section.

​
Plain HTML

Add a <script> tag before the closing </body> tag in your HTML template: 
<script>
  PASTE_YOUR_SNIPPET_HERE;
</script>

​
Enable or Disable Tracking

On the Project Settings AI Traffic tab, use the toggle switch to enable or disable tracking at any time. When disabled, incoming events are rejected and no new data is recorded.

​
Rate Limits

Each tracking snippet is rate-limited to 1,000 requests per minute. This is enough for most sites. If your site exceeds this limit, excess requests are silently dropped.

​
Crawler Sensor Capabilities

AI crawlers and answer fetchers often do not run JavaScript, so DevTune also supports server-side crawler sensors. Use one server-side crawler sensor per hostname to avoid double-counting.
CapabilityCloudflare pullEdge middleware
Best forHostnames already proxied by Cloudflare, including docs or static hosts that cannot run app middleware.Vercel, Next.js, or self-hosted apps where request middleware can run.
SetupZone ID, readonly API token, and an orange-cloud proxied hostname.Server-side ingest key plus @devtune/ai-traffic in your proxy or middleware.
Data flowDevTune pulls hourly Cloudflare analytics buckets.Your edge code pushes matched AI bot requests asynchronously.
FreshnessHourly sync after Cloudflare analytics are available.Near real-time push into DevTune’s ingest queue.
GranularityHourly aggregate buckets by page, user agent, bot class, and status.One event per matched request with URL, path, user agent, bot class, and optional status.
HistoryCan backfill the recent Cloudflare analytics window available to the zone.Starts collecting only after middleware is installed and deployed.
Status codesReal edge status classes for proxied traffic, including 404s.Known for redirects and denials returned by middleware. Normal page responses are status unknown unless you add route wrappers or known 404 path patterns.
CoverageOnly traffic that passes through the orange-cloud hostname.Only routes reached by your app middleware.
AvoidUsing Cloudflare and middleware on the same hostname.Double-proxying an app through Cloudflare just to collect traffic.
Recommended split:
  • Use Cloudflare pull for docs, marketing, or static hostnames that already sit behind Cloudflare.
  • Use edge middleware for Vercel, Next.js, and app hostnames where you do not want a Cloudflare-to-app double proxy.
  • Keep a hostname on one crawler sensor at a time. If you move a hostname from Cloudflare pull to middleware, remove or pause the Cloudflare hostname connection.

​
Edge Middleware Setup

Install @devtune/ai-traffic, keep your ingest key in a server-side environment variable, and call the package only for likely machine user agents. Keep the prefilter broad enough for future crawler and fetcher names; the package still performs the authoritative registry match before ingesting an event, but the cheap prefilter keeps normal browser traffic out of the middleware tracking path. Set batchSize: 1, flushIntervalMs: 0, and waitUntilRegistryRefresh: false in middleware so background ingest work is scheduled immediately, and hourly registry refreshes do not extend middleware waitUntil duration. 
// proxy.ts (Next.js 16; use middleware.ts/middleware() on older Next.js)
import { createDevTuneAiTraffic } from '@devtune/ai-traffic';
import {
  NextResponse,
  type NextFetchEvent,
  type NextRequest,
} from 'next/server';

const aiTraffic = createDevTuneAiTraffic({
  ingestKey: process.env.DEVTUNE_AI_TRAFFIC_INGEST_KEY!,
  batchSize: 1,
  defaultStatus: null,
  flushIntervalMs: 0,
  waitUntilRegistryRefresh: false,
});

const aiTrafficMachineUserAgentHints = [
  'bot',
  'crawler',
  'spider',
  'fetch',
  'agent',
  'gpt',
  'claude',
  'anthropic',
  'perplexity',
  'openai',
  'notebooklm',
  'google-extended',
  'bytespider',
  'meta-external',
  '-ai',
  'ai-',
];

function isLikelyMachineTraffic(request: NextRequest) {
  const userAgent = request.headers.get('user-agent')?.toLowerCase();

  return aiTrafficMachineUserAgentHints.some((hint) =>
    userAgent?.includes(hint),
  );
}

export function proxy(request: NextRequest, event: NextFetchEvent) {
  const response = NextResponse.next();

  if (isLikelyMachineTraffic(request)) {
    aiTraffic.trackRequest(request, event, null);
  }

  return response;
}
When your middleware returns a redirect, denial, or custom response, pass response.status instead of null. For normal pass-through page requests, keep null because the middleware cannot observe the final page status.

​
Proxy Setup

You can proxy the tracking endpoint through your own domain when you need first-party routing for your deployment setup. Teams should still honor consent requirements and local privacy rules before collecting traffic data.

​
How It Works

Instead of the snippet sending data to https://devtune.ai/api/v1/llm-traffic/collect, you configure a reverse proxy, so requests go to a path on your own domain.

​
Vercel / Next.js

Add a rewrite to next.config.ts: 
async rewrites() {
  return [
    {
      source: '/dt/:path*',
      destination:
        'https://devtune.ai/api/v1/llm-traffic/:path*',
    },
  ];
}

​
Netlify

Add to netlify.toml: 
[[redirects]]
  from = "/dt/*"
  to = "https://devtune.ai/api/v1/llm-traffic/:splat"
  status = 200
  force = true

​
Cloudflare

Create a redirect rule or Worker that proxies /dt/* to https://devtune.ai/api/v1/llm-traffic/*.

​
nginx

location /dt/ {
  proxy_pass https://devtune.ai/api/v1/llm-traffic/;
  proxy_set_header Host devtune.ai;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_set_header X-Forwarded-Host $host;
}
After setting up the proxy, update the beacon URL in your snippet to use the first-party path.

​
Multi-Site Tracking

You can use the same tracking snippet across multiple websites or subdomains. The dashboard automatically detects distinct domains and provides a domain filter so you can view traffic for each site individually or in aggregate.

​
Verifying Installation

After installing the snippet:
  1. Visit your website in a browser
  2. Open the DevTune AI Traffic dashboard
  3. You should see traffic appearing within a few minutes
If no data appears, check:
  • The snippet is present in your page source
  • The snippet key matches the one shown in Project Settings
  • Ad blockers are not blocking the request
  • Tracking is enabled on the AI Traffic tab

​
Next Steps