> ## Documentation Index
> Fetch the complete documentation index at: https://docs.devtune.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Install edge middleware

> Record AI crawls, answer fetches and 404 demand from your Next.js, Express or Node app with @devtune/ai-traffic, using an ingest key or a linked Vercel project.

Use the middleware package for application servers and edge runtimes. It forwards only matched AI bot requests, so normal browser traffic is never sent to DevTune.

## Before you start

* A paid plan that includes AI Traffic, and project management permission. See [Set up AI Traffic](https://docs.devtune.ai/ai-traffic/setup.md#before-you-start).
* An app you can deploy with request middleware, such as Next.js, Express or Node.

## Generate the ingest key

1. In **Settings › Integrations**, open **Edge middleware** under **Crawler sensors**.
2. Select **Generate ingest key**.
3. Copy the **Server-side ingest key** into a server or edge environment variable as `DEVTUNE_AI_TRAFFIC_INGEST_KEY`. Never expose it in browser code.

On Vercel, you can [link the Vercel project](#vercel-oidc-without-an-ingest-key) instead.

## Install the package

```bash theme={null}
pnpm add @devtune/ai-traffic
```

Keep the ingest key in a server-side environment variable. Choose the adapter for the runtime that receives your application requests; every adapter performs the same package-owned crawler filtering before sending matched traffic.

For a single-origin Express or Node server, also set `DEVTUNE_AI_TRAFFIC_ORIGIN` to the application's canonical public origin, such as `https://www.example.com`. Pinning the origin prevents client-supplied forwarded headers from changing the hostname recorded in DevTune. If one process serves multiple tracked hostnames, omit the fixed origin only when a trusted proxy overwrites `X-Forwarded-Host` and `X-Forwarded-Proto`, or create a separate adapter instance with its own `origin` for each hostname.

### Next.js

For Next.js 16, add `proxy.ts`. For older Next.js projects, use the same body in `middleware.ts` and export `middleware()` instead.

```typescript theme={null}
// proxy.ts
import { createDevTuneAiTrafficMiddleware } from '@devtune/ai-traffic';
import {
  NextResponse,
  type NextFetchEvent,
  type NextRequest,
} from 'next/server';

const trackAiTraffic = createDevTuneAiTrafficMiddleware({
  ingestKey: process.env.DEVTUNE_AI_TRAFFIC_INGEST_KEY!,
});

export function proxy(request: NextRequest, event: NextFetchEvent) {
  trackAiTraffic(request, event);

  return NextResponse.next();
}
```

### Express

```typescript theme={null}
import express from 'express';
import { createDevTuneAiTrafficExpressMiddleware } from '@devtune/ai-traffic/express';

const app = express();

app.use(
  createDevTuneAiTrafficExpressMiddleware({
    ingestKey: process.env.DEVTUNE_AI_TRAFFIC_INGEST_KEY!,
    origin: process.env.DEVTUNE_AI_TRAFFIC_ORIGIN!,
  }),
);
```

The middleware calls `next()` immediately and records the actual response status after Express emits `finish`.

### Node HTTP

```typescript theme={null}
import { createServer } from 'node:http';

import { createDevTuneAiTrafficNodeHandler } from '@devtune/ai-traffic/node';

const trackAiTraffic = createDevTuneAiTrafficNodeHandler({
  ingestKey: process.env.DEVTUNE_AI_TRAFFIC_INGEST_KEY!,
  origin: process.env.DEVTUNE_AI_TRAFFIC_ORIGIN!,
});

createServer((request, response) => {
  trackAiTraffic(request, response);

  response.statusCode = 200;
  response.end('ok');
}).listen(3000);
```

The hook records the final `response.statusCode` without blocking or changing the response.

## Vercel OIDC without an ingest key

On Vercel, edge middleware can send traffic without `DEVTUNE_AI_TRAFFIC_INGEST_KEY`. Link the Vercel project in DevTune, and each deployment authenticates with the short-lived OIDC token Vercel issues to it.

You need:

* A site deployed on Vercel. Other hosts keep using the ingest key.
* `@devtune/ai-traffic` 0.11.0 or later.

### Link the Vercel project

In **Settings › Integrations**, open **Edge middleware** under **Crawler sensors**. Under **Vercel projects**:

1. Enter the **Vercel team ID**. In Vercel, it is in the team's **Settings › General**, under **Team ID**, and starts with `team_`.
2. Enter the **Vercel project ID**. It is in the project's **Settings › General**, under **Project ID**, and starts with `prj_`.
3. Select **Add Vercel project**.

The link works as soon as you add it. Anyone with project management permission can add or remove links. To send traffic from several Vercel projects, such as a docs site and a marketing site, add a link for each one.

### Configure the middleware

The same screen shows the **DevTune project ID**. Copy it into an environment variable as `DEVTUNE_AI_TRAFFIC_PROJECT_ID` in every linked Vercel project. The project ID is not a secret.

Pass it as `projectId` and leave out `ingestKey`:

```typescript theme={null}
// proxy.ts
import { createDevTuneAiTrafficMiddleware } from '@devtune/ai-traffic';
import {
  NextResponse,
  type NextFetchEvent,
  type NextRequest,
} from 'next/server';

const trackAiTraffic = createDevTuneAiTrafficMiddleware({
  projectId: process.env.DEVTUNE_AI_TRAFFIC_PROJECT_ID!,
});

export function proxy(request: NextRequest, event: NextFetchEvent) {
  trackAiTraffic(request, event);

  return NextResponse.next();
}
```

The Express and Node HTTP adapters take the same `projectId` option when they run on Vercel.

### Switch from an ingest key

You can keep `ingestKey` set while you switch. When both are set, the middleware sends with the Vercel token and uses the key whenever the token can't be used: Vercel issued no token, Vercel couldn't exchange it, DevTune rejected it, or DevTune couldn't check it.

```typescript theme={null}
const trackAiTraffic = createDevTuneAiTrafficMiddleware({
  projectId: process.env.DEVTUNE_AI_TRAFFIC_PROJECT_ID!,
  ingestKey: process.env.DEVTUNE_AI_TRAFFIC_INGEST_KEY,
});
```

Once the link shows **Last seen** with a time, remove `ingestKey` and `DEVTUNE_AI_TRAFFIC_INGEST_KEY` from that project and redeploy. The ingest key is shared by every site that sends traffic to this DevTune project, so retire it only after every one of them has stopped using it: each Vercel project has its own link showing **Last seen**, and no other host still sends with the key. Then retire the key in Settings: select **Rotate key**, then either **Revoke previous key** right away or leave the previous key to stop working after 24 hours. If **Rotate key** is unavailable, a previous key is still active, so first select **Revoke previous key** once no deployment uses it.

### Preview deployments

DevTune accepts Vercel-token batches from production deployments only. To accept a linked project's previews too, turn on **Preview deployments** on its link. Preview traffic then mixes into the same AI Traffic data as production. Turn it off to go back to production only. Deployments in a custom Vercel environment are never accepted with the Vercel token, even with **Preview deployments** on.

This setting covers only the Vercel token. A preview deployment that also has `ingestKey` set falls back to the key, and DevTune accepts the key from any deployment. To keep previews out while you still use the key, set `DEVTUNE_AI_TRAFFIC_INGEST_KEY` only for Vercel's Production environment. Do this only when `DEVTUNE_AI_TRAFFIC_PROJECT_ID` is set in every environment: a deployment with neither fails at startup.

### Link status

A new link shows **Not seen yet** until DevTune accepts its first batch. After that, it shows the Vercel team and project names and when the link was last seen.

If a link stays **Not seen yet** after a deploy that received crawler traffic:

1. Check both IDs on the link against Vercel. A mistyped ID never matches, and DevTune records nothing from that deployment. Remove the link and add it again with the right IDs.
2. Check that `projectId` in the middleware matches the **DevTune project ID** shown in Settings.
3. Check that **Snippet and middleware collection** is switched on in **Settings › Integrations**. While it is off, DevTune rejects every batch.
4. Check which Vercel environment received the traffic. Previews count only with **Preview deployments** on, and custom environments never count.

### Remove a link

Select **Remove** on a link to stop accepting that Vercel project's batches. If the middleware still has an ingest key configured, it falls back to the key. Otherwise its traffic stops reaching DevTune.

## Verifying crawler identity

A user agent is a claim, not an identity. The middleware sends the address each matched crawler request came from, so DevTune can check it against the operator's published networks and then discard it. Requests that can't be checked are kept with their identity unknown. See [crawler identity checks](https://docs.devtune.ai/ai-traffic/advanced-setup.md#crawler-identity-checks) for the operators DevTune can check and what each verdict means.

On Vercel, and on Express or Node with no proxy in front, nothing extra is needed. If a proxy sits in front of your app, set `trustedProxy` so the middleware can find the crawler's own address:

| Deployment | Setting |
| - | - |
| Cloudflare in front of the app | `trustedProxy: 'cloudflare'` |
| CloudFront directly in front of the app | `trustedProxy: 'cloudfront'` — see [CloudFront](https://docs.devtune.ai/ai-traffic/advanced-setup.md#cloudfront) |
| Another reverse proxy in front | A numeric `trustedProxy` offset — see [trusted proxies](https://docs.devtune.ai/ai-traffic/advanced-setup.md#trusted-proxies) |
| Send no address | `trustedProxy: 'none'`; traffic is stored as unverified |

Keep `@devtune/ai-traffic` up to date. AI Traffic pages show an upgrade notice when your middleware is behind the latest release; see [package versions](https://docs.devtune.ai/ai-traffic/advanced-setup.md#package-versions) for what each release adds.

## Delivery and statuses

Delivery is best effort. A batch that fails, or stays rate-limited past its retries, is dropped and not resent, and drops are silent unless you pass a `logger`. To send events immediately on short-lived runtimes, or to tune retries and queue size, see [delivery](https://docs.devtune.ai/ai-traffic/advanced-setup.md#delivery).

Express and Node record the completed response status. A Next.js proxy can't see the final route status after pass-through, so those requests are status unknown and don't reach 404 Demand unless you [capture the status](https://docs.devtune.ai/ai-traffic/advanced-setup.md#response-statuses).

## Check the middleware is working

1. Deploy, then request a page the middleware runs on with a crawler user agent. Replace the address with your own:

   ```bash theme={null}
   curl -A "GPTBot/1.3" https://www.example.com/
   ```

2. In **Settings › Integrations**, reload the page. Until DevTune accepts its first event, the **Edge middleware** row reads **Installation has not been confirmed.** Within a few minutes it reads **Active**, with **Last ingest** and a time, and lists your hostname. A linked Vercel project also changes from **Not seen yet** to **Last seen**.

3. Don't expect the test request in the verified totals. It didn't come from OpenAI's network, so it isn't counted as a verified crawl.

## If something goes wrong

If your app fails at startup, the middleware couldn't read its settings:

| Error | What to do |
| - | - |
| `DevTune AI Traffic needs an ingestKey or a projectId` | Set `DEVTUNE_AI_TRAFFIC_INGEST_KEY` or `DEVTUNE_AI_TRAFFIC_PROJECT_ID` in every environment the proxy runs in, or skip creating the middleware when neither is set |
| `DevTune AI Traffic projectId must be a DevTune project UUID` | Set `DEVTUNE_AI_TRAFFIC_PROJECT_ID` to the **DevTune project ID** shown where you link Vercel projects |

On Vercel, variables are set per environment, so a variable set only for Production leaves Preview deployments without it.

If the row doesn't change:

* Check that `DEVTUNE_AI_TRAFFIC_INGEST_KEY`, or `DEVTUNE_AI_TRAFFIC_PROJECT_ID` on Vercel, is set in the environment you deployed.
* Check that **Snippet and middleware collection** is switched on in **Settings › Integrations**. While it is off, DevTune rejects every batch.
* Pass a `logger` to see failed or dropped sends. See [delivery](https://docs.devtune.ai/ai-traffic/advanced-setup.md#delivery).
* On Vercel without an ingest key, work through [Link status](#link-status).

## Next steps

* **[Advanced sensor settings](https://docs.devtune.ai/ai-traffic/advanced-setup.md)** - identity checks, proxies, delivery tuning and package versions
* **[Set up AI Traffic](https://docs.devtune.ai/ai-traffic/setup.md)** - connect website analytics, Search Console and the tracking snippet
* **[Reading AI Traffic](https://docs.devtune.ai/ai-traffic/dashboard.md)** - what each view counts


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.