Skip to main content
Paid tests use your saved test apps, requests and agent/model choices. A Full test runs each scenario up to eight times on every plan, and stops a scenario after four runs when the first four agree. A test app you tick under Hosting and requirements is one more test app: a Full test runs each request it asks there, so a test app that doesn’t ask one request adds fewer runs. Quick test, Custom and Re-run this scenario can run fewer, and cost less. You need permission to manage the project to request or cancel a test.

Choose a test

The run count drives the credits a test reserves and spends. A test at 8 runs per scenario (a Full test, or Custom at 8) reserves credits for every run, stops a scenario after four runs when the first four agree, and charges only for the runs that ran. The rest of the reservation comes back when the test finishes. Only a test that covers every test app, request and agent in the setup, with 3 or more runs per scenario, measures pick rates, creates loss Actions and adds a point to the trend. Any other test is marked Directional: it shows what each agent picked, not a rate. Open Re-run this scenario from a scenario in the Matrix or from a run’s detail. A re-run is its own test: it doesn’t replace the scenario’s rate. Every choice uses the same estimate, reservation and refund rules below, and each counts as the setup’s one paid test while it runs.

Estimate

  1. Open Agent Picks › Overview. If the project has several setups, select the setup to test with the Setup picker.
  2. Select Run test. With several setups, the dialog names the setup the test runs in, and so does the reservation line.
  3. Choose Full test, Quick test or Custom. For a Quick test, choose the test app. For Custom, choose the test apps, requests, agents and runs per scenario.
  4. Review the run count and the Expected and High credit estimates.
  5. Select Confirm and reserve credits to queue the test.
Expected is the estimated spend. For a Full test, or Custom at 8 runs per scenario, it assumes most scenarios stop after four runs, as they usually do. Re-run this scenario doesn’t stop early, at any run count. High prices all eight and is the amount reserved from your account’s shared credit balance before the test starts. Changing test apps, requests or models changes the estimate. If the setup or estimate changed after you reviewed it, request a fresh estimate and confirm again. Only one paid test can be queued or running per setup, including re-tests. Each setup of a project can have its own test running at the same time. Requesting another for a setup while its test is active does not reserve more credits.

Settlement and refunds

Completed runs charge their recorded usage. Failed runs, runs cancelled before starting, and runs a scenario didn’t need because its first four agreed charge no credits. A run cancelled after starting charges its recorded spend. When a completed run or a run cancelled after starting has unknown spend, it charges the greater of its recorded spend and its share of the reservation. The share is the high reservation divided by the planned runs. Total charges never exceed the reservation, and settlement does not take additional credits. Unused credits return when all active runs have settled. The paid-test status shows Reserved, Settled and Refunded, with the charge for each run under Runs. Returned credits keep their original expiry; a refund does not turn expired allowance credits into purchased credits. Select Cancel test to stop a queued or running paid test. Unstarted runs are released. In-flight runs must stop and settle before their unused credits return, so cancellation can remain pending for a while.

Credit shortfalls

If the balance cannot cover the high estimate, no test is queued and no credits are reserved. The app shows how many more credits you need and a Buy a credit pack link to Billing. Ask a member with billing permission to buy credits if you cannot. Credits are shared across the account. Other usage can reduce the balance between the estimate and confirmation.

Plans and trials

Agent Picks has its own subscription. Its included credits join the account’s shared pool each month; changing or cancelling it does not change your AI Search subscription. Every plan crawls your site, including your docs, so losses, Actions and briefs can check which of your pages cover the job. If you also have AI Search, the two plans’ page allowances add together: AI Search Plus with Agent Picks Pro tracks up to 3,500 pages per project. Plus and Pro offer a 14-day trial without a card for eligible accounts. Trial eligibility is account-wide across Agent Picks and AI Search; an account with subscription history subscribes through Billing instead. An Agent Picks trial includes the free starter test and adds no monthly Agent Picks credits. You can run further paid tests during the trial if your shared balance covers the high reservation. A trial does not make custom-job runs available when the app says they are unavailable.

Billing history

Open Billing › Transactions to inspect the account ledger:
  • Agent Picks test, marked Pending, is the high reservation.
  • After settlement, Agent Picks test estimate is marked Replaced by final amount and crossed out.
  • The settled Agent Picks test row shows the final charge. A test that failed settles as Agent Picks test failed.
The replaced estimate and final amount are not two charges. See Credit transactions for the shared ledger.