Product / Agent Analytics
Watch AI crawlers work your site.
GPTBot, ClaudeBot, PerplexityBot and eleven more — what they read, what they're blocked from, and what that costs you in answers. One edge snippet, live in minutes.
CrawlersReferralsPagesLogs
Agents · 30d48.2K REQUESTS · 14 AGENTS
GPTBotOpenAI21,408ALLOWED
PerplexityBotPerplexity11,872ALLOWED
ClaudeBotAnthropic8,455PARTIAL
Google-ExtendedGoogle6,203ALLOWED
Alert: ClaudeBot 403s on 214 /help URLs — robots.txt rule from 2024. Action #87 →
01
Log-level truth
Real edge requests, not synthetic checks — by agent, path, status and day.
02
Config alerts
robots.txt and llms.txt misconfigurations flagged with the exact rule — and the fix.
03
AI referral attribution
The human traffic AI sends back — by platform, landing page and conversion.
Setup
Live in minutes, not sprints
1
Add the edge snippet
One line at the CDN or middleware layer — Cloudflare, Vercel, Fastly, nginx.
2
Crawlers appear within the hour
14 known AI agents classified automatically; unknown bots fingerprinted.
3
Misconfigurations become actions
Every blocked path is scored by the answer visibility it costs you.
Edge snippet~1 MIN
// cloudflare worker · answr agent analytics
import { answr } from "@answr/edge";
export default answr("aw_live_7f2…", { site: "nike.com" });
No request blocking< 0.3 ms overheadLog retention 13 mo
Coverage
14AI agents
GPTBot · ClaudeBot · PerplexityBot + 11 more, out of the box
Attribution
4.8K/moreferred visits
Human traffic AI sends back, tied to landing pages
Found on first scan
214blocked URLs, median
Misconfigurations scored by the answers they cost
Find out what the crawlers can't read.
Get a demoSee pricing