Free AI Crawler Impact Report
Upload your server log file to see exactly which AI bots are crawling your site, how much bandwidth they consume, and get a ready-to-use robots.txt snippet.
Drop your log file here
Supports CloudFront, Apache/Nginx access logs (.log, .txt, .gz, .zip)
Your file is processed entirely in your browser. No data is uploaded to any server.
Analyzing log file...
AI Crawler Breakdown
| Bot Name | Category | Requests | % of Total | Bandwidth | Unique IPs | Top Paths |
|---|
Crawl Timeline
Most Crawled Paths (AI Bots)
Generated robots.txt
robots.txt
Customize this based on your needs. You may want to allow some AI crawlers for discovery while blocking others that consume excessive bandwidth.
Want continuous AI crawler monitoring?
LogLens monitors your traffic in real-time, verifying bot claims against official IP ranges and alerting you to changes.
Real-time alerts
IP verification
Historical trends
robots.txt monitoring