Verified-bot.
If your crawler is verified at the edge, no enrolment is required. Begin requests at the published tariff. Settlement is taken at delivery.
- A registered crawler user-agent.
- A published, verifiable bot identity.
- Outbound requests from declared address ranges.
- Compliance with /robots.txt and the published rate ceilings.