Acceptable Use Policy
Last updated: November 2026
1. Authorized Crawling Only
Users may only crawl websites they own or for which they have express consent from the site operator. SolidLabs enforces domain verification for large-scale, recurring, and authenticated crawls.
2. Prohibited Behaviors
You may not use SolidLabs to conduct denial-of-service attacks, bypass CAPTCHA protections, scan internal private network infrastructure (SSRF), or scrape copyrighted media at abusive frequencies.
3. robots.txt and Rate Limits
Our crawler honors RFC 9309 robots.txt exclusion rules and applies bounded concurrency to prevent overloading destination web servers.