OrganicOS Crawler
Last updated: October 2026
If you found this page in your server logs, a request came from the OrganicOS crawler. OrganicOS is an SEO and revenue analytics product. We crawl a website only when its owner (or someone they have authorised) adds it to OrganicOS and runs a site audit.
How to recognise it
Every request sends this user-agent:
OrganicOS-Crawler/1.0 (+https://www.organicos.online/crawler)We never pretend to be a search engine, a browser or a residential visitor. A request claiming this user-agent that ignores your robots.txt is not us. Please tell us about it.
What it does
- Fetches public HTML pages, robots.txt and XML sitemaps. It does not submit forms, log in or run checkout flows.
- Follows robots.txt by default. A rule for
User-agent: OrganicOS-CrawlerorUser-agent: *applies. - Crawls at a rate set by the site owner in OrganicOS. The default is at most 5 requests per second, and the crawler backs off and pauses when your server answers 429 (Too Many Requests).
Allowing the crawler
If you own the site and want OrganicOS to audit it, allow the crawler narrowly. Do not turn off your bot protection or firewall, and do not allow traffic based on the user-agent alone. Anyone can copy a user-agent.
- Cloudflare Pro, Business or Enterprise: add a WAF custom rule with the Skip action that skips only Super Bot Fight Mode. Limit it to your own hostname and to our user-agent and our network identity (below). Leave every other protection on.
- Cloudflare Free (Bot Fight Mode): Cloudflare does not let Bot Fight Mode be bypassed for a single bot. Contact us and we will work out the safest option for your site.
- Other CDNs and firewalls: use the same pattern: a narrow exception on your hostname, never a global switch-off.
Network identity
We don’t publish a fixed list of IP addresses yet, because our outbound addresses are not guaranteed to stay the same. Publishing a list that might change would break your rule without warning. Until a stable list is published here, contact us for help setting up an exception that is safe for your site.
Contact
Questions, problems or reports of misuse: support@organicos.online. Include the date, time and URL from your logs so we can trace the request.