Skip to main content

OrganicOS Crawler

Last updated: October 2026

If you found this page in your server logs, a request came from the OrganicOS crawler. OrganicOS is an SEO and revenue analytics product. We crawl a website only when its owner (or someone they have authorised) adds it to OrganicOS and runs a site audit.

How to recognise it

Every request sends this user-agent:

OrganicOS-Crawler/1.0 (+https://www.organicos.online/crawler)

We never pretend to be a search engine, a browser or a residential visitor. A request claiming this user-agent that ignores your robots.txt is not us. Please tell us about it.

What it does

  • Fetches public HTML pages, robots.txt and XML sitemaps. It does not submit forms, log in or run checkout flows.
  • Follows robots.txt by default. A rule for User-agent: OrganicOS-Crawler or User-agent: * applies.
  • Crawls at a rate set by the site owner in OrganicOS. The default is at most 5 requests per second, and the crawler backs off and pauses when your server answers 429 (Too Many Requests).

Allowing the crawler

If you own the site and want OrganicOS to audit it, allow the crawler narrowly. Do not turn off your bot protection or firewall, and do not allow traffic based on the user-agent alone. Anyone can copy a user-agent.

  • Cloudflare Pro, Business or Enterprise: add a WAF custom rule with the Skip action that skips only Super Bot Fight Mode. Limit it to your own hostname and to our user-agent and our network identity (below). Leave every other protection on.
  • Cloudflare Free (Bot Fight Mode): Cloudflare does not let Bot Fight Mode be bypassed for a single bot. Contact us and we will work out the safest option for your site.
  • Other CDNs and firewalls: use the same pattern: a narrow exception on your hostname, never a global switch-off.

Network identity

We don’t publish a fixed list of IP addresses yet, because our outbound addresses are not guaranteed to stay the same. Publishing a list that might change would break your rule without warning. Until a stable list is published here, contact us for help setting up an exception that is safe for your site.

Contact

Questions, problems or reports of misuse: support@organicos.online. Include the date, time and URL from your logs so we can trace the request.