About GroundtruthBot

GroundtruthBot/1.0 (+https://tillingdata.com/bot; crawler@tillingdata.com) is an automated crawler operated by Tilling Data (engine: groundtruth). It collects public, marketing-facing company web pages to build an open US company database.

How it behaves

• Obeys robots.txt strictly; denied hosts are never fetched.
• Identifies itself with a contact URL and email in the User-Agent.
• Rate-limited per host (about one request every ten seconds, ≤5 pages/host/day).
• Reads public marketing pages only — no logins, no proxies, no evasion.
• Does not render JavaScript and follows no paywalled or private links.

Contact

To request exclusion or ask a question, contact crawler@tillingdata.com. We honour opt-outs promptly.