agentcrawls.com
Docs-mirror crawler + R2 server · tool
Worker
agentcrawls-web — launched pre-expansion. Design system: legacy hardcoded hex (predates the shared token set).
What it is
Python Worker (Pyodide) serving the R2-mirrored docs content, accepting authenticated pushes from the local scrapyd pipeline, and running a daily human-out-of-the-loop incremental crawl on its own Cron Trigger. Headless — no HTML shell of its own, so the swatch here is the family's neutral gray, not a site accent.
Live activity
No contracts or tasks tagged source_site=agentcrawls.com yet.