Content & Copyright

This page is for two audiences: people curious how Tetrachrome's crawler works, and site owners who've noticed TetrachromeBot in their access logs and want to know what it is, how to control it, or how to have their content removed from the index.

• About the crawler
• How to block or restrict it
• Removal & opt-out
• How indexed content is used
• Copyright

About the crawler

Tetrachrome's crawler identifies itself with the following User-Agent on every request:

TetrachromeBot/0.1 (+https://tetrachrome.net/content-policy.html; research crawler, robots.txt and Crawl-delay honored)

It is a small, single-operator research project — not a commercial search engine, not operated at scale, and not affiliated with any other "AP", "BBC", or similar bot despite crawling those and other public news sources. Concretely, it:

• fetches and obeys robots.txt before crawling any page on a domain, including the domain's Crawl-delay directive if one is specified (used as the minimum gap between requests to that host, or our own default pacing, whichever is longer)
• rate-limits itself per host regardless of what robots.txt says, so it never hammers a single site even where robots.txt permits faster access
• backs off automatically (with exponential delay) on repeated 403/429/5xx responses from a host, rather than continuing to retry a site that's rejecting it
• only follows links within the domains it's explicitly seeded to crawl — it doesn't crawl the open web indiscriminately

How to block or restrict it

The immediate, self-serve way to stop Tetrachrome from crawling your site — no need to contact anyone — is robots.txt. It's honored on every request. To block the whole site:

User-agent: TetrachromeBot
Disallow: /

Or to block specific paths only, or set a minimum delay between requests:

User-agent: TetrachromeBot
Disallow: /private/
Crawl-delay: 10

Changes to robots.txt take effect the next time Tetrachrome revisits your domain — it does not cache a robots.txt result indefinitely across separate crawl runs.

Removal & opt-out

robots.txt is the immediate, self-serve way to opt out — it's enforced automatically and takes effect on Tetrachrome's next visit to your domain, without anyone needing to process a request by hand. This is still a small, single-operator project without a staffed support channel or a removal-request form, so if you need something removed sooner than the next crawl, or need to reach a person directly, robots.txt plus this page is the extent of what's available today — a maintained contact address will be added here if that changes.

How indexed content is used

For each crawled page, Tetrachrome stores extracted text (for search ranking), a short snippet, and at most one representative image — always alongside a link back to the original URL and, where available, the site's own name. Full page content is not republished: search results show a short excerpt, not the article. Every result links to the original source as the way to read the full piece.

Copyright

Tetrachrome does not claim ownership over any content it indexes. All crawled text, images, and other material remain the property of their original publishers. Short snippets and thumbnail-sized preview images are used for the purpose of describing and linking to that original content, consistent with standard search-engine indexing practice — the same purpose robots.txt itself exists to let a site owner control.