What it is
The Topicweave site checker reads a website’s public pages and writes a private report about how its projects, writing and structured data connect. It only runs when a person asks for a check at topicweave.com/check. It is not a search engine, and it doesn’t build an index of the web.
How to recognize it
Every request carries this user agent:
Mozilla/5.0 (compatible; TopicweaveSiteCheck/1.0; +https://topicweave.com/bot)
Requests come from Vercel’s servers, so the address belongs to Vercel, not to us.
What it reads, and how politely
- Public pages only. Your sitemap first, then pages your own pages link to. Nothing behind a login, no forms submitted, nothing changed.
- At most 330 pages per check, and one check per site per day.
- Four requests at a time, each stopped after 10 seconds or 2 MB. If your server says it’s busy, it waits and tries once more, then moves on.
- Raw HTML only. It doesn’t run your scripts, load images or fill in analytics.
- It follows robots.txt. If your robots.txt disallows a page for
TopicweaveSiteCheck(or for every crawler), it isn’t read.
How to turn it away
Add this to your site’s robots.txt and no check will read your pages:
User-agent: TopicweaveSiteCheck Disallow: /
The person who asked will see a note that your site asks not to be read.
What happens to what it reads
The report goes only to the person who asked for it, at the email they gave. It quotes short pieces of your public pages so each finding can be checked. To suggest which specialties deserve a page of their own, the page titles and the phrases we counted on your pages are sent to Anthropic’s Claude; nothing else is. Page data is deleted after 30 days; the findings are kept so a later check can show what changed. We never email the business that was checked, and we don’t sell or share what the checker reads. More in our privacy notice.
Questions
Write to hello@topicweave.com. A person reads every message, and if you’d like your site never to be checked, we’ll add it to a blocklist the same day.