UserPathBot

UserPathBot is the crawler behind UserPath’s site check. It reads the public pages of sites whose owners use UserPath, so they can see SEO problems like missing titles, broken links and noindex tags.

What it reads

Only the site its owner added to UserPath: robots.txt, the sitemap, the home page and the hundred pages visitors open most, plus a sample of the links on them. It sends this user agent:

UserPathBot/1.0 (+https://userpath.app/bot)

How often

Once a day, and when the owner asks for a check (at most every ten minutes). A few requests at a time with a pause between them, and it gives up on a page after ten seconds.

Keeping it out

UserPathBot follows robots.txt. To keep it off your site, add:

User-agent: UserPathBot
Disallow: /

Questions or problems: [email protected].