Robot Catcher

This page should be visited by robots only.

If you manually got to this page, please use the back button of you browser.

A robot should NOT follow this link

Special keywords: Patrick Gautschi robots foobar followed


The Robots Exclusion Protocol

The Robots Exclusion Protocol is a method that allows Web site administrators to indicate to visiting robots which parts of their site should not be visited by the robot. In a nutshell, when a Robot vists a Web site, say http://www.foobar.com/, it firsts checks for http://www.foobar.com/robots.txt. If it can find this document, it will analyse its contents for records like: User-agent: * Disallow: / to see if it is allowed to retrieve the document. The precise details on how these rules can be specified, and what they mean, can be found in: Web Server Administrator's Guide to the Robots Exclusion Protocol HTML Author's Guide to the Robots Exclusion Protocol The original 1994 protocol description, as currently deployed. The revised Internet-Draft specification, which is not yet completed or implemented.