[gopher] Re: Veronica-2 and robot exclusion

[John Goerzen <jgoerzen@xxxxxxxxxxxx>]

On Wed, Jun 18, 2003 at 10:44:55AM -0700, Cameron Kaiser wrote:
> > Is this 'invisible' use of robots.txt allowed?
> 
> This would be the way I'd do it, and this is easy to do with a link list
> (like a Bucktooth gophermap file). There's probably no harm if the robots.txt
> file does get presented to the user, though.

It's pretty easy with UMN (and PyGopherd UMN module) as well:

Just add an entry in .names:

Path=./robots.txt
Type=X

-- John