[gopher] Re: Veronica-2 and robot exclusion

["Ruliz Galaxor" <killspy@xxxxxxxxxxxxxxxxxx>]

> I re-read the long thread back in January '01 where we went over how to
> implement a robot exclusion standard for Gopherspace, and the solution
that
> had the widest support (except from yours truly at that time) was to
> re-use the existing robots.txt (for your convenience, the current 1994
> standard is available at
>
> gopher://gopher.floodgap.com/0/v2/robotstxt.txt

Should the robots.txt be visible in the index if you request it normally, or
is it also allowed to be invisible?

Invisible as in:
Requesting the index doesnt show any robots.txt -> gopher://foo.bar.com/
but requesting robots.txt works -> gopher://foo.bar.com/0/robots.txt

Is this 'invisible' use of robots.txt allowed?

greets,

Ruliz Galaxor
gopher://kabel218067.mobiel.utwente.nl/