[gopher] Re: Getting file info from an URI??

[Stefan Rieken <StefanRieken@xxxxxxxxxxxx>]

On 15 Jun 2001 05:39:31 -0700, em@nuel wrote:

> The real diffirence is
> that the web assumes that, given a URL, you get all the meta-data from
> what is returned when the URL is requested.  Gopher works the other
> way around - the directory gives the meta-data, which is what you use
> when you display the content.

Right.

> Unfortunately, the gopher:// URL
> doesn't carry arround the meta-data.  My solution with web->gopher was
> to pass all that stuff around continuously, which is why the URLs are
> extremely long (they contain the title, mime type, etc.)

And "incompatible" with "standard" gopher:// URI's[*].

> Yes, Gopher+ does help with the info request, but it's an inefficient
> way of doing things because you need to do two requests each time (one
> for the meta-data, one for the content), so it's best avoided if
> possible, even when it is available.

Well, if that's what is needed to get it done. Remember that the
metadata request is only a short request, and that although e.g. HTTP
does this in one request, also there you just need to have the metadata.

Although the thought of having to make two connections each time is a
little ugly, I think in practice it can be efficient enough. And
although maybe my "pool" probably wasn't a good idea, well, you can
still always use a "cache" for metadata. (Hey, this is not the `80's
anymore, and I'm not writing the C64 client, so I guess a little
overhead on the client side shouldn't really problem ;-)

> I'm don't really know the details about the gopher:// URL structure,
> but my impression is that it basically contains the full request line
> passed to the gopher server, so it could be something like
> "gopher://1/fooF+Ftext/html"; to request the selector "1/foo" with
> Gopher+ as type text/html.  Getting the info is more complex than just
> appending "F!" to whatever comes after the //.

Nah, I was more thinking of following the "standard"[*] gopher URI's.
The user (the bloke that types the URI's) should not need to ask
metadata. That's what the gnome-vfs method does internally. The user
just sees the results of that.

So no, there should never be more tab stuff in the URI than probably a
search query or so[*]. And the program will translate that to selector
strings (with or without any special stuff after the tab) internally

> 
> > Can one assume Gopher+ nowadays?
> 
> No.  There are lots of servers that aren't, including important ones
> like floodgap.

Hmm, well, shoot.

I guess my best bet to find out the metadata is then probably:

1) Try an "information" request for a file
2) If the server is not Gopher+, it will probably interpret this as a
normal requrest and start returning the file. Depending on the situation
(whether the file is to be downloaded anyway, or not), I could then
download part of the file and abort the transfer, or download the whole
file, and "sniff" the mime type using gnome-vfs's mime type database.

This, possibly combined with a metadata cache (containing the info from
dir listings and earlier "!" requests) to eliminate most of the
redundant lookups could, at least in theory, do the trick.

> > 2) Make a pool of recently downloaded directory listings from
> >    which one can search for the given selector and get the file
> >    description and type.
> 
> Yikes.  And it doesn't help if the user simply enters a URL without
> navigating to it (but you can probably live with that... it just won't
> have a title and stuff).

Uh... Oh, yeah! B-/

I guess I blocked myself from figuring *that* out on my own ;-) Anyway,
I'll probably make it a cache now (read: eventually), so that it's role
becomes supplementary, and not required.

Greets,

Stefan

[*] OK, I guess someone has to ask this sooner or later: is there ANY
kind of (de facto) standard for Gopher URI's?

I guess this one's commonly accepted:
gopher://<host>[:<port>]/<selector(urlencoded)>

What do most programs use for Veronica queries? I use:
gopher://<host>[:<port>]/<selector(u.)>?<query(u.)>

And my religion is that the queries shouldn't include "extra stuff" like
the proposed "F!". I think asking for information to the server,
choosing between different file types etc. should not be handled by the
URI. They should either be handled by the program through dialogs (to
the user) and special requests (to the server), or be ignored -- OK, or
they should be hackable into the URI, but this should NOT be considered
standard behavior ;-)

I'm not gonna ask you what you use for forms, because I will declare war
on these crappy forms one of these days, and solve that mess first (yea,
right ;-)

--
"The main problem is getting something that works without crashing the
NT/ 2000 client" -- Jeremy Allison, Samba development team