Complete.Org: Mailing Lists: Archives: gopher: October 2005:
[gopher] Re: New Gopher Wayback Machine Bot
Home

[gopher] Re: New Gopher Wayback Machine Bot

[Top] [All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index] [Thread Index]
To: gopher@xxxxxxxxxxxx
Subject: [gopher] Re: New Gopher Wayback Machine Bot
From: John Goerzen <jgoerzen@xxxxxxxxxxxx>
Date: Thu, 13 Oct 2005 00:22:34 -0500
Reply-to: gopher@xxxxxxxxxxxx

On Wed, Oct 12, 2005 at 11:28:26PM -0500, John Goerzen wrote:
> On Wed, Oct 12, 2005 at 09:07:52PM -0500, Benn Newman wrote:
> I will be doing more work tomorrow to make it be more round-robin so
> as to not put much load on any one server at once.

And this bit is now done as well.  It will make one connection to
every server in its "to-spider" list before it ever connects to one
again.

I'll also keep an eye on it and make sure nobody gets pounded at the
very end.

-- John



[Prev in Thread] Current Thread [Next in Thread]