[gopher] Re: quux.org archives off-line?

[Cameron Kaiser <spectre@xxxxxxxxxxxx>]

> Well, I wasn't trying to point fingers at any particular person or
> server...the point of my question was:  Do "dead" selectors ever get
> pruned from the V2 results?

There is an autoprune (it is intentionally made wimpy but it does work), but
only at the host level. If a host continues to answer the robot does not
generally recrawl it unless a certain 'staleness' interval has elapsed which
is based on site activity and selector size, so if there is major selector
churn the robot may not notice.

However, this would not necessarily solve this situation since data is
being returned and the robot is intentionally programmed to only kill 
resources with no data. I've toyed with making it understand itemtype 3 but
this obviously falls down if the selector is something other than 1 or 7.

-- 
------------------------------------ personal: http://www.cameronkaiser.com/ --
  Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@xxxxxxxxxxxx
-- Le fascisme est la dictature ouverte de la bourgeoisie. -- Georg Dimitrov --