Submitted by : SimonMichael at: 2004-02-03T11:01:01+00:00 (13 years ago)
Name :
Category : Severity : Status :
Optional subject :  
Optional comment :

zwiki.org is unusually slow and conflict-prone again. The reason is we are being crawled by one or more robots, and the skin now includes more expensive links for them to trigger. We need to look at the general issue of robot-robustness again.

Those hidden links I added for access keys are a problem :(, especially things like /print (rendering many pages) and /clearCache (causing large pages to be re-rendered next time). For now I am removing at least these two urls from the zwiki.org skin.

Standard links like contents, recent changes, index may be expensive as well. Right now there's a different contents url for each page, maybe we need to get by with just one.


comments:

property change --SimonMichael, Thu, 05 Feb 2004 13:45:45 -0800 reply
Title: IssueNo0699 current skin is too open to robots, makes sites slow => IssueNo0699 current skin is too open to robots

semi-serious --SimonMichael, Thu, 05 Feb 2004 13:55:52 -0800 reply
This is not serious in most cases, but it's serious enough for a large wiki which allows robots and has the print dtml method installed (zwiki.org).

improved for 0.27.1 --SimonMichael, Fri, 06 Feb 2004 01:05:54 -0800 reply
The clearCache and print links are disabled, and the contents link now defaults to non-scrolling (so just one contents url per wiki).

property change --SimonMichael, Fri, 06 Feb 2004 01:08:14 -0800 reply
Status: open => closed

the fix is irritating for users, though --simon, Sat, 17 Apr 2004 19:30:05 -0700 reply
Title: IssueNo0699 current skin is too open to robots => IssueNo0699 too many urls for robots to crawl Severity: serious => normal Status: closed => open

allowed contents to scroll again, cost is reasonable --simon, Sat, 29 May 2004 09:45:25 -0700 reply
Status: open => closed