Some notes made during the Zope 3 wiki move (from http://www.zope.org/DevHome/Wikis/DevSite/Projects/ComponentArchitecture , zope 2.6, zwiki 0.21 to http://zope3.zwiki.org , zope 2.9, zwiki 0.57). May be of interest to other wiki migrators and zwiki developers.

Zope 3 wiki move notes

2006/10/14

open questions

  • stx or rst as default markup ?
  • show subtopics or off ?
  • max_identified_links ?
  • WikiName links on or off ?
  • single bracket links on or off ?
  • mail out edits or comments by default ?
  • mail-in on or off ?
  • webdav, ftp ?

migration procedure

This is what it boiled down to after several reiterations of the process. This seems to be a relatively efficient sequence.

  • export
  • import
  • rename folder if desired, or (recommended) move everything to a btreefolder
  • configure folder permissions (later, errors might reveal some bad page permissions which should be reset also)
  • configure folder properties, including - page types - anti spam options - mail options
  • FrontPage/upgradeAll
  • FrontPage/setupCatalog again
  • (may even require FrontPage/setupTracker to get all required catalog fields ?)
  • delete old zwiki dtml pages
  • clean out bad wiki & page subscriptions, replace usernames with email addresses

issues noted/possible issues

parents adjusted during import.. does import trigger an outline update too soon ?

even with no catalog, renaming folder was slow

to avoid conflicts, leave long operations alone to complete, and/or use batch arg

page types were migrated to rst instead of stx (fixed, needs testing)

"could not import class 'ZWikiRegulations?' from module 'Products.ZWiki.ZWikiPage'" could be cleaned up

after upgradeall only a third of pages had correct path in catalog, perhaps due to wiki page with id "path"

upgrading failed on pages with non-ascii:

 2006-10-14T13:25:28 BLATHER ZWiki failed to upgrade page 225/0 UnicodeForText: Traceback (most recent call last):
File "/zope1/Products/ZWiki/Admin.py", line 76, in upgradeAll
  p.preRender(clear_cache=1)
File "/zope1/Products/ZWiki/ZWikiPage.py", line 280, in preRender
  self.setPreRendered(self.pageType().preRender(self))
File "/zope1/Products/ZWiki/pagetypes/rst.py", line 45, in preRender
  t = self.format(t)
File "/zope1/Products/ZWiki/pagetypes/rst.py", line 36, in format
  initial_header_level=RST_INITIAL_HEADER_LEVEL-1
File "/usr/lib/zope2.9/lib/python/reStructuredText/__init__.py", line 137, in HTML
  settings = settings)
File "/usr/lib/zope2.9/lib/python/reStructuredText/__init__.py", line 90, in render
  config_section='zope application')
File "/usr/lib/zope2.9/lib/python/docutils/core.py", line 407, in publish_parts
  enable_exit_status=enable_exit_status)
File "/usr/lib/zope2.9/lib/python/docutils/core.py", line 516, in publish_programmatically
  output = pub.publish(enable_exit_status=enable_exit_status)
File "/usr/lib/zope2.9/lib/python/docutils/core.py", line 194, in publish
  self.settings)
File "/usr/lib/zope2.9/lib/python/docutils/readers/__init__.py", line 65, in read
  self.input = self.source.read()
File "/usr/lib/zope2.9/lib/python/docutils/io.py", line 307, in read
  return self.decode(self.source)
File "/usr/lib/zope2.9/lib/python/docutils/io.py", line 104, in decode
  raise UnicodeError(
UnicodeError: Unable to decode input data.  Tried the following encodings: 'ascii'.
(UnicodeDecodeError: 'ascii' codec can't decode byte 0xa3 in position 6264: ordinal not in range(128))

If you have non-ascii characters in your page names or content: the
default skin used to specify iso-8859-1 character encoding; it now
specifies utf-8.  So your wiki's character encoding may change, and
non-ascii characters may no longer display correctly. You'll have to
convert these manually (see http://zwiki.org/IssueNo0682 for tips).

/usr/lib/zope2.9/lib/python/OFS/subscribers.py:74: DeprecationWarning?: Products.ZWiki.ZWikiPage.ZWikiPage.manage_beforeDelete is deprecated and will be removed in Zope 2.11, you should use event subscribers instead, and meanwhile mark the class with <five:deprecatedManageAddDelete/>

logs

It's a good idea to set zope's logging level to BLATHER and watch the event.log to see what's going on. Here's a representative sample to show what to expect.

import

2006-10-14T14:18:55 BLATHER ZWiki regenerating outline data for wiki ComponentArchitecture
2006-10-14T14:18:56 BLATHER ZWiki adjusting SecurityFramework's parents from ['Zope.App.Security'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting Zope.App.Security's parents from ['Zope.App'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting BasicObjectSecurity's parents from ['Zope.App.Security'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting SteveExplainsObjectHubToGaryOnIRC's parents from ['Zope.ObjectHub'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting Zope.ContextWrapper.ContextMethod's parents from ['Zope.ContextWrapper'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting Zope.App.OFS's parents from ['Zope.App'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting Zope.App.OFS.Services's parents from ['Zope.App.OFS'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting Zope.App.Caching's parents from ['Zope.App'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting DifficultyLevel's parents from ['DevelCookbook'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting RecipeTemplate's parents from ['DevelCookbook'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting Preface's parents from ['DevelCookbook'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting NewApplicationControl's parents from ['DevelCookbook'] to []
2006-10-14T14:18:56 BLATHER ZWiki adjusting WritingTestsAgainstInterfaces's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting ZwikiForZope3's parents from ['Zope3Products'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting VocabularyFields's parents from ['SchemaIntroduction'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting ComponentsOverview's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting AdvancedPythonConcepts's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting ChangingExceptionViews's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting ZopePublicLicense's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting NewTALESExpr's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting UsingEventPackage's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting OnlineHelpForZopeApps's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting CreativeCommonsAttributionNoDerivsNonCommercialLicense's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting LuceneTextIndex's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting Zope3GUI's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting SyncTTWCodeToFS's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting Spacesuits's parents from ['DevelCookbook'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting MultiCheckBoxWidget's parents from ['SchemaIntroduction'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting Xickens CMS's parents from ['Xickens CMS'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting leticia2602's parents from ['plotnik_'] to []
2006-10-14T14:18:57 BLATHER ZWiki adjusting germish's parents from ['MessageBoard'] to []
2006-10-14T14:22:27 BLATHER ZWiki regenerating outline data for wiki recycle_bin
2006-10-14T14:22:27 BLATHER ZWiki adjusting TestPage's parents from ['TestPage'] to []

upgradeAll

2006-10-14 14:37:58 WARNING ZODB Could not import class 'ZWikiRegulations' from module 'Products.ZWiki.Regulations'
2006-10-14 14:37:58 BLATHER ZWiki upgrading/reindexing/pre-rendering all pages:
2006-10-14 14:37:59 WARNING ZODB Could not import class 'ZWikiRegulations' from module 'Products.ZWiki.ZWikiPage'
2006-10-14 14:38:05 BLATHER ZWiki upgraded page_type attribute of HelpPage
2006-10-14 14:38:05 BLATHER ZWiki upgraded HelpPage
2006-10-14 14:38:06 BLATHER ZWiki upgraded page_type attribute of path
2006-10-14 14:38:06 BLATHER ZWiki upgraded path
2006-10-14 14:38:07 BLATHER ZWiki upgraded page_type attribute of RecentChanges
2006-10-14 14:38:07 BLATHER ZWiki upgraded page_type attribute of UserOptions
2006-10-14 14:38:07 BLATHER ZWiki upgraded page_type attribute of SearchPage
2006-10-14 14:38:08 BLATHER ZWiki upgraded and pre-rendered page 1/702 HelpPage
2006-10-14 14:38:08 BLATHER ZWiki upgraded page_type attribute of JumpTo
2006-10-14 14:38:08 BLATHER ZWiki upgraded JumpTo
2006-10-14 14:38:08 BLATHER ZWiki upgraded and pre-rendered page 2/702 JumpTo
2006-10-14 14:38:08 BLATHER ZWiki upgraded RecentChanges
2006-10-14 14:38:09 BLATHER ZWiki upgraded and pre-rendered page 3/702 RecentChanges
2006-10-14 14:38:09 BLATHER ZWiki upgraded page_type attribute of RemoteWikiLinks
2006-10-14 14:38:09 BLATHER ZWiki upgraded RemoteWikiLinks
2006-10-14 14:38:10 BLATHER ZWiki upgraded and pre-rendered page 4/702 RemoteWikiLinks
2006-10-14 14:38:10 BLATHER ZWiki upgraded page_type attribute of RemoteWikiURL
2006-10-14 14:38:10 BLATHER ZWiki upgraded RemoteWikiURL
2006-10-14 14:38:11 BLATHER ZWiki upgraded and pre-rendered page 5/702 RemoteWikiURL
2006-10-14 14:38:11 BLATHER ZWiki upgraded page_type attribute of SandBox
2006-10-14 14:38:11 BLATHER ZWiki upgraded SandBox
2006-10-14 14:38:11 BLATHER ZWiki upgraded and pre-rendered page 6/702 SandBox
2006-10-14 14:38:12 BLATHER ZWiki upgraded SearchPage
2006-10-14 14:38:13 BLATHER ZWiki upgraded and pre-rendered page 7/702 SearchPage
2006-10-14 14:38:13 BLATHER ZWiki upgraded page_type attribute of StructuredText
2006-10-14 14:38:13 BLATHER ZWiki upgraded StructuredText
2006-10-14 14:38:13 BLATHER ZWiki upgraded and pre-rendered page 8/702 StructuredText
2006-10-14 14:38:13 BLATHER ZWiki upgraded page_type attribute of StructuredTextExample

...

2006-10-14 14:38:50 BLATHER ZWiki upgraded placeless
2006-10-14 14:38:50 BLATHER ZWiki renaming placeless (placeless) to placeless (Placeless)...
/usr/lib/zope2.9/lib/python/OFS/subscribers.py:74: DeprecationWarning: Products.ZWiki.ZWikiPage.ZWikiPage.manage_beforeDelete is deprecated and will be removed in Zope 2.11, you should use event subscribers instead, and meanwhile mark the class with <five:deprecatedManageAddDelete/>
  DeprecationWarning)
2006-10-14 14:38:51 ERROR Zope.ZCatalog uncatalogObject unsuccessfully attempted to uncatalog an object with a uid of /wiki.zope.org/zope3/placeless.
/usr/lib/zope2.9/lib/python/OFS/subscribers.py:74: DeprecationWarning: Products.ZWiki.ZWikiPage.ZWikiPage.manage_afterAdd is deprecated and will be removed in Zope 2.11, you should use event subscribers instead, and meanwhile mark the class with <five:deprecatedManageAddDelete/>
  DeprecationWarning)
2006-10-14 14:38:51 BLATHER ZWiki indexing /wiki.zope.org/zope3/Placeless
2006-10-14 14:38:51 BLATHER ZWiki indexing /wiki.zope.org/zope3/Placeless
2006-10-14 14:38:52 BLATHER ZWiki rename complete
2006-10-14 14:38:52 BLATHER ZWiki upgraded and pre-rendered page 58/702 Placeless

...

2006-10-14 14:42:08 BLATHER ZWiki upgraded page_type attribute of NetAid
2006-10-14 14:42:08 BLATHER ZWiki upgraded NetAid
2006-10-14 14:42:08 BLATHER ZWiki failed to upgrade page 180/702 NetAid: Traceback (most recent call last):
  File "/zope1/Products/ZWiki/Admin.py", line 83, in upgradeAll
    p.preRender(clear_cache=1)
  File "/zope1/Products/ZWiki/ZWikiPage.py", line 280, in preRender
    self.setPreRendered(self.pageType().preRender(self))
  File "/zope1/Products/ZWiki/pagetypes/rst.py", line 45, in preRender
    t = self.format(t)
  File "/zope1/Products/ZWiki/pagetypes/rst.py", line 36, in format
    initial_header_level=RST_INITIAL_HEADER_LEVEL-1
  File "/usr/lib/zope2.9/lib/python/reStructuredText/__init__.py", line 137, in HTML
    settings = settings)
  File "/usr/lib/zope2.9/lib/python/reStructuredText/__init__.py", line 90, in render
    config_section='zope application')
  File "/usr/lib/zope2.9/lib/python/docutils/core.py", line 407, in publish_parts
    enable_exit_status=enable_exit_status)
  File "/usr/lib/zope2.9/lib/python/docutils/core.py", line 516, in publish_programmatically
    output = pub.publish(enable_exit_status=enable_exit_status)
  File "/usr/lib/zope2.9/lib/python/docutils/core.py", line 194, in publish
    self.settings)
  File "/usr/lib/zope2.9/lib/python/docutils/readers/__init__.py", line 65, in read
    self.input = self.source.read()
  File "/usr/lib/zope2.9/lib/python/docutils/io.py", line 307, in read
    return self.decode(self.source)
  File "/usr/lib/zope2.9/lib/python/docutils/io.py", line 104, in decode
    raise UnicodeError(
UnicodeError: Unable to decode input data.  Tried the following encodings: 'ascii'.
(UnicodeDecodeError: 'ascii' codec can't decode byte 0x85 in position 689: ordinal not in range(128))

...

2006-10-14 14:55:53 BLATHER ZWiki upgraded page_type attribute of ProjectStatus
2006-10-14 14:55:53 BLATHER ZWiki upgraded ProjectStatus
2006-10-14 14:55:54 BLATHER ZWiki upgraded and pre-rendered page 699/702 ProjectStatus
2006-10-14 14:55:54 BLATHER ZWiki upgraded page_type attribute of Zope330
2006-10-14 14:55:54 BLATHER ZWiki upgraded Zope330
2006-10-14 14:55:56 BLATHER ZWiki upgraded and pre-rendered page 700/702 Zope330
2006-10-14 14:55:56 BLATHER ZWiki committing
2006-10-14 14:55:56 BLATHER ZODB.FileStorage _oid2tid size 10260 lookups 16384 hits 15781 rate 96.3%
2006-10-14 14:56:10 BLATHER ZWiki upgraded page_type attribute of Zope32Maintainance
2006-10-14 14:56:10 BLATHER ZWiki upgraded Zope32Maintainance
2006-10-14 14:56:11 BLATHER ZWiki upgraded and pre-rendered page 701/702 Zope32Maintainance
2006-10-14 14:56:11 BLATHER ZWiki upgraded page_type attribute of Zope33Maintainance
2006-10-14 14:56:11 BLATHER ZWiki upgraded Zope33Maintainance
2006-10-14 14:56:11 BLATHER ZWiki upgraded and pre-rendered page 702/702 Zope33Maintainance
2006-10-14 14:56:11 BLATHER ZWiki regenerating outline data for wiki zope3
2006-10-14 14:56:13 BLATHER ZWiki upgrade complete, 702 pages processed in 797.020000s, 0.880781 pages/s