Zwiki Developer’s Guide

This guide is to explain current Zwiki development practices and to collect helpful guidelines.

Zwiki repository policies

Here are some guidelines for committers to the main Zwiki repository.

File layout

See README for a list of documentation files.

Directory overview:

ZWiki/ main source files
plugins/ functionality-adding plugins
tracker/ zwiki issue tracker plugin
rating/ zwiki page rating plugin
latexwiki/ page types supporting LaTeX
mathaction/ page types supporting math tools
tools/ administrator command-line tools
pagetypes/ standard zwiki page types
Extensions/ external methods
scripts/ example zope/zwiki scripts
i18n/ translation files
skins/ skin templates & skin-based content
zwiki/ the standard zwiki skin for zope & plone
wikis/ wiki templates
basic/ the standard pages and scripts for a new wiki
profiles/ CMF/Plone GenericSetup configuration

Currently,

  • there is no separate tests directory. Each source file has a companion file with _tests appended to the name, in the same directory. This sometimes is cluttersome but it makes tests easier to find, grep, and makes plugin tests easier.
  • similarly, there is no separate docs directory. Documentation files are kept in the most appropriate directory, with ALLCAPITALIZED names to make them stand out.
  • most source files have capitalized names. Some newer files have lower-case names.

Documentation guidelines

Most Zwiki documentation is kept in the http://zwiki.org wiki. Developer-specific docs are under http://zwiki.org/DeveloperNotes. Developer documentation is to some extent moving to files within the codebase, like this file. These typically use restructured text markup. To browse these filesystem docs online, use http://zwiki.org/repos/ZWiki or the darcsweb interface.

Filesystem vs wiki docs

Filesystem docs:

  • faster for one developer wielding a text editor
  • or several devs who are good with darcs
  • revision controlled - secure, trackable
  • can work offline
  • can get out of sync, uncommited changes here and there
  • can be auto-generated - epydoc, test output, author stats etc.
  • can be published (but not edited directly) online
  • less visible and less pretty for readers
  • better than wiki for large single documents

current uses:

  • CHANGES - release notes, also published at zwiki.org/ReleaseNotes
  • CONTRIBUTORS - official contributor list
  • GPL, LICENSE - license and copyright status
  • REPOPOLICY - copyright and other developer policies for this repo
  • README_tests - developer test documentation
  • README - basic product info
  • TODO - project plans & todos
  • other READMEs - basic info for each directory
  • plugins/latexwiki/LICENSE.txt,NOTES.txt,README.txt - latexwiki-specific copyright info, dev notes, admin/user notes
  • plugins/mathaction/INSTALL.txt,LICENSE.txt,Notes,README.txt,UPGRADE - needs consolidation like the above
  • TESTRESULTS - unit test output updated hourly, published at zwiki.org/TestResults
  • misc/ - public notes & snippets, published but not committed
  • MYNOTES/ - simon’s notes & snippets, not published
  • MYNOTES/RelNotesTemplate - template for CHANGES

Wiki docs:

  • slower to view and slower to edit
  • more accessible and more editable for a wide audience
  • one master copy used by everyone - simple
  • no special tools needed
  • email integration - notification and mailin
  • good for indexing and searching many documents (local and google search)
  • formatting aids - table of contents, headings, tables
  • more effort to protect and maintain; more vulnerable to decay
  • more expensive to host - every page takes memory & speed
  • more public - indexed quickly, for ever
  • can organize a large number of documents in a hierarchy

current uses:

  • zwiki.org, including
  • NextReleaseNewsDraft - unused
  • GeneralDiscussion
  • InternationalisationDiscussion
  • IssueTracker
  • plone.demo.zwiki.org
  • free zwikis, including kpug zope3 docs, leo, etc.
  • zopewiki.org, joyful.com etc.

Thoughts:

  • use filesystem for developers, wiki for users/admins/general public
  • use text files anywhere for fast developer notes
  • one requirement, keep them current - but what to do with old/temporary notes ?
  • no special docs directory, just upper-case, no suffix
  • publish all filesystem docs online, like rdoc or within wiki (they will be googlable ?)
  • restructured text and moving docs between FS and wiki encouraged
  • wiki needs to be fast

Code docs

We have the following kinds of in-code documentation:

  • ALLCAPS documentation files in any directory, like this README

  • File docstrings

  • Class docstrings

  • Method/function docstrings

  • Comments

    • comments are sometimes used instead of the docstrings above. This may or may not be appropriate, I don’t know.
    • a section heading within a file, class or method.
    • a short explanatory comment appended to a line of code.
    • longer explanatory comments when necessary.

    The Zwiki code has a lot of chatty comments. Nowadays we should try to minimise their use eg by making them unnecessary or by using intention-revealing names.

  • Names

    Names of directories, files, classes, methods, variables are where documentation meets code. They are perhaps our most powerful tool. :)

    Use clarifying and intention-revealing names wherever possible to minimise the need for other documentation.

    Choosing good names for small chunks of code helps us to build up our own mini-language (DSL) for Zwiki-building, which helps us to say more with less code.

Conciseness

Concise, not over-verbose, non-redundant docs are good. To help us write better docs, we could try to follow this length guideline when possible:

doc files any length
file docstrings one paragraph
class docstrings one paragraph
method docstrings one sentence, plus one per argument, or as needed
comments one sentence or fragment, on one line

Docstrings

The first sentence should be a concise summary of the object’s purpose.

Older docstrings follow the PEP guideline to separate the first line from the rest with a blank line, but we have abandoned this to save vertical space.

It may sometimes be ok to omit the line breaks at the docstring’s start and end if it doesn’t hurt readability.

Describe method arguments and return values when needed. We might want to come up with a standard haskellish way of noting the expected input and output types and whether side effects are expected.

Function/method contracts

We are considering ways to document function and method “contracts”. Here’s an example. We are most interested in

  • their input (argument) and output (return value) types
  • their degree of statefulness/statelessness. Are they
    • pure functions depending only on the arguments ?
    • well-behaved methods which depend on/modify only self ?
    • or do they depend on/modify other pages, the wiki folder, other things ?

Understanding statefulness in particular should help us to identify and gather together the most stateful code, reduce state dependencies, and increase the proportion of pure and semi-pure functional code, which will greatly aid testing, debugging and reliability.

Here is a possible convention for now: when you touch a method or function, review the docstring and code and try add a comment describing the contract. The contract specifies the return type(s), whether it redirects, state that it depends on other than the arguments, state that it modifies other than the return value, and any other effects. Here are some examples:

def method(args) # -> return type(s) [; redirects] [; depends on: ...] [; modifies: ...] [; other effects: ...]

def htmlquote(self, text): # -> string

def asAgeString(self,time): # -> string | empty string; depends on: self (for current time)

def handleSubtopicsProperty(self,subtopics,REQUEST=None): # -> none ; modifies: self

def revisionNumberBefore(self, username): # -> revision number | none
    # depends on: self, revisions

def expungeEditsEverywhereBy(self, username, REQUEST=None, batch=0): # -> none
    # depends on: all pages, revisions ; modifies: all pages, revisions

def upgradeAll(self,render=1,batch=0,REQUEST=None): # -> none
    # depends on: wiki
    # modifies: wiki (folder, pages, dtml methods, catalog, outline, revisions..)

A few more notes:

These don’t have to be perfectly correct; even a rough guess could be useful. Do review the contract for correctness every time you change some code. Contracts should ideally be cumulative, ie reflect the contracts of methods called by this one - looking at code one or two levels deep should be enough. Very common features like the dependency on self for permission checking are omitted. It may be better to keep contracts on one line for easy scanning, even if long. Things we depend on or modify include: self (usually the current page object), folder (the wiki folder properties and acquired context), wiki (the folder and all pages in it), catalog, outline, revisions, request.

Code guidelines

In general, it’s best to follow, first, the current guidelines in this file, second, the style of existing Zwiki code, and third, standard Python best practices.

Vertical space is precious, use long lines

IMHO, vertical space is more valuable than horizontal space. It’s more valuable to see more lines on the screen than to see the full contents of every line. Keeping lines within 80 characters can obscure the shape of code constructs, especially with functional coding style, and also means we are less likely to see the whole method at once. Long lines, even over 80 characters, may sometimes be preferable.

This assumes you have your editor truncate long lines, not wrap them. I do this, and maximize the window when I need more visibility on the line ends.

Message passing style, functional programming style, etc.

The Zwiki codebase is influenced by Smalltalk. Eg:

  • we encapsulate state and behaviour as objects when that makes sense
  • we break up code into many small methods with intention-revealing names to form an expressive domain-specific language.
  • “tell, don’t ask”, ie just send messages, ignore what’s returned. We haven’t used this and I don’t know how it applies to us; it’s something to think about.

Recent code is also influenced by functional programming. Eg:

  • we minimise the use of variables and the use of procedural/sequential code (do this, then that, then the other). Instead, we compose functions (/methods) where possible.
  • we prefer referentially-transparent functions where possible (return value depends only on the arguments - no side effects).

Think “fewer moving parts”. Variables, imperative sequenced instructions, and side effects are often unnecessary. When we cut them out, there is less to break and the code is more reliable and more testable.

More about naming

Try to give procedures (functions/methods with side effects) verb names. Try to give variables and side-effect-free functions noun names.

Security

To specify method permissions, we sometimes use explicit zope security declarations, sometimes omit the docstring, and sometimes use the _ prefix. The latter two interfere with readability and should be phased out. We probably want to move to an all methods private by default policy, and explicity declare permissions on everything that we want to expose.

Imports

Imports are usually at the start of the file and in three groups: python, then zope, then zwiki imports.

Testing

We currently use pyunit for unit testing and a bit of doctest + mechanize for functional testing.

http://palladion.com/home/tseaver/obzervationz/2008/unit_testing_notes-20080724 has useful guidelines for improving our unit tests. Excerpt:

Tests which conform to these rules and guidelines have the following properties:

  • The tests are straightforward to write.
  • The tests yield excellent coverage of the AUT.
  • They reward the developer through predictable feedback (e.g., the growing list of dots for passed tests).
  • They run quickly, and thus encourage the developer to run them frequently.
  • Expected failures confirm missing / incomplete implementations.
  • Unexpected failures are easy to diagnose and repair.
  • When used as regression tests, failures help pinpoint the exact source of the regression (a changed contract, for instance, or an underspecified constraint).
  • Writing such tests clarifies thinking about the contracts of the code they test, as well as the dependencies of that code.