Submitted by : simon at: 2009-10-27T20:14:45-07:00 (8 years ago)
Name :
Category : Severity : Status :
Optional subject :  
Optional comment :

Reported by saper - like #1447, but different:

Traceback (innermost last):

   * Module ZPublisher.Publish, line 196, in publish_module_standard
   * Module ZPublisher.Publish, line 146, in publish
   * Module Zope2.App.startup, line 222, in zpublisher_exception_hook
   * Module ZPublisher.Publish, line 115, in publish
   * Module ZPublisher.mapply, line 88, in mapply
   * Module ZPublisher.Publish, line 41, in call_object
   * Module Products.ZWiki.Views, line 710, in editform
   * Module Products.ZWiki.Views, line 731, in editform
   * Module Shared.DC.Scripts.Bindings, line 311, in __call__
   * Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec
   * Module Products.PageTemplates.PageTemplateFile, line 113, in _exec
   * Module Products.PageTemplates.PageTemplate, line 105, in pt_render
     <PageTemplateFile at /isoc/wiki/editform used for /isoc/wiki/PrawoPrasowe>
   * Module StringIO, line 271, in getvalue

UnicodeDecodeError

He sees this with zope 2.9.9. I haven't been been able to reproduce with 2.11 or 2.12. It seems likely to be the handling of request/form/text in editform.

StringIO? unicode problem --saper, Thu, 29 Oct 2009 19:02:52 -0700 reply

http://docs.python.org/library/stringio.html says:
The StringIO? object can accept either Unicode or 8-bit strings, but mixing the two may take some care. If both are used, 8-bit strings that cannot be interpreted as 7-bit ASCII (that use the 8th bit) will cause a UnicodeError? to be raised when getvalue() is called.

This is what happens: the output.buflist[206]? (contains contents of <textarea> tag) is of type'str', not 'unicode', although there are some other type'unicode' elements. Adding talsafe() does not help, since this is noop on zope < (2,10).

Better traceback --saper, Thu, 29 Oct 2009 20:08:18 -0700 reply

Adding attempt = "".join(self.buflist) somewhere close to the end of the TAL.TALInterpreter?.FasterStringIO?.write() reveals better stacktrace:

Traceback (innermost last):
 Module ZPublisher.Publish, line 115, in publish
 Module ZPublisher.mapply, line 88, in mapply
 Module ZPublisher.Publish, line 41, in call_object
 Module Products.ZWiki.Views, line 710, in editform
 Module Products.ZWiki.Views, line 731, in editform
 Module Shared.DC.Scripts.Bindings, line 311, in __call__
 Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec
 Module Products.PageTemplates.PageTemplateFile, line 113, in _exec
 Module Products.PageTemplates.PageTemplate, line 104, in pt_render
  - <PageTemplateFile at /isoc/wiki/editform used for /isoc/wiki/PrawoPrasowe>
 Module TAL.TALInterpreter, line 238, in __call__
 Module TAL.TALInterpreter, line 281, in interpret
 Module TAL.TALInterpreter, line 749, in do_useMacro
 Module TAL.TALInterpreter, line 281, in interpret
 Module TAL.TALInterpreter, line 457, in do_optTag_tal
 Module TAL.TALInterpreter, line 442, in do_optTag
 Module TAL.TALInterpreter, line 437, in no_tag
 Module TAL.TALInterpreter, line 281, in interpret
 Module TAL.TALInterpreter, line 772, in do_defineSlot
 Module TAL.TALInterpreter, line 281, in interpret
 Module TAL.TALInterpreter, line 542, in do_insertText_tal
 Module TAL.TALInterpreter, line 839, in write
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 56: ordinal not in range(128)

fixed in darcs --simon, Fri, 30 Oct 2009 11:35:23 -0700 reply

Status: open => closed

Thanks a lot for the help tracking this down. It affects Zopes < 2.10; with those versions we now monkey-patch PageTemplateFile?'s StringIO? method to make it unicode-safe, as recommended.

Confirmed, problem solved. --saper, Fri, 30 Oct 2009 14:23:58 -0700 reply