Submitted by : simon at: 2006-04-21T10:25:36+00:00 (11 years ago)
Name :
Category : Severity : Status :
Optional subject :  
Optional comment :

This one I can reproduce myself:

File "/zope-2.7/lib/python/ZPublisher/Publish.py", line 101, in publish
  request, bind=1)
File "/zope-2.7/lib/python/ZPublisher/mapply.py", line 88, in mapply
  if debug is not None: return debug(object,args,context)
File "/zope-2.7/lib/python/ZPublisher/Publish.py", line 39, in call_object
  result=apply(object,args) # Type s<cr> to step into published object.
File "/zope-2.7.8/lib/python/Products/ExternalMethod/ExternalMethod.py", line 225, in __call__
  try: return f(*args, **kw)
File "/zope1/Products/ZWiki/Extensions/mailin.py", line 551, in mailin
  in_reply_to=m.inreplyto
File "/zope1/Products/ZWiki/Editing.py", line 202, in comment
  t = self.pageType().preRender(self,t)
File "/zope1/Products/ZWiki/pagetypes/rst.py", line 36, in preRender
  t = self.format(t)
File "/zope1/Products/ZWiki/pagetypes/rst.py", line 27, in format
  initial_header_level=RST_INITIAL_HEADER_LEVEL-1
File "/zope-2.7.8/lib/python/reStructuredText/__init__.py", line 134, in HTML
  settings = settings)
File "/zope-2.7.8/lib/python/reStructuredText/__init__.py", line 87, in render
  config_section='zope application')
File "/zope-2.7.8/lib/python/docutils/core.py", line 407, in publish_parts
  enable_exit_status=enable_exit_status)
File "/zope-2.7.8/lib/python/docutils/core.py", line 516, in publish_programmatically
  output = pub.publish(enable_exit_status=enable_exit_status)
File "/zope-2.7.8/lib/python/docutils/core.py", line 194, in publish
  self.settings)
File "/zope-2.7.8/lib/python/docutils/readers/__init__.py", line 65, in read
  self.input = self.source.read()
File "/zope-2.7.8/lib/python/docutils/io.py", line 307, in read
  return self.decode(self.source)
File "/zope-2.7.8/lib/python/docutils/io.py", line 104, in decode
  raise UnicodeError(

UnicodeError?: Unable to decode input data. Tried the following encodings: 'utf-8'. (UnicodeDecodeError?: 'utf8' codec can't decode bytes in position 540-542: invalid data)

Relevant bits of the email:

Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

> -----Message d'origine-----
> De : ...
> Envoyé : 20 avril 2006 18:30
> À : ...

... --simon, Sat, 28 Apr 2007 03:44:57 +0000 reply

Name: '#1256 unicode error preventing mail-in' => '#1256 mailins must be utf8-encoded ?'

fixed in darcs --simon, Thu, 23 Aug 2007 12:49:18 -0700 reply

Status: open => closed

mailin now respects the Content-Type header and should be able to handle messages in iso8859-1 and other encodings.

For code review: we are now doing something like this:

payload = firstplaintextpart.get_payload(decode=1)
content_encoding = self.msg.get_content_charset('ascii') # 'ascii' is fallback
payloadutf8 = payload.decode(content_encoding).encode('utf-8')
self.body = self.cleanupBody(payloadutf8)