Submitted by : nobuyuki-koyoshi at: 2007-09-04T13:09:48-07:00 (10 years ago)
Name :
Category : Severity : Status :
Optional subject :  
Optional comment :

I had this error message when I saved page with non-ascii characters.

Traceback message:

Traceback (innermost last):

Module ZPublisher.Publish, line 202, in publish_module_standard
Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish
Module ZPublisher.Publish, line 150, in publish
Module Zope2.App.startup, line 221, in zpublisher_exception_hook
Module ZPublisher.Publish, line 119, in publish
Module ZPublisher.mapply, line 88, in mapply
Module ZPublisher.Publish, line 42, in call_object
Module Products.ZWiki.Editing, line 313, in edit
Module Products.ZWiki.Editing, line 134, in create
Module Products.ZWiki.Editing, line 425, in handleRename
Module Products.ZWiki.Editing, line 633, in rename
Module Products.ZWiki.Editing, line 675, in changeIdPreservingCreator
Module OFS.CopySupport, line 339, in manage_renameObject
Copy Error

I think this error is a same on 0.59.0, maybe utf-8 decode of wiki name?? (#1365, #714 and #1330)

... --Simon Michael, Tue, 04 Sep 2007 14:06:56 -0700 reply

Thanks for the report. Do you see this with Zwiki 0.60rc1 ?

... --nobuyuki-koyoshi, Tue, 04 Sep 2007 18:59:46 -0700 reply

Thank you. Yes, I see this with Zwiki 0.60rc1. But wiki every pages are not broken. I can see every pages from subtopics or contents link.

subtopics:
 全角文字のWiki名のIE動作
  å

¨è§’文字のWiki名のIE動ä½

å ¨è§’æ–‡å­—ã®Wiki名のIEå‹•ä½ = X_c3_a5_c2_85_c2_a8_c3_a8_c2_a7_c2_92_c3_a6_c2_96_c2_87_c3_a5_c2_ad_c2_97_c3_a3_c2_81_c2_aeWiki_c3_a5_c2_90_c2_8d_c3_a3_c2_81_c2_aeIE_c3_a5_c2_8b_c2_95_c3_a4_c2_bd_c2_9c

全角文字のWiki名のIE動作

``å

¨è§’文字のWiki名のIE動ä½``

... --nobuyuki-koyoshi, Tue, 04 Sep 2007 19:08:22 -0700 reply

全角文字のWiki名のIE動作:

[http://zwiki.org/X_e5_85_a8_e8_a7_92_e6_96_87_e5_ad_97_e3_81_aeWiki_e5_90_8d_e3_81_aeIE_e5_8b_95_e4_bd_9c]?

garbled characters name page name URL :

X_c3_a5_c2_85_c2_a8_c3_a8_c2_a7_c2_92_c3_a6_c2_96_c2_87_c3_a5_c2_ad_c2_97_c3_a3_c2_81_c2_aeWiki_c3_a5_c2_90_c2_8d_c3_a3_c2_81_c2_aeIE_c3_a5_c2_8b_c2_95_c3_a4_c2_bd_c2_9c

encoding --simon, Wed, 05 Sep 2007 09:22:35 -0700 reply

Some of your examples on this page look corrupted to me, I think because they use Shift-JIS encoding. Current Zwiki requires that you use UTF-8 encoding always, like this:

[http://zwiki.org/X_e5_85_a8_e8_a7_92_e6_96_87_e5_ad_97_e3_81_aeWiki_e5_90_8d_e3_81_aeIE_e5_8b_95_e4_bd_9c]?

This should always display correctly in page bodies, page names, bracket links, contents, recent changes etc. The url will be horrible though (http://zwiki.org/[http://zwiki.org/X_e5_85_a8_e8_a7_92_e6_96_87_e5_ad_97_e3_81_aeWiki_e5_90_8d_e3_81_aeIE_e5_8b_95_e4_bd_9c] - better ideas welcome!).

With your browser set to utf-8 encoding, can you reproduce the error ? On this site ?

Yes, using UTF-8 encoding in this test --koyoshi, Tue, 11 Sep 2007 08:14:58 -0700 reply

Oh! Thank you Simon for your comments.

My Test report's evironments are used UTF-8. Zope is not able to use Shift_JIS encording for Japaese language. About 5 years ago, I was using EUC-JP with Zope-2.5.1. But now I am using UTF-8 with Zoepe-2.8.0 or laters for Internationalizing uses. May be, almost of Japanese zope users are using UTF-8 now.

Now, I have some questions. Is Zwiki.org site drived with Zope-2.10.x? And if with Zope-2.10.x, I want to testing in yor site's SandBox for this problem.

May I do it? At least, There is no risk in doing.(I think.)

I have two testing envoronments now. 2 server gived the same result. I am unable to find an effective move ,now.

  1. Server1
  • OS: RHEL5 x68_64 ,Zope-2.10.4, ZWiki-0.60.0rc1, PlacelessTranslationService?-1.2.1
  1. Server2
  • OS: Wimdows2000Pro ,Zope-2.10.3, ZWiki-0.60.0rc1, PlacelessTranslationService?-1.2.1

# Thanks. And excuse me. I can't put two sentences together. I have a problem with English expressions. I'm a native Japanese.

Yes, using UTF-8 encoding in this test --Simon Michael, Tue, 11 Sep 2007 08:19:59 -0700 reply

No problem. Yes, please test on this wiki, it won't do any harm. Yes, Zwiki is compatible with Zope 2.10.

Yes, using UTF-8 encoding in this test --simon, Sat, 15 Sep 2007 10:30:52 -0700 reply

Hi Koyoshi (please let me know if I used the wrong name). To answer your question more precisely: yes zwiki.org is running Zope 2.10.4.

As I understand it right now:

Am I understanding correctly ? Thanks.

Almost correctly ( + supplemental report) --koyoshi, Sun, 16 Sep 2007 21:58:17 -0700 reply

Hi Simon. Thank you very much.

> * the problem is only when using IE6 (what about IE7 ?)

only when using IE6 (not IE7) with charset=utf-8(auto selected)

> * the problem is only with non-ascii characters in the page name (non-ascii characters page body works fine)

Yes This ploblem is only with non-ascii characters name page, when it's edited. ( and non-ascii characters page body works fine and have nothing changes.)

> * on your servers, when you edit a non-ascii-named page in IE6, you get the above error

Yes, I get the error 100% of the time. when do it.

> * when you do the same thing on zwiki.org, it works ? but renames the page to http:/zwiki.org/.... ?

Now, when I test on zwiki.org. zwiki.org work normally. but when I tested on zwiki.org at last week. zwiki.org renamed an ediing page as http://zwiki.org/X_e5_85_a8_e8_a7_92_e6_96_87_e5_ad_97_e3_81_aeWiki_e5_90_8d_e3_81_aeIE_e5_8b_95_e4_bd_9c. There was a liitle difference between zwiki.org and my servers. My servers never rename editing pages. and always make a child page that named unreadable and missing decode characters.

If there is no difference between zwiki.org and my servers on version zwiki ,Zope and PTS. I think this problem is on only my servers setup. then, I tried to change PTS version(1.2.1 => 1.3.5) and setting of zope.conf. But, I can not get a way.I was wondering if I could ask you a favor. let me know why do zwiki retry to decode existing page name.

Some simple examples follow.

I think probably this is a common problem in multi-byte character languages(ex. Chinese,Korean and Japanese).

thanks.

... --simon, Thu, 20 Sep 2007 08:42:34 -0700 reply

Name: '#1376 I can not edit non-ascii name page of ZWiki 0.60rc1(Zope2.10, IE)' => '#1376 Copy error when saving a non-ascii-named page in IE6'

Almost correctly ( + supplemental report) --simon, Thu, 20 Sep 2007 08:50:14 -0700 reply

Ok. I think you will see the same problem in a simple non-plone wiki. If so we can forget about plone and PTS versions. I'm going to forget about the one-time weird page rename on zwiki.org for now. Another difference between zwiki.org and your server could be character encoding. Eg:

your question --simon, Thu, 20 Sep 2007 08:52:06 -0700 reply

Re "let me know why do zwiki retry to decode existing page name" - sorry, I don't yet understand what you mean ?

... --nobuyuki-koyoshi, Sat, 06 Oct 2007 21:52:47 -0700 reply

I'm very sory, I'm ashamed to say, I can't too this english sentence, sory. In other words, I don't understand working of edit() in Editing.py. Why It makes Page X_c3_af_c2_bc_c2_a1 when editing a page ef_bc_a1 ?

After I commented, I found messages in log/event.log. example:

2007-10-04T04:27:23 ERROR Zope.SiteErrorLog http://localhost:8080/MiRe/X_e3_83_86_e3_82_b9_e3_83_88ZWiki_e6_97_a5_e6_9c_ac_e8_aa_9e_e5_90_8d/update
Traceback (innermost last):
  Module ZPublisher.Publish, line 119, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module Products.ZWiki.Editing, line 333, in edit
  Module Products.ZWiki.Editing, line 425, in handleRename
  Module Products.ZWiki.Editing, line 633, in rename
  Module Products.ZWiki.Editing, line 675, in changeIdPreservingCreator
  Module OFS.CopySupport, line 339, in manage_renameObject
Copy Error:
<HTML>
<HEAD>
<TITLE>Invalid Id</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF">
<FORM ACTION="manage_main" METHOD="GET" >
<TABLE BORDER="0" WIDTH="100%" CELLPADDING="10">
<TR>
  <TD VALIGN="TOP">
  <BR>
  <CENTER><B><FONT SIZE="+6" COLOR="#77003B">!</FONT></B></CENTER>
  </TD>
  <TD VALIGN="TOP">
  <BR><BR>
  <CENTER>
  The id "X_e3_83_86_e3_82_b9_e3_83_88ZWiki_e6_97_a5_e6_9c_ac_e8_aa_9e_e5_90_8d" is invalid - it is already in use.
  </CENTER>
  </TD>
</TR>
<TR>
  <TD VALIGN="TOP">
  </TD>
  <TD VALIGN="TOP">
  <CENTER>
  <INPUT TYPE="SUBMIT" VALUE="   Ok   ">
  </CENTER>
  </TD>
</TR>
</TABLE>
</FORM>
</BODY></HTML>

when I clicked Edit botton on http://localhost:8080/MiRe/X_e3_83_86_e3_82_b9_e3_83_88ZWiki_e6_97_a5_e6_9c_ac_e8_aa_9e_e5_90_8d/editform as zwiki page of テストZWiki日本語名.

Please, look line 27 The id "X_e3_83_86_e3_82_b9_e3_83_88ZWiki_e6_97_a5_e6_9c_ac_e8_aa_9e_e5_90_8d" is invalid - it is already in use..

p.handleRename(), in Editing.py 333 line, rised this eror message.

Then, I tryed to add try: and except:pass in this line for analysis. and I do so others. I regret to say I can't get good results.

my environments was changed.

  • OS: RHEL5 x68_64 ,Zope-2.10.4, ZWiki-0.60.0, PlacelessTranslationService-1.4.0
  • OS: Windows2000Pro ,Zope-2.10.4, ZWiki-0.60.0, PlacelessTranslationService-1.4.0

thanks.

At this time, my ad hok patch for analysis --koyoshi, Tue, 09 Oct 2007 00:34:58 -0700 reply

It enabled to save a non-ascii-named page. But, created a garbled name page.

  1. Stop Raise Exception CopyError

## 333 line Editting.py edit():

try:
    p.handleRename(title,leaveplaceholder,updatebacklinks,REQUEST,log)
except:
    pass
  1. Stop Create a garbled name Page again

## 307 line Editting.py edit():

try:
    return self.create(page,
                   text or '', # string expected here
                   type,
                   title,
                   REQUEST,
                   log,
                   subtopics=subtopics) # creating a page  ##No.02
except:
    pass
  1. Save a non-ascii-named page that edited : probably dengerous

## 307 line add Editting.py edit():

p = self

Untested patch 2007-10-15 --nobuyuki-koyoshi, Sun, 14 Oct 2007 09:25:58 -0700 reply

tal:attributes="value page" 49 line editpage.pt is not safty. It like #1330's patch .

I think this patch is a strong candidate for this issue. I'll try to test other Zope servers in some days.

Could anyone try to apply this patch on your test zope servers?

UNTESTED PATCH:

Q:\InstallFiles\ZWiki\ZWiki-0.60.0\ZWiki\skins\zwiki>diff -du editform.pt editform_my_patch.pt
--- editform.pt 2007-09-16 07:21:56.000000000 +0900
+++ editform_my_patch.pt       2007-10-15 00:34:30.000000000 +0900
@@ -46,7 +46,7 @@
           <input name="timeStamp" type="hidden" value=""
                  tal:attributes="value timestamp" />
           <input name="page" type="hidden" value="page"
-                 tal:attributes="value page" />
+                 tal:attributes="value python:here.talsafe(page)" />
           <input name="leaveplaceholder" value="" type="hidden"
                  tal:condition="python:creating #no placeholder when creating" />
           <div align="center">
@@ -222,4 +222,4 @@
       </div>
     </div>
   </body>
-</html>
+</html>
\ No newline at end of file

Untested patch 2007-10-15 --betabug, Sun, 14 Oct 2007 10:16:45 -0700 reply

Tested your patch with IE6 (W2k), Safari and Firefox (on Mac), it does not break anything. I don't know if it fixed anything, because I could not reproduce the problem before.

That hidden input field should not really contain values that are not ascii, e.g. after rendering it should be something like value="X_e3_83_86_e3_82_b9_e3_83_88" - but maybe that "should" is the bug.

... --nobuyuki-koyoshi, Mon, 15 Oct 2007 09:17:45 -0700 reply

Hi Betabug thank you for your quickly testing and comment.

I had thought so, that you said. But, this patch is working perfectly with my two test servers(W2K,RHLE5). Sure, It not make CopyError and garbled named page with IE6,firefox and seamonkey.

I wonder how it comes that. I feel a need for analysis. network packets, configuration of zope and os. Please, Give me one week to do.

thanks.

committed to darcs --simon, Thu, 01 May 2008 20:42:53 -0700 reply

Status: open => closed

Ah, that last fix looks good. I have committed to both darcs repos. Thank you very much!