calendar migration failed with utf8 chars

[xingjing]

Hi,

I am migrating our institute's calendar using perl/soap. I get the old appointment info from the datebase then create soap envelops to send to zimbra's soap API. Most appointments are migrated without problem. But when appointment's organizer, attendee, location, summary or comment contain special chars, the appointment failed to get into zimbra. The following is a failed envelop:

<ns0:CreateAppointmentRequest xmlns:ns0="urn:zimbraMail">
  <m l="291">
    <inv allday="0" loc="" name="Dinner with Ângela" fb="B" type="event" transp="O">
      <or a="asmith@myschool.edu" d="Ana smith"/>
      <s d="20060309T191500"/>
      <e d="20060309T220000"/>
    </inv>
    <e p="Ana smith" a="asmith@myschool.edu" t="t"/>
    <su>Dinner with Ângela</su>
    <mp ct="multipart/alternative">
      <mp ct="text/plain">
        <content>The following is a meeting request migrated from OpenGroupware:Dinner with Ângela Time:March 9, 2006, 19:15:00 - 22:00:00 (GMT-05) America/New_York Organizer:&quot;Ana smith&quot;&lt;asmith@myschool.edu&gt; Location:Tina´s,  Resources:
 Invitees: &quot;Ana smith&quot;&lt;asmith@myschool.edu&gt;; </content>
      </mp>
      <mp ct="text/html">
        <content>&lt;h3&gt;The following is a meeting request migrated from OpenGroupware:&lt;/h3&gt;&lt;p&gt;&lt;table border=&apos;0&apos;&gt;&lt;tr&gt;&lt;th align=left&gt;Subject: &lt;/th&gt;&lt;td&gt;Dinner with Ângela&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;th align=left&gt;Organizer: &lt;/th&gt;&lt;td&gt;Ana smith&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;th align=left&gt;Location: &lt;/th&gt;&lt;td&gt;Tina´s&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;th align=left&gt;Time: &lt;/th&gt;&lt;td&gt;March 9, 2006, 19:15:00 - 22:00:00 (GMT-05) America/New_York&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;th align=left&gt;Invitees: &lt;/th&gt;&lt;td&gt;&quot;Ana smith&quot;&lt;asmith@myschool.edu&gt;; &lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/p&gt;</content>
      </mp>
    </mp>
  </m>
</ns0:CreateAppointmentRequest>

The error I got is

<ns0:Fault xmlns:ns0="http://www.w3.org/2003/05/soap-envelope">
  <ns0:Code>
    <ns0:Value>soap:Sender</ns0:Value>
  </ns0:Code>
  <ns0:Reason>
    <ns0:Text>parse error: Error on line 2 of document  : The element type &quot;ns0:Envelope&quot; must be terminated by the
 matching end-tag &quot;&lt;/ns0:Envelope&gt;&quot;. Nested exception: The element type &quot;ns0:Envelope&quot; must be term
inated by the matching end-tag &quot;&lt;/ns0:Envelope&gt;&quot;.</ns0:Text>
  </ns0:Reason>
  <ns0:Detail>
    <ns1:Error xmlns:ns1="urn:zimbra">
      <ns1:Code>service.PARSE_ERROR</ns1:Code>
      <ns1:Trace>com.zimbra.common.service.ServiceException: parse error: Error on line 2 of document  : The element type &qu
ot;ns0:Envelope&quot; must be terminated by the matching end-tag &quot;&lt;/ns0:Envelope&gt;&quot;. Nested exception: The ele
ment type &quot;ns0:Envelope&quot; must be terminated by the matching end-tag &quot;&lt;/ns0:Envelope&gt;&quot;.
Code:service.PARSE_ERROR
        at com.zimbra.common.service.ServiceException.PARSE_ERROR(ServiceException.java:201)
        at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:112)
        at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:272)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:174)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
        at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)
        at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:148)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.handler.RewriteHandler.handle(RewriteHandler.java:176)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:313)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:844)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:644)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Caused by: org.dom4j.DocumentException: Error on line 2 of document  : The element type &quot;ns0:Envelope&quot; must be terminated by the matching end-tag &quot;&lt;/ns0:Envelope&gt;&quot;. Nested exception: The element type &quot;ns0:Envelope&quot; must be terminated by the matching end-tag &quot;&lt;/ns0:Envelope&gt;&quot;.
        at org.dom4j.io.SAXReader.read(SAXReader.java:350)
        at org.dom4j.io.SAXReader.read(SAXReader.java:264)
        at com.zimbra.common.soap.Element.parseXML(Element.java:373)
        at com.zimbra.common.soap.Element.parseXML(Element.java:371)
        at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:106)
        ... 27 more
</ns1:Trace>
    </ns1:Error>
  </ns0:Detail>
</ns0:Fault>

I think it breaks at "Ângela". Basically the error says the soap envelop could not be parsed. But how can I make it work? The same thing happens whereever the special chars appeared in organizer, attendee, location, summary, mp or comment.

Any help will be greatly appreciated,

Jing

[jholder]

What version? I think there was a bug open on this, so you might have an older version. If it's 4.5.10, please search Bugzilla for it. If you can't find it, please file it.

[xingjing]

We are using zimbra version 5.0 - RC2.

I am wondering if replacing "Â" with "Â" would work. But there are so many chars I could not possibly replace them all. It got have a better way to do so.

BTW, I checked bugzilla and did not find anything helpful. Should I file a bug?

Jing

[xingjing]

I tried to replace "Â" with "\302", "\xC2", "Â", "Â", all of them get parser error. Anyone have better idea?