WebDAV Contacts - Gnome Evolution

[hspcd]

Does anyone know how to properly setup Evolution so that users can access their Zimbra Contacts via WebDAV? The Zimbra connector for Evolution is not appealing since the project looks dead.

We don't get errors after adding a WebDAV address book pointing at our ZCS server but we also don't see contacts. We're running ZCS NE 6.0.2 and Evolution 2.28.

Thanks,
Clay

[gracedman]

Yes, this is a problem with the Evolution webdav contact code. It will only accept vcards with mime-type text/x-vcard but that is deprecated. Zimbra returns the more current text/vcard. There is a patch available to disable this check at https://bugzilla.gnome.org/show_bug.cgi?id=566330

We have applied it and we now see our contacts. Our next problem is that we are not able to add new contacts. Evolution returns an unhelpful "other error" message. The debug from the evolution-data-server says:

libebookbackendgoogle-WARNING **: create resource 'http://user@mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/298DA791-26023D54-54B99BB3.vcf' failed with http status: 406

Any reason why we are getting this code and how to enable write access?

By the way, we needed to use a URI of the form:

http://zimbra.mydomain.com/dav/user@...tacts/?fmt=vcf

[uxbod]

406 Not Acceptable
    The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request

When that error is received check /opt/zimbra/log/mailbox.log for any errors.

[gracedman]

Thanks for the pointer; it was very revealing. It looks like this problem is related to the malformed DNS request where the domain is placed in front of the host. Here is a packet trace of the failed contact addition:

PUT /dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf HTTP/1.1

Host: mydomain.com@zimbra.mydomain.com

User-Agent: Evolution/2.28.3.1

If-None-Match: *

Content-Type: text/x-vcard

Authorization: Basic anN1bGxpdmFuQHBhY2lmZXJhLmNvbTpNYXlhMTB5c2l1cw==

Content-Length: 434



BEGIN:VCARD

VERSION:3.0

TEL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK,VOICE:+1 (207) 123-4567

EMAIL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK:nowhere@example.com

X-MOZILLA-HTML:FALSE

X-EVOLUTION-VIDEO-URL:

FBURL:

CALURI:

X-EVOLUTION-BLOG-URL:

X-EVOLUTION-FILE-AS:Contact\, Evolution

N:Contact;Evolution;;;

FN:Evolution Contact

NOTE:

X-EVOLUTION-SPOUSE:

NICKNAME:

X-EVOLUTION-ASSISTANT:

X-EVOLUTION-MANAGER:

ROLE:

TITLE:

URL:

END:VCARDHTTP/1.1 406 Not Acceptable

Server: nginx

Date: Sat, 08 May 2010 15:40:19 GMT

Transfer-Encoding: chunked

Connection: keep-alive


Notice the host field in the put packet.

On the Zimbra side mailboxlog file we see:

[btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/] aname=user@mydomain.com;ip=172.30.10.73;ua=Evoluti on/2.28.3.1;] FileUploadServlet - Received file: name=null, size=95, id=ab0a860b-d897-49f9-8589-dca6a468556e:144feae3-c0f1-48df-9fa3-d8f3b97dc02c

2010-05-08 11:40:19,511 INFO [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evoluti on/2.28.3.1;] dav - DavServlet operation PROPFIND to /home/user@mydomain.com/Contacts/ (depth: one) finished in 16ms

2010-05-08 11:40:19,579 INFO [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evoluti on/2.28.3.1;] FileUploadServlet - Received file: name=null, size=434, id=ab0a860b-d897-49f9-8589-dca6a468556e:a0854774-b6c4-4fbe-863e-52381ac2dd21

2010-05-08 11:40:19,579 INFO [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evoluti on/2.28.3.1;] dav - sending http error 406 because: no DAV resource for folder

2010-05-08 11:40:19,579 INFO [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evoluti on/2.28.3.1;] dav - DavServlet operation PUT to /home/user@mydomain.com/Contacts/ (depth: zero) finished in 1ms

I'm going to have to go fishing through the code to find where that is as a search on Gnome bugzilla shows the bug but no patch.

[gracedman]

Hello, all. We are starting to make significant headway integrating Evolution and Zimbra. Most of the issues are with Contacts but we have one problem with Calendar. We can make new appointments but there is always a 75 second delay when we create them. Evolution completely hangs during this time.

I suspect the problem may be in our setup. We are using NGINX and an external Zimbra MTA. The query comes into the internal Zimbra server. It is then redirected to the SMTP gateway which then proxies back to the internal server. The 75 second delay appears to be introduced by the SMTP gateway. It gets the DAV PUT and the appointment is created immediately but then it stops talking to Evolution for 75 seconds as we see in the simultaneous packet traces we took on the client, the internal Zimbra server, and the SMTP gateway.

We are getting a message in nginx.log:

[info] 11445#0: *60623 client sent invalid method while reading client pipelined request line, client: 172.x.x.1, server: smtp.mycompany.com, request: "BEGIN:VCALENDAR"

What does this mean and is it important? Thanks - John

[uxbod]

Gracedman,

check what has been set for

su - zimbra
zmprov gd domain.com zimbraPublicServiceHostname

as the request may be going to the front-end NGINX instance before hitting the back end.

[gracedman]

We're making progress but it is a bit like trying to run through a briar patch. In addition to the patch referenced above, someone kindly provided a patch on https://bugzilla.gnome.org/show_bug.cgi?id=604650 which corrects the malformed URI.

We no longer need the bogus entry in /etc/hosts for mycompany.com@zimbra.mycompany.com and things are generally much cleaner. However, we still have the 75 second delay and now have it when creating new contacts as well as new appointments. Once the appointments is created, Evolution does a DAV GET (I assume to verify creation). However, after the creation, Evolution sends an ACK packet acknowledging the 201 creation notification but then Zimbra just sits for 75 seconds before closing the connection. Once the connection is closed, Evolution immediately does the DAV GET, Zimbra immediately responds and control returns to the user. My guess is the 75 second delay is actually a time out for inactivity on the TCP connection and the real problem is that Evolution is not closing the connection. I've posted this to the Evolution email list and we'll see what comes back. The patch also means the source must NOT contain a trailing "/". It also should not use /?fmt=vcf as this appears to be interpreted as a directory when trying to create a new contact as it generates an directory not found error message.

However, there are other issues. Although it looks like the contact is created, it is not displayed anywhere. Evolution does not display it unless one does a search for it. ZWC does not display it even if one searches for it yet Zimbra says it is created. Here is an excerpt from mailbox.log:

ua=Evolution/2.28.3.1;] FileUploadServlet - Received file: name=null, size=95, id=ab0a860b-d897-49f9-8589-dca6a468556e:02538e74-e259-4da3-bd7f-f704e2511857

ua=Evolution/2.28.3.1;] dav - DavServlet operation PROPFIND to /home/user@mycompany.com/Contacts/ (depth: one) finished in 18ms

ua=Evolution/2.28.3.1;] FileUploadServlet - Received file: name=null, size=421, id=ab0a860b-d897-49f9-8589-dca6a468556e:3ffe5b8e-dcc8-4152-a00a-4135f3b6df25

ua=Evolution/2.28.3.1;] dav - DavServlet operation PUT to /home/user@mycompany.com/Contacts/60033751-7FECC549-35868C16.vcf (depth: zero) finished in 32ms

Any idea why the contact is not being displayed?

Finally, free/busy is not working. It appears that it is not even trying but we have not determined that definitively yet.

[gracedman]

SLOWLY getting closer to resolution. We can now see the newly created contacts in Evolution but they do NOT appear in the Zimbra Web Client. We do see them in the database. There are a few differences. The metadata reflects that it was created via vcard. Most noticeably, the type is different. All of our other contacts including VCF cards imported from emails in ZWC are type 6. The ones created by Evolution are type 8.

What is the difference between a type 6 and a type 8?

I tried changing the entry manually in mysql but then ZWC generated a java error while accessing the Address Book.

There may be other differences but that's the one which jumps out. I also noticed that none of the type 8 contacts have synchronized with our Blackberries (running NotifySync with which we are very happy). Here is a comparison of a database entry from a vcard imported via ZWC and three other contacts - two created directly in Evolution and the last created in Evolution by importing an attached VCF file:

| 5 | 95020 | 6 | NULL | 7 | 95020 | 95020 | 1273564396 | 0 | NULL | NULL | NULL | 0 | 0 | Oskar Andreasson | NULL | NULL | d3:fldd5:email21:someone@gmail.com6:fileAs1:29:fir stName5:Oskar8:fullName16:Oskar Andreasson8:lastName10:Andreasson11:vcardXProps268 :{"X-EVOLUTION-MANAGER":"","X-MOZILLA-HTML":"FALSE","X-EVOLUTION-SPOUSE":"","X-EVOLUTION-VIDEO-URL":"","X-EVOLUTION-LAST-USE":"2004-01-14","X-EVOLUTION-USE-SCORE":"2.441834","X-EVOLUTION-FILE-AS":"Andreasson, Oskar","X-EVOLUTION-ASSISTANT":"","X-EVOLUTION-BLOG-URL":""}e1:vi10ee | 538490 | 1273564396 | 538490 |
<snip>
mysql> select * from mail_item where type=8;
<snip>
| mailbox_id | id | type | parent_id | folder_id | index_id | imap_id | date | size | volume_id | blob_digest | unread | flags | tags | sender | subject | name | metadata | mod_metadata | change_date | mod_content |
<snip>
| 5 | 95015 | 8 | NULL | 32959 | 95015 | 95016 | 1273562162 | 419 | 1 | +ochiEApLrmaTzyBTLUKo9VvAEk= | NULL | 0 | 0 | NULL | 5514ED0A-6E3B5032-56DCD56A.vcf | 5514ED0A-6E3B5032-56DCD56A.vcf | d2:cr22:me@mycompany.com2:ct14:text/directory1:f145:BEGIN:VCARD
VERSION:3.0
TEL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK,VOICE:+44 9867898757
EMAIL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK:crutch@mycompany.net ...2:ua18:Evolution/2.28.3.11:vi10ee | 538466 | 1273562434 | 538454 |

| 5 | 95017 | 8 | NULL | 7 | 95017 | 0 | 1273563955 | 433 | 1 | n0G0dHqPyAtC2CSjvcsUFC1i2kE= | NULL | 0 | 0 | NULL | 67EEC11E-50F1AEF2-40E65C16.vcf | 67EEC11E-50F1AEF2-40E65C16.vcf | d2:cr22:me@mycompany.com2:ct14:text/directory1:f147:BEGIN:VCARD
VERSION:3.0
TEL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK,VOICE:+1 (207) 222-3456
EMAIL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK:noone@mycompany.net ...2:ua18:Evolution/2.28.3.11:vi10ee | 538483 | 1273563955 | 538483 |

| 5 | 95021 | 8 | NULL | 7 | 95021 | 0 | 1273564514 | 149 | 1 | HEMk6nrh31lFMzTEwStZ8q1yfVk= | NULL | 0 | 0 | NULL | 507BACE6-404DD0AC-0A1E3C7F.vcf | 507BACE6-404DD0AC-0A1E3C7F.vcf | d2:cr22:me@mycompany.com2:ct14:text/directory1:f149:BEGIN:VCARD
VERSION:3.0
FN:Michael Alexeev
N:Alexeev;Michael;;;
X-EVOLUTION-FILE-AS:Alexeev\, Michael
EMAIL:him@gmail.com
END:VCARD2:ua18:Evolution/2.28.3.11:vi10ee | 538510 | 1273564514 | 538510 |

We'd dearly like to get this working. Then I can finally go to bed and sleep like a normal human being!

[gracedman]

A little more digging turned up this article:
Ajcody-Mysql-Topics - Zimbra :: Wiki
With this table:
/** Item is a standard {@link Folder}. */
public static final byte TYPE_FOLDER = 1;
/** Item is a saved search {@link SearchFolder}. */
public static final byte TYPE_SEARCHFOLDER = 2;
/** Item is a user-created {@link Tag}. */
public static final byte TYPE_TAG = 3;
/** Item is a real, persisted {@link Conversation}. */
public static final byte TYPE_CONVERSATION = 4;
/** Item is a mail {@link Message}. */
public static final byte TYPE_MESSAGE = 5;
/** Item is a {@link Contact}. */
public static final byte TYPE_CONTACT = 6;
// /** Item is a {@link InviteMessage} with a <tt>text/calendar</tt> MIME part. */
// public static final byte TYPE_INVITE = 7; // SKIP 7 FOR NOW!
/** Item is a bare {@link Document}. */
public static final byte TYPE_DOCUMENT = 8;
/** Item is a {@link Note}. */
public static final byte TYPE_NOTE = 9;
/** Item is a memory-only system {@link Flag}. */
public static final byte TYPE_FLAG = 10;
/** Item is a calendar {@link Appointment}. */
public static final byte TYPE_APPOINTMENT = 11;
/** Item is a memory-only, 1-message {@link VirtualConversation}. */
public static final byte TYPE_VIRTUAL_CONVERSATION = 12;
/** Item is a {@link Mountpoint} pointing to a {@link Folder},
* possibly in another user's {@link Mailbox}. */
public static final byte TYPE_MOUNTPOINT = 13;
/** Item is a {@link WikiItem} */
public static final byte TYPE_WIKI = 14;
/** Item is a {@link Task} */
public static final byte TYPE_TASK = 15;
/** Item is a {@link Chat} */
public static final byte TYPE_CHAT = 16;

So it looks like Zimbra is interpreting the vcf as a document rather than a contact. How do we fix that?

I noticed in REST documentation that one can specify:
dav/user@domain/FolderName?fmt=vcf to tell the parser that the data is to be interpreted as a vcard. However, we cannot do that in Evolution because it appends the card to the URI and we wind up with an invalid URI such as:

dav/user@domain/FolderName?fmt=vcf/somecard.vcf

I would hope that Zimbra would recognize the data is in vcf format because of the extension. Evolution is sending:
dav/user@domain/FolderName/somecard.vcf

Any ideas? Thanks - John

[gracedman]

The wrestling match continues! From what we can tell, Zimbra is misinterpreting the .vcf file. We tested using curl to send REST calls directly. We had some interesting results. If we sent a curl command formated as per the REST documentation (ZCS 6.0:Zimbra REST API Reference:Import Contacts - Zimbra :: Wiki) but using /dav/ instead of /home/, the import fails. If we use /home/, it succeeds. If we change Evolution to try to use /home/ instead of /dav/, it fails to read the contacts.

Evolution does not add the ?fmt-vcf. In fact, doing so creates a problem because it embeds it as part of the URI and we get something like dav/user@domain/FolderName?fmt=vcf/somecard.vcf as mentioned previously. But it does send the file with a .vcf extension.

I then tried the curl import by sending a file with a .vcf extension but not specifying ?fmt=vcf. The result was different from Evolution but it was also broken. Rather than create a new mail_item with type 8, it created multiple mail_item entries all with type 6 but each one had the metadata of one line of the vcard, i.e., it interpreted each newline character as a separate record.

It would seem it to behove the Contact parser to not only look for an explicit format parameter but also to naturally interpret .vcf files as vcards rather than csv files or Zimbra documents.