Calendar "left behind" when email account re-created?

[ypong]

[Solved: had to reboot all the servers, mailstores as well as proxies]

I have a "funny" problem here, which can be replicated very easily (Zimbra OSE 7.1.3 on Ubuntu 10.04 x64, cluster configuration as per below), using the Zimbra Admin GUI:
1) create account1@domain1; this creates the account on server1
2) move this account to a new domain, account1@domain2 (still on server1)
3) delete account1
4) create account1@domain2, on server2

at point (2), everything is still ok in terms of email and caldav access

At point (4), email is ok, but the caldav access no longer works

if I then do these steps:
5) delete account1@domain2, on server2
6) create account1@domain2, on server1

then the caldav access works again

I suspect even without the extra steps of changing domains, the problem will still occur, but I haven't had time to check this out as an extra replication step.

I checked using the zmorphan script (but running manually), and there are no orphans when comparing the output from the mysql database and the ldap directory. I also checked using zmprov getAllCalendarResources, but that only shows dedicated calendar resources, not the calendar associated with each account.

It's almost as if the deletion of the account, doesn't clean up the caldav portion of the account, in a multi-server environment?

FYI my setup is two proxies and three mailstores, with multiple domains (some dedicated on one mailstores, others spread across).

[ypong]

anybody encountered this problem?

[ypong]

I've further trimmed down the replication steps:
1) create account1@domain1; this creates the account on server1
2) delete account1
3) create account1@domain1, on server2
at point (1), everything is still ok in terms of email and caldav access

At point (3), email is ok, but the caldav access no longer works

4) delete account1@domain1, on server2
6) create account1@domain1, on server1

then the caldav access works again

Very strange.

Anybody know how to debug caldav further?

[ypong]

I did more digging around, you can find out the id and group id for a user by doing this:
mysql -Ns -e "select id, group_id from zimbra.mailbox where account_id=<whatever>"

Then, depending on the groupid, you can then run some queries against the relevant mboxgroup<ID>.aappointment table, e.g.:
SELECT * FROM mboxgroup10.appointment WHERE mailbox_id=20

Taking the item_id from this results, you can then run this query to verify that the relevant calendar items do exist on this particular mailstore:

select * from mboxgroup10.mail_item where mailbox_id=20 and id in (<item_id1>, <item_id2>, <etc>)

so I've verified that the calendar items are definitely on this mailstore. This narrows it down to perhaps a proxy issue on the nginx side?

[ypong]

and further verification, I used cadaver for testing. I used two different urls:

1. https://domain1/dav/account1@domain1
2. http://server1/dav/account1@domain1

Using (1), cadaver shows no collection
Using (2), cadaver shows the collection as expected

so if I go straight to the correct mailstore, I can find the collection. If I go via the proxy, I cannot.

And if I look in the logs:
(1)

2012-02-16 18:33:03,624 INFO  [btpool0-213://domain1/dav/account1@domain1/] [name=account1@domain1;aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] dav - DavServlet operation OPTIONS to /home/account1@domain1/ (depth: zero) finished in 8ms
2012-02-16 18:33:03,676 INFO  [btpool0-213://domain1/dav/account1@domain1/] [aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] FileUploadServlet - saveUpload(): received Upload: { accountId=d9cc0f3d-4fcf-4866-bc7f-54bea1dce0f5, time=Thu Feb 16 18:33:03 SGT 2012, size=288, uploadId=185b2c33-2af5-49ef-8778-d27e39489426:6a3e9fc7-3d8f-41f9-91fc-796b2638f75d, name=null, path=null }
2012-02-16 18:33:03,683 INFO  [btpool0-213://domain1/dav/account1@domain1/] [name=account1@domain1;aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] dav - DavServlet operation PROPFIND to /home/account1@domain1/ (depth: zero) finished in 7ms
2012-02-16 18:33:04,326 INFO  [btpool0-213://domain1/dav/account1@domain1/] [aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] FileUploadServlet - saveUpload(): received Upload: { accountId=d9cc0f3d-4fcf-4866-bc7f-54bea1dce0f5, time=Thu Feb 16 18:33:04 SGT 2012, size=288, uploadId=185b2c33-2af5-49ef-8778-d27e39489426:04e905a8-7aec-46f9-ad5a-9c648e5fe70d, name=null, path=null }
2012-02-16 18:33:04,333 INFO  [btpool0-213://domain1/dav/account1@domain1/] [aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] dav - DavServlet operation PROPFIND to /home/account1@domain1/ (depth: one) finished in 7ms

(2)

2012-02-16 18:34:10,625 WARN  [btpool0-304://server1/dav/account1@domain1/] [ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] log - Committed before 406 null
2012-02-16 18:34:17,142 INFO  [btpool0-303://server1/dav/account1@domain1/] [name=account1@domain1;aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] dav - DavServlet operation OPTIONS to /home/account1@domain1/ (depth: zero) finished in 0ms
2012-02-16 18:34:17,147 INFO  [btpool0-303://server1/dav/account1@domain1/] [aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] FileUploadServlet - saveUpload(): received Upload: { accountId=d9cc0f3d-4fcf-4866-bc7f-54bea1dce0f5, time=Thu Feb 16 18:34:17 SGT 2012, size=288, uploadId=070915f2-860e-477d-aecc-4050049e08e2:3d90c3b4-8f7a-40d3-9dbe-4f960c40328c, name=null, path=null }
2012-02-16 18:34:17,148 INFO  [btpool0-303://server1/dav/account1@domain1/] [name=account1@domain1;aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] dav - DavServlet operation PROPFIND to /home/account1@domain1/ (depth: zero) finished in 2ms
2012-02-16 18:34:18,199 INFO  [btpool0-303://server1/dav/account1@domain1/] [aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] FileUploadServlet - saveUpload(): received Upload: { accountId=d9cc0f3d-4fcf-4866-bc7f-54bea1dce0f5, time=Thu Feb 16 18:34:18 SGT 2012, size=288, uploadId=070915f2-860e-477d-aecc-4050049e08e2:ce16eb8d-a81e-4812-b652-4400073ab587, name=null, path=null }
2012-02-16 18:34:18,207 INFO  [btpool0-303://server1/dav/account1@domain1/] [aname=account1@domain1;ip=10.25.4.8;ua=cadaver/0.23.3 neon/0.29.6;] dav - DavServlet operation PROPFIND to /home/account1@domain1/ (depth: one) finished in 9ms

you can see that other than the additional line in (2) (the direct & successful retrieval by cadaver on the correct server) which is a WARN about "Committed before 406 null", the lines are pretty much identical (except the upload id).

So I have to assume this is indeed a proxy error. I've already restarted the proxy servers, and indeed also rebooted them, is there anything else I can try, to correct this problem?

[ypong]

I rebooted all the servers this morning (mailstores and proxies), and now things are working. Whew!