Z-Push Zimbra Backend (Push Email, Contacts, Appointments & Tasks)

[jonallport]

UPDATE to the UPDATE:

I've had enough and I'm giving up!

Everything's working file so long as I don't try to import my old appts back in!!!!

Must be something in there that Z-Push/MfE doesn't like. Think I'll spend a bit of time tomorrow putting my appts back in by hand and see how it goes.

Thanks to Vincent (LFC Fan) for all the help he has given me.

[liverpoolfcfan]

UPDATE:

I've (exported and) deleted all of the appts from my calendar - it syncs!
Added appts to calendar - syncs!

I re-imported the SECOND calendar - error!

I conclude that it must be a problem with handling 'extra' calendars, but this feature was working great in r36. I don't want to have to resort to user files as I'd then have to configure each user individually - much work, me no like!

I might have to fall back to Z-Push 1.3 with BEZ r3x and lose on-handset accept/tentative/decline.

So - at this stage can you confirm you have all 16 appointments from your primary canendar sync'ing and no errors

But, when you add the other calendar with the additional 4 calendar entries - it breaks - right ?

For a test, can you create a user XML file modeled on the one I sent you - but only including the main Calendar folder (id="10") under the appointments section.

The re-import your second calendar. See if it syncs OK when it is told not to include that calendar.

If it does, then edit each of the 4 appointments - one at a time - and move them to your main calendar. Then, re-sync. See if there is a particular appointment that breaks the sync.

Thanks for your patience. It would be great for everybody if we could get to the bottom of this.

Additionally, when I get to the office this afternoon - I will remove my user XML file completely and see what happens with my multiple calendars.

[jonallport]

I (think) I found the culprit!

After removing appointments and syncing on-by-one (from last August forward) the sync was successful again once I'd removed an appt with the following criteria:

Zero-width appt (0930-0930) with reminder
Created on handset (with previous Z-P/BEZ) & synced up from there

Everything's in sync on my account at the moment so if I get the chance later I'll create a new account, import the old ICS and see if it syncs. then we can play about with it without me losing my calendar (again) or all my appts moving to the L.A. timezone (again).

[liverpoolfcfan]

I (think) I found the culprit!

After removing appointments and syncing on-by-one (from last August forward) the sync was successful again once I'd removed an appt with the following criteria:

Zero-width appt (0930-0930) with reminder
Created on handset (with previous Z-P/BEZ) & synced up from there

Everything's in sync on my account at the moment so if I get the chance later I'll create a new account, import the old ICS and see if it syncs. then we can play about with it without me losing my calendar (again) or all my appts moving to the L.A. timezone (again).

Great news. Delighted you are back in sync.

Will try out a zero minute appointment later too to see if I can reproduce your issue - or if it was just something corrupt that got into your zimbra calendar from the initial sync.

EDIT: zero minute appointments are fine - both ways. The problem was corruption of the original appointments - see next post for detailed explanation.

[liverpoolfcfan]

For anyone who experiences a break in sync - most likely to occur after coming from an old version of the zimbra backend up to the more recent ones - I think we have found the source of the problem.

A unique ID assigned to appointments created on the phone and passed over to zimbra was encoded in one way by early backend releases. The encoding was changed in later releases to match that being used by some other backends.

But we didn't realise that appointments that were created with the old encoding and synced over to the zimbra server would sit on both the phone and the server quite happily up until the day you did a full resync of the phone. As soon as that happened the phone would try to read the corrupted UID passed back to it from the server, and would fail to sync.

If you suspect you have run into this,

1. De-select Calendar from your sync selections on the phone.
2. Sync
3. Turn on full debug in z-push
4. Re-Select Calendar in your sync selections on the phone
5. Sync (which will likely fail along the way)

The edit your debug.txt file and search for instances of "POOMCAL:UID" - There should be one of these for each appointment sync'ed

Some examples of good UID's are

06/29/10 15:24:53 [4967] I        <POOMCAL:UID>
06/29/10 15:24:53 [4967] I         E98riA594EGxg52cUI30B1
06/29/10 15:24:53 [4967] I        </POOMCAL:UID>

06/28/10 15:31:10 [25880] O        <POOMCAL:UID>
06/28/10 15:31:10 [25880] O        003e4ec3-fbf6-422e-8656-13048c461f1f
06/28/10 15:31:10 [25880] O        </POOMCAL:UID>

Note the I and O in the first column of the debug text means "I"ncoming to the server or "O"utgoing from the server. The two examples above are "I" - from my Nokia phone, and "O" from my zimbra server.

The formats can vary slightly from one phone to another - and also from zimbra to the phone.

From jonallport's log file - we identified the appointments that were causing his sync to fail. All fields in the appointments were present and correct, except that the the UID's in the log were like this

06/28/10 15:31:10 [25880] O        <POOMCAL:UID>
06/28/10 15:31:10 [25880] O        

NÖ
06/28/10 15:31:10 [25880] O        </POOMCAL:UID>

06/28/10 15:31:10 [25880] O        <POOMCAL:UID>
06/28/10 15:31:10 [25880] O        F
NÔÖ
06/28/10 15:31:10 [25880] O        </POOMCAL:UID>

The UID's for the two appointments were not long strings as in the good examples above - but instead were multi-line short strings with extended characters. As the corrupted UID is stored in zimbra as a custom extra property - it is not editable through the GUI in any way. So, at this time the only option for bad appointments such as these is to delete them from the server and recreate them.

Once jonallport deleted and recreated his bad appointments (2 out of 20 appointments), his sync is working correctly again.

Thanks to jonallport for his patience while we tried to identify the cause of the issue. As it only affects appointments originated from the phone, hopefully it will not cause too many headaches for people. And, hopefully this procedure will make it easy for people to identify their bad appointments and replace them.

[jonallport]

I just want to add a public, and big, thank-you for Vincent (LFC Fan) who has been incredibly helpful and went out of his way to help me sort this out.

[inaut]

Hi!

At first maybe there is a typo on Line 1713

if (count($this->_virtualAddressbook) > 0)

should be

if (count($this->_virtualAddressbooks) > 0)

Why does it sync a shared Contact Folder to the Phone, and not my Primary Contact folder? I can see it in debug.txt that the FolderID which is used is a shared Contact Folder.
I tried it with the XML file and without the XML File - its the same.
If i delete the shared Contact Folder and start a new Sync on the Phone it works.

Is it true, when I think, that only the primary Folders (Calendar, Contact) should be synced when no XML file exists?


inaut

[liverpoolfcfan]

At first maybe there is a typo on Line 1713

I think you are probably right on the typo. Hopefully gnosbush will pick it up and fix. Though it does not seem to impact syncing directly for me at least.

Why does it sync a shared Contact Folder to the Phone, and not my Primary Contact folder? I can see it in debug.txt that the FolderID which is used is a shared Contact Folder.
I tried it with the XML file and without the XML File - its the same.
If i delete the shared Contact Folder and start a new Sync on the Phone it works.

No idea on that one. I get my primary, and shared contacts - from both root level and sub folders. Tested with and without XML files.

Is it true, when I think, that only the primary Folders (Calendar, Contact) should be synced when no XML file exists?

I think all we can say is it should do what ever your phone device's default behaviour is. For Calendar/Tasks and Contacts, I think that means you get everything sync'ed that you have access to in zimbra web client GUI. For Email, you get whatever folders your email client subscribes to - for me this is Inbox and Sent by default. But, all other folders I can see in the zimbra GUI can be seen in the Folder Manager on the phone, and they can all be selected for Subscription.

By the way I primarily use a Nokia E71 with Mail for Exchange 3.00.050.

If gnosbush is listening, perhaps he can chime in with the default behaviour of his Palm Pre

[inaut]

I have that issue with an HTC Hero & Desire with Android 2.1

[liverpoolfcfan]

I have that issue with an HTC Hero & Desire with Android 2.1

A quick Google search seems to indicate that the ActiveSync support built into Android is far from perfect.

Look for a PM from me.