iSync connector retrieving multiple contact ghosts instead of single current entry

[jgore]

I have a nice clean Contacts folder in Zimbra. I'm trying to sync it with Address Book, using Zimbra Apple iSync Connector 5.0.1743.

I delete all entries in Address Book. I tell iSync Connector to sync, Contacts folder only, Manually. I tried both with Reset Sync Data and without, with identical results. Here is an example of the problem:

I have a contact in Zimbra, John Doe, with company ACME. In Address Book, I have three entries: John Doe, company ACME, which contains nothing but the Note; John Doe, company ACME-America, with all the fields populated, but some information out of date; and John Doe, company ACME Enterprises, also all empty except the Note.

At some point in my life, I had that person listed under ACME, ACME-America, and ACME Enterprises. But never at the same time, and not now in Zimbra. I confirmed that by exporting the csv from Zimbra, and by querying mail_item table in mysql. There is only one entry.

Where on Earth are the three entries in Address Book coming from???

This is just one example. Eyeballing it, about 1/3 of my 1100+ entries get duplicated or triplicated by the iSync.

[sculi2000]

Hey there. What you're describing is certainly puzzling. Just a couple of questions: do you have any shared contact folders? Are the John Doe's in the "Emailed Contacts" folder? I know you said you selected only the Contacts folder, but since you're sure you emptied Address Book, and the three John Doe's always show up, they must be coming from somewhere.

You can take a look at the logs in ~/Library/Logs/Zimbra/*. They might provide a clue where the contacts are being pulled from.

Take care,

Scott

[jgore]

Scott,

No, John Doe is not in Emailed Contacts. (And neither are any of the others that get duplicated.)

I used to have two shared folders (Customers and Vendors). John Doe was in Customers, under the company ACME-America. The ACME International entry wasn't in any of the folders.

I had created those folders myself and then shared them with other users. I suspected that iSync may be going into those folders, so I deleted both of them and emptied the trash. That, however, did not change my iSync results at all.

In the sync log, John Doe shows this way:

[Fri Dec 14 12:37:38 200] <?xml version="1.0"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
<soap:Header>
<context xmlns="urn:zimbra">
<change token="161721"></change>
</context>
</soap:Header>
<soap:Body>
<GetContactsResponse xmlns="urn:zimbraMail">
...
<cn fileAsStr="Doe, John" id="134305" rev="161376" d="1197607814000" l="7">
<a n="homePostalCode">80015</a>
<a n="lastName">Doe</a>
<a n="homeStreet">19999 E Square Cir</a>
<a n="workCity">Aurora</a>
<a n="department">America</a>
<a n="homePhone">555-555-4446</a>
<a n="homeFax">555-555-3918</a>
<a n="workPostalCode">80999-9999</a>
<a n="mobilePhone">555-555-4432</a>
<a n="homeCity">Aurora</a>
<a n="email">john.doe@acme.com</a>
<a n="company">ACME</a>
<a n="homeState">CO</a>
<a n="workFax">555-555-1458</a>
<a n="workStreet">3333 South Yellowbrick Road, Ste 123</a>
<a n="workState">CO</a>
<a n="workPhone">555-555-1421</a>
<a n="notes">Office:
info
info
info</a>
...
<a n="firstName">John</a>
<a n="jobTitle">President</a>
</cn>
</GetContactsResponse>
</soap:Body>
</soap:Envelope> (382)
...
[Fri Dec 14 12:37:39 200] pushing 134305/EmailAddress/home/1 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Email Address";
contact = (134305);
type = home;
value = "john.doe@acme.com";
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/PhoneNumber/home fax/1 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Phone Number";
contact = (134305);
type = "home fax";
value = "555-555-3918";
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/PhoneNumber/work fax/1 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Phone Number";
contact = (134305);
type = "work fax";
value = "555-555-1458";
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/PhoneNumber/mobile/1 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Phone Number";
contact = (134305);
type = mobile;
value = "555-555-4432";
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/PhoneNumber/work/1 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Phone Number";
contact = (134305);
type = work;
value = "555-555-1421";
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/PhoneNumber/home/1 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Phone Number";
contact = (134305);
type = home;
value = "555-555-4446";
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/StreetAddress/home/1 with record {
city = Aurora;
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Street Address";
contact = (134305);
"postal code" = 80015;
state = CO;
street = "19999 E Square Cir";
type = home;
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305/StreetAddress/work/1 with record {
city = Aurora;
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Street Address";
contact = (134305);
"postal code" = "80999-9999";
state = CO;
street = "3333 South Yellowbrick Road, Ste 123";
type = work;
} (382)
[Fri Dec 14 12:37:39 200] pushing 134305 with record {
"com.apple.syncservices.RecordEntityName" = "com.apple.contacts.Contact";
"company name" = ACME;
department = America;
"email addresses" = ("134305/EmailAddress/home/1");
"first name" = John;
"job title" = President;
"last name" = Doe;
notes = "Office:\ninfo\ninfo\Ufffd\n...info";
"phone numbers" = (
"134305/PhoneNumber/home fax/1",
"134305/PhoneNumber/work fax/1",
"134305/PhoneNumber/mobile/1",
"134305/PhoneNumber/work/1",
"134305/PhoneNumber/home/1"
);
"street addresses" = ("134305/StreetAddress/home/1", "134305/StreetAddress/work/1");
} (382)


And that's it.

Jacob

[sculi2000]

Well from that snippet of log, it looks like the iSync Connector is doing the correct thing. Perhaps the problem is with Address Book and Sync Services. You mentioned that you cleaned out Address Book...how specifically did you do this?

Thanks,

Scott

[jgore]

Scott,

I selected all entries and hit the delete key. During the "Sync Now" operation in the plug in, when it asked me if it was ok to delete everything in Zimbra Contacts, I clicked the Cancel button. Then it proceeded with the sync-to-the-computer count-off.

There are two strange things going on: the presence of 3 entries after the sync instead of one, and the fact that the entry that has data in it is with company = "ACME America" instead of just "ACME", which is what's in Zimbra and what the log says. The entry with just "ACME" for the company has no data in it, except the name of the contact and the name of the company.

I tried to find some other place from where iSync could be picking up entries, but in my Address Book preferences "Syncronize my contacts.. iMac" is unchecked (and I don't even have a .Mac account), "Synchronize with Exchange" and "Synchronize with Yahoo!" are also unchecked, the LDAP server is unchecked (and it's pointing to the Zimbra server and I never could get it to work anyway).

I do have Palm Desktop installed (I'm moving to the iPhone from a Treo)... could pressing "Sync Now" in the Zimbra iSync gadget be causing entries to be picked up from the Palm database?

Thanks,
Jacob

Thanks,

Jacob

[sculi2000]

Well this is the complicating factor. The iSync Connector is not syncing directly with Address Book. It's syncing to Sync Services, as is Address Book. So when you're deleting all the contacts out of AB, it's not necessarily deleting everything in the Sync Services database.

An interesting exercise might be to take a look at what's in the Sync Services database. There is an Apple supplied tool called Syncrospector that will allow you to inspect many different pieces of the Sync Services framework. It's aimed primarily at developers, but in this case it might be useful. You can find it at: <https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?fileID=20824> You'll need an ADC account (which is free) to get it.

To answer your question, yes the Palm Desktop might very well be the cause of this problem. We'll know more after looking at what the Sync Services database looks like.

Thanks,

Scott

[jgore]

Thanks for the pointer to Syncrospector, it's quite an eye-opener on how much stuff is lurking in the background. I guess I'll need to go through the tutorial, because the app's GUI is about as self-explanatory as learning to use Unix by reading the source code.... In the meantime, if you have any hints, I'd appreciate them (like how to find the John Doe records, or duplicate records in general, or even just how to delete recods).

Thanks,

Jacob

[sculi2000]

:-) Yep, it took me awhile to get the hang of it. The challenging aspect to it is that you can't search (at least I haven't found a way) to search for arbitrary record data. You can only search for records by id, which initially isn't that helpful, because you have no idea what the id is.

What I typically do in this case is to select "Truth" in the top left pop up menu. You'll then have *all* the data in the Sync Services truth database. Click on "Entity Name" in that table...this will order the records so you'll have an easier time hunting down the one you want. You are interested in entities of type "com.apple.contacts.Contact". Clicking on instances of them in turn will show you precisely what is still in the database. If it has a deleted symbol, then well..it has been deleted. Otherwise, it is still extant.

It's not for the faint of heart, but it should provide some answers to the ghosts that are lurking about on your machine.

Hope that helps,

Scott

[jgore]

Well, I got that far, and saw that my Truth Database was full of bull***t.

The question is, how do I get rid of it?

I'll be happy to just wipe all of the com.apple.contacts.* records, and then sync everything in from Zimbra's Contacts, but I can't find any controls in Syncrospector do delete records from the Truth Database.

Thanks,

Jacob

[jgore]

After reading up on the Truth Database and syncing, it dawned on me that when I tell it to sync after I delete all entries in Address Book and iSync pops up the panel that it's going to delete a couple thousand entries, it's not talking about deleting them on Zimbra, but rather in the Truth Database.

So I told it to allow the deletions, and sure enough, all of the entries are shown as "deleted" in Syncrospector.

But now when I use the Zimbra iSync Connector to sync Contacts, absolutely nothing happens. The log gets written in ~/Library/Logs/Zimbra, and it shows all the entries that should be syncing to the computer. But no new contacts entries show up in the Truth Database, the only thing there are the old ones that had been deleted.

How does one tell if the sync between Zimbra and the truth db even happening? Here's what shows up in the concole app during the sync:

2007-12-19 18:27:01.628 SyncServer[290] A Sync Server is already running on this computer, exiting process.
mkdir: /Users/jgore/Library/Application Support/Zimbra: File exists
[Wed Dec 19 18:27:24 200] Mac OS X version is 1049 (295)
2007-12-19 18:28:24.870 System Preferences[291] authenticate finished: exit status = 0
===== Wednesday, December 19, 2007 6:28:51 PM America/Denver =====
[Wed Dec 19 18:28:59 200] Mac OS X version is 1049 (301)
2007-12-19 18:29:39.181 System Preferences[291] sync finished: exit status = 2


I suspect exit status > 0 is not good news?