Z-Push Zimbra Backend (Push Email, Contacts, Appointments & Tasks)

[sbourdette]

There is nothing incorrect in that session ID changed line. it is normal for the Logoff command to close the session. There must be more going on ?

Have you correctly configured the 'state' folder
define('STATE_DIR', 'state');
and is it writeable by the web server ?

Have you added the 4 configuration settings to your php.ini or .htaccess file ?

Is there any error message in your web server log file (ssl_error_log by default on apache) ?

- State folder corrctly configured and chmod to 777

i have a .htaccess file at the root of z-push with this content :

php_flag magic_quotes_gpc off
php_flag register_globals off
php_flag magic_quotes_runtime off
php_flag short_open_tag on

I have no ssl activated so no ssl_error_log

in my php.ini I added :
max_execution_time = 120
memory_limit = 128M

In my Inbox i have some foldername with "é" and "è" it might be the source of the issue ?

[tja]

first of all: thx for the work on the z-push backend

i use it for some time now, at least after one of my android phones got gingerbread as in froyo the calendar sync always locked up, looped endlessly and used way too much battery.
calendar sync didnt finish so no calendar sync in froyo on my devices.
contacts & mail did work fine in froyo.

currently tested and working (except the things below):
- samsung sgs, cyanogen 7.0.x beta
- samsung galaxy tab, samsung 2.3.3 JQ1
- zte blade, cyanogen 7.0.x
- iphone 3g

im using t-push 1.5.1 and zimbra49 atm.

problems (on the galaxy tab i use currently):

calendar:
having calendar shares of other zimbra users seems to disturb syncing as shared calendar entries seems to added on the handset every sync.
and: recurring identical calendar entries (birthdays f.e.) get duplicated on the phone every sync.

contacts:
i have some contacts with multiple mobile numbers where only one number gets synced to the phone.

(maybe this errors have nothing to do with z-push and/or the zimbra backend but are instead bugs in the android syncing apps.)

zimbra UI:
the zimbra webclient seems to refresh every second if z-push is active and you use the "push" mode in the android handsets, the mail preview page is losing focus every couple of seconds. using some bigger intervall and not "push" has fixed this annoying behaivor for me.

[liverpoolfcfan]

first of all: thx for the work on the z-push backend

Thank you. I appreciate it

i use it for some time now, at least after one of my android phones got gingerbread as in froyo the calendar sync always locked up, looped endlessly and used way too much battery.

Just highlights one of the issues with Android - as we have had the exact opposite experience with Samsung Galaxy S. Froyo (2.2.1) worked great for everything. 2.3.3 seems to have broken Calendar sync such that it keeps syncing every minute. There is no new content synced - it just does a new sync, and thereafter does a new Ping request, and soon the server is flooded with Ping requests all from the same device. Turn off Calendar and everything settles down again.

[tja]

2.3.3 seems to have broken Calendar sync such that it keeps syncing every minute. There is no new content synced - it just does a new sync, and thereafter does a new Ping request, and soon the server is flooded with Ping requests all from the same device. Turn off Calendar and everything settles down again.

yeah, thx for the hint.
i checked and see the same thing with the galaxy tab and samsungs original gingerbread firmware but at least it is syncing - 2.2 on the tab did never sync calendar here.
your sgs 2.3.3 fw isnt samsungs by chance ?!? when i get back the sgs i will re-check with the 2.3.4 cyanogen fw on it.

the conclusion for me is to stay away from samsungs firmwares - im just waiting for a working version of cyanogen for the tab ...

in the scope of a business - not just fooling around like me - i wonder if the samsung firmware would work bugfree with an exchange server and if, what would make the difference - i have no exchange server at hand to test this.
rolling out custom firmware for employees seems uuuhhmm problematic to me.

[jasonferrara]

Hello,

Can you send details on how to get "HTML email when used with a patched install of the as12 branch of z-push"

Thanks

I'd like to know too. I know how to get the as12 branch, but whats the patch?

[liverpoolfcfan]

I'd like to know too. I know how to get the as12 branch, but whats the patch?

Still testing ... not quite ready yet. I will announce it here when it is ready.

[jasonferrara]

Still testing ... not quite ready yet. I will announce it here when it is ready.

I wouldn't mind helping to test and debug. I have a wide variety of android and iOS devices available.

[liverpoolfcfan]

Release 50 of the z-push zimbra backend has just been uploaded to SourceForge

Z-Push Zimbra Backend | Download Z-Push Zimbra Backend software for free at SourceForge.net


Here is a summary of the changes.

THESE RELEASE NOTES FOR RELEASE 50 INCLUDE ALL DETAILS OF RELEASE 49 AS SO MANY 
INSTALLATIONS ARE STILL RUNNING RELEASE 48 (OR EARLIER). THERE IS JUST ONE 
MAJOR CHANGE MADE BETWEEN RELEASES 49 & 50. IT DOES NOT ALTER THE FUNCTIONALITY 
BUT CHANGES SOME PHP FUNCTIONS USED TO IMPLEMENT THE FUNCTIONALITY IN ORDER TO 
MINIMISE THE POSSIBILITY OF AN OUT-OF-MEMORY CRASH WHEN PROCESSING EMAILS

PLEASE READ THESE RELEASE NOTES COMPLETELY BEFORE IMPLEMENTING RELEASE 50
THIS IS EXPECIALLY CRITICAL IF YOU ARE TURNING ON ANY OF THE NEW FEATURES
AND ARE UPGRADING AN EXISTING INSTALLATION FROM RELEASE 48 OR EARLIER

AS A RESULT OF THE REWRITE OF EMAIL HANDLING, TWO ADDITIONAL INCLUDE FILES
ARE NEEDED - mime.php and mimePart.php - WHICH ARE USED FOR BUILDING EMAILS
THESE CAN BE PLACED IN THE INCLUDE OR BACKEND FOLDER (BUT JUST ONE OF THEM)
EMAIL SYNC WILL NOT WORK WITHOUT THEM 

Changes Made To Revision 50:
 - Removed use of Mail_MimeDecode for synching emails out to the device, and
   for getting details of the original email for Smart Forward/Reply as it 
   was found to be consuming huge chunks of memory whenever an email had large 
   attachments. This could cause the sync to crash. It would continue to
   crash every time the phone tried to sync the mail folder until such time
   as the message was moved on the server, or was outside of the window for
   days to sync. Replaced functionality with native zimbra soap calls.
   Mail_MimeDecode is still needed for SendMail from the phone to handle the
   raw rfc822 data sent to zimbra - hopefully people will not be sending 
   huge attachments from their phones but, even if the send does fail, it 
   will not affect the sync of emails out to the phone.
 - Fixed filtering Calendar/Contacts/Task folders when virtual is set to false
   for a folder type and SmartFolders is enabled
 - Removed function FromHeaderFix - as it is not used
 - Removed function GetAttachmentBody - as it is not used
 - Removed function GetBody - as it is not used
 - Changed text/html encoding from base64 to quoted-printable for mime body
   as it generally produced smaller output

Changes Made To Revision 49:

 - HTML email when used with a patched install of the as12 branch of z-push
   from the z-push SVN repository. I will document the installation process
   separately in the near future. It has been tested with Android 2.3 and
   Windows Mobile 6.5 to date. I have no access to Apple devices so cannot
   do any testing on those - so they may still need some work.
   
   The remainder fo the changes below apply to installations under either an 
   official z-push 1.5.x release, or the patched as12 release.
 
 - Added new options to the config file to allow disabling of any folder
   type system-wide. See details in notes at top of zimbra.php file 
 - Added new SmartFolders feature - to allow users to control the list of 
   folders they wish to sync to their devices without using XML files 
   NOTE: If you wish to enable SmartFolders - you MUST stop sync and clear
   all old state data from the state folder. Even if no SmartFolder names
   are used, the sequence and naming of the folders will change
   NOTE: If SmartFolders are enabled, XML files will not be processed 
   To enable the feature set ZIMBRA_SMART_FOLDERS to true in config.php
   Once enabled, the final character in a folders name can take on special
   meanings as follows :-
    "-" Do not include this folder or any subfolders thereof
    "." Include this folder - But do not include any subfolders thereof
 - Changed Attachments handling to allow attachments in multi-part messages
   to be returned. Also allows attaching of an email as a .eml file
 - Added ability to filter out Completed Tasks during sync - Note: As zimbra 
   does not store a Completed Date, the last modified timestamp is used
 - Added Reminders for Tasks (for zimbra 7.0+) - need a zcs 6.0.nn user to test
   if it will work on zcs 6.0.nn also
 - Enhancement in SendMail to include original message text for reply/forward
   This is done in preference to attaching the original as Android devices are
   unable to open .eml attachments natively at this time
 - Added check in SendMail for z-push version to ensure we properly format the
   path to the "state" directory - change is to match the config.php change to
   the definition of STATE_DIR introduced by z-push 1.5 
 - Fix in SendMail to honour ZIMBRA_ENFORCE_VALID_EMAIL when set to true. If
   the from email address used is not a valid one for the account it is 
   replaced by the default sender email address. 
 - Fix in SendMail to add sender name to the address in all cases. This is a 
   generalized fix based on the original premise of the ZIMBRA_NOKIA_MFE_FIX
   as it seems that iPhone and Android also send emails without a sender name
 - Fixed GetSearchResults to match response 'keys' to latest zpushdefs.php 
 - Fixed possible memory leaks and reduced memory usage by unset()ing rtf and 
   mime decoder class instances, and various arrays
 - Removed the input parameter from the params array send to Mail_mimeDecode
   as it is already passed to the class on creation, and created and unset a
   params object to ensure that memory is cleaned up
 - Fix for special characters (",&,<,>) in user passwords
 - Added fix for Euro symbol & other non-standard cp1252 characters
 - Changes made to MoveMessage to prevent items being moved from one account to
   another, and from one folder type to another. These restrictions are to
   prevent items becoming inaccessible on the device. See comments in function
   for more information.
 - Removed username from Sync Request log entry for z-push 1.5 and later as 
   the z-push engine is already logging it
 - Fix for Categories field from HTC Desire (Android 2.2) - Phone sends an
   empty string Categories field instead of an array. The blank field caused an
   error in CreateTagRequest. Fix to check that (is_array(input->categories))
 - Fix added for iPhone/iPad/etc Meeting Request/Response handling as it appears
   that the Apple client sends off meeting requests/responses directly from the
   handset instead of letting the server handle them 
 - Fix in PROVISIONING getPolicyKey to avoid writing "validate" records in  
   database for android devices 
 - Fix in PROVISIONING generatePolicyKey to limit upper bound on random number 
   generator to 2147483647 (as some phones cannot handle larger numbers)
 - Added function ReportMemoryUsage (called on Logoff) to track memory use
 - Added new functions that are needed for ActiveSync protocol version 12
   - ImportMessageFlag - use of this required a change to request.php
   - GetSettings - used for Out-Of-Office
   - SetSettings - used for Out-Of-Office, and storing Handset data
   - Updated GetSearchResults to add mailbox search capability  


PLEASE READ THESE RELEASE NOTES COMPLETELY BEFORE IMPLEMENTING RELEASE 50
THIS IS EXPECIALLY CRITICAL IF YOU ARE TURNING ON ANY OF THE NEW FEATURES

[myriad]

I have set up Z-Push and your backend correctly (I think) but even though I can create an account successfully in my Samsung Vibrant (using Cyanomod 7) I can send emails fine using the default mail client but I don't see any mail in the app. I get an icon for the 'combined inbox' but I don't see any mail or calendars or contacts either!

Looking at the debug.txt seems to suggest that everything is good:

06/03/11 23:40:39 [26047] [rick@myriad.ca] Start
06/03/11 23:40:39 [26047] [rick@myriad.ca] Z-Push version: 1.5.3-685
06/03/11 23:40:39 [26047] [rick@myriad.ca] Client IP: 199.7.156.43
06/03/11 23:40:39 [26047] [rick@myriad.ca] Client supports version 2.5
06/03/11 23:40:39 [26047] [rick@myriad.ca] PHP version: 5.3.2-1ubuntu4.9 - z-push Zimbra Backend version: 50
06/03/11 23:40:39 [26047] [rick@myriad.ca] START Logon { username = rick@myriad.ca; domain = ; password = <hidden> }
06/03/11 23:40:39 [26047] [rick@myriad.ca] Session Id - NEW [14179]
06/03/11 23:40:39 [26047] [rick@myriad.ca] END Logon { connected = true; uid = rick }
06/03/11 23:40:39 [26047] [rick@myriad.ca] START Setup { userid = rick@myriad.ca; devid = androidc1501101591; protocolversion = 2.5; useragent = Android/0.3 }
06/03/11 23:40:39 [26047] [rick@myriad.ca] Smart Folders ENABLED - User Profile files will be ignored
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:927 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:931 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:932 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1090 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1091 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1101 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1139 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1144 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1090 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1091 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] /var/www/myriad/sync/backend/zimbra.php:1101 Undefined index: wiki (8)
06/03/11 23:40:39 [26047] [rick@myriad.ca] END Setup { 43 Folders Loaded }
06/03/11 23:40:39 [26047] [rick@myriad.ca] POST cmd: FolderSync
06/03/11 23:40:39 [26047] [rick@myriad.ca] Initializing folder diff engine
06/03/11 23:40:39 [26047] [rick@myriad.ca] START GetFolderList
06/03/11 23:40:39 [26047] [rick@myriad.ca] END GetFolderList { found 34 folders }
06/03/11 23:40:39 [26047] [rick@myriad.ca] Found 34 folder changes

Where do I start to troubleshoot this problem? Any help would be gratefully received.

[slite]

Hi there,

Thanks for the great work, just love it ^^
But i also got 1 question and 2 issues which i would like to resolve, maybe you can help me somehow.

Q1. Is there some sync window for calendar entrys? Seems like everthing that lies back or in the future some ammount of days isnt synced to my phone. Is there any way to change this?

Jumping right into the issues, both calendar related.

I1. Wrong timezone. I have everything in Timezone Europe/Amsterdam. (Servertime, Appointments, z-push config.php, Phone).
The sync somehow mangles with the timezones, the time stays correct though because both android and zimbra calculate the local time. New Appointment on Zimbra (GMT+1, zimbra doesnt show DST in its timezone listbox, does it?)-> somehow gets GMT on the Phone -> change Appointment on the Phone (which current timezone is GMT+2 because of DST) -> gets GMT-1 on Zimbra but still with correct local time. Any ideas what i could do/check to resolve this?

I2. Force close of Calendar when i want to choose visible calendars. This only happens when i have a corporate calendar added (Cyanogen 7.0.3 on Desire Z), works fine with only google accounts. Can't check if its corporate account in general or z-push/zimbra only because i have no real exchange account to check with. Please let me know if you need any logs/infos.

Apart from that looks very solid and good, i might also get onto the board in the near future and start improving stuff when i can spare a bit more time. First promising sync project for zimbra+android in years!