[SOLVED] Automatically pop external accounts - zmprov script

[gregg1ep00]

This topic has been discussed at length both on these forums and Bugzilla:

https://www.zimbra.com/forums/instal...p3-server.html
Polling PoP email accounts
https://www.zimbra.com/forums/instal...nal-email.html
Bug 21199 – add ability to set autopolling in admin console
Bug 22114 – Expose external account polling interval in UI
Bug 33151 – data polling interval inheritance issue

From what I gather, the zimbraDataSourcePollingInterval attribute must be set for each data source, as it does not inherit the value set at the COS, domain, or account level (please correct me if I'm wrong on this). I had never run across a script to provision the datastore. I created a script to do just that.

Here are the core commands used:

# To list all accounts in a particular domain:
zmprov gaa example.com
#          ^^^^^^^^^^^
#          domain name

# To list all external accounts for a particular user (and automatically escape any spaces):
zmprov gds someone@example.com | grep zimbraDataSourceId | awk -F': ' '{print $2}'
#          ^^^^^^^^^^^^^^^^^^^
#          account name

# To set the zimbraDataSourceName:
zmprov mds someone@example.com {01234567-0123-0123-0123-0123456789ab} zimbraDataSourcePollingInterval 60
#          ^^^^^^^^^^^^^^^^^^^
#          account name        ^^^^^
#                              zimbraDataSourceId                                                     ^^
#                                                                                               interval

And here's the full script:

#!/bin/bash

# This is a script to go through all users of a domain
# and set the zimbraDataSourcePollingInterval attribute
# for all external accounts to the specified value.

# $1 = domain name
# $2 = polling interval in seconds

# Default Settings
DEFAULT_INTERVAL=60    # Default polling interval is 60 seconds

# Return codes
S_OK=0
E_WRONGARGS=85        # Wrong arguments


USAGE="Usage: `basename ${0}` -d domain-name [-i interval]"
INTERVAL=${DEFAULT_INTERVAL}

# ------- Begin Script -------

# Parse command-line arguments
while [ $# -gt 0 ]; do
    case "${1}" in
        -d|--domain)
            DOMAIN=${2}
            shift
            ;;

        -i|--interval)
            INTERVAL=${2}
            shift
            ;;

        *)
            echo "${USAGE}"
            exit ${E_WRONGARGS}
    esac

    shift                # Check next set of parameters
done

# Make sure we have the domain name
if [ -z "${DOMAIN}" ]
then
    echo "${USAGE}"
    exit ${E_WRONGARGS}
fi

# Make sure there are no .ds files
rm -f *.ds

# Make sure the commands.zmprov file is gone
if [ -e commands.zmprov ]
then
    rm -f commands.zmprov
fi

echo "Processing intervals for domain ${DOMAIN} - setting interval to ${INTERVAL}..."

# Loop through all accounts in domain
for ACCOUNT in `zmprov gaa ${DOMAIN}`
do
    echo "Searching account ${ACCOUNT}..."

    # Create a list of data source names for this account
    zmprov gds ${ACCOUNT} | grep zimbraDataSourceId | awk -F': ' '{print $2}' > ${ACCOUNT}.ds

    # Loop through all data source names and set the interval
    for DSID in `cat ${ACCOUNT}.ds`
    do
        echo "Found data source ${DSID}..."

        # Set the interval
        echo "mds ${ACCOUNT} ${DSID} zimbraDataSourcePollingInterval ${INTERVAL}" >> commands.zmprov
    done

    # When we're finished with the data source file, remove it
    rm -f ${ACCOUNT}.ds
done

# Run the commands
if [ -e commands.zmprov ]
then
    echo "Running zmprov..."
    cat commands.zmprov
else
    echo "Nothing to do!"
fi

zmprov < commands.zmprov

# Delete the file
rm -f commands.zmprov

# All is well, exit here
exit ${S_OK}
# ------- End Script -------

Hopefully this will help some newcomers (like myself) to Zimbra. If anyone has any suggestions on how to extend this script, or if you find it useful, please let me know.

Cheers,
Greg

[uxbod]

Welcome to the forums

Thank you for your contribution to the community

[gregg1ep00]

Tweaked script a little bit.

1. The script didn't work at all if the zimbraDataSourceName contained special characters (other than spaces). I'm now using zimbraDataSourceId instead of zimbraDataSourceName.
2. Enhanced performance slightly by generating a list of zmprov mds commands to be executed once the discovery has taken place. Per mmorse at this thread:
   

   To increase performance:
   
   "Every time you run zmprov or zmmailbox, it has to crank up a new jvm. But you can run a batch of commands in a single zmprov or zmmailbox command. It would be better to aggregate the list of sync folder commands and then send all to one invocation of zmmailbox."

Greg

[gregg1ep00]

My original post has disappeared, so here is the script:

#!/bin/bash

# This is a script to go through all users of a domain
# and set the zimbraDataSourcePollingInterval attribute
# for all external accounts to the specified value.

# $1 = domain name
# $2 = polling interval in seconds

# Default Settings
DEFAULT_INTERVAL=60    # Default polling interval is 60 seconds

# Return codes
S_OK=0
E_WRONGARGS=85        # Wrong arguments


USAGE="Usage: `basename ${0}` -d domain-name [-i interval]"
INTERVAL=${DEFAULT_INTERVAL}

# ------- Begin Script -------

# Parse command-line arguments
while [ $# -gt 0 ]; do
    case "${1}" in
        -d|--domain)
            DOMAIN=${2}
            shift
            ;;

        -i|--interval)
            INTERVAL=${2}
            shift
            ;;

        *)
            echo "${USAGE}"
            exit ${E_WRONGARGS}
    esac

    shift                # Check next set of parameters
done

# Make sure we have the domain name
if [ -z "${DOMAIN}" ]
then
    echo "${USAGE}"
    exit ${E_WRONGARGS}
fi

# Make sure there are no .ds files
rm -f *.ds

# Make sure the commands.zmprov file is gone
if [ -e commands.zmprov ]
then
    rm -f commands.zmprov
fi

echo "Processing intervals for domain ${DOMAIN} - setting interval to ${INTERVAL}..."

# Loop through all accounts in domain
for ACCOUNT in `zmprov gaa ${DOMAIN}`
do
    echo "Searching account ${ACCOUNT}..."

    # Create a list of data source names for this account
    zmprov gds ${ACCOUNT} | grep zimbraDataSourceId | awk -F': ' '{print $2}' > ${ACCOUNT}.ds

    # Loop through all data source names and set the interval
    for DSID in `cat ${ACCOUNT}.ds`
    do
        echo "Found data source ${DSID}..."

        # Set the interval
        echo "mds ${ACCOUNT} ${DSID} zimbraDataSourcePollingInterval ${INTERVAL}" >> commands.zmprov
    done

    # When we're finished with the data source file, remove it
    rm -f ${ACCOUNT}.ds
done

# Run the commands
if [ -e commands.zmprov ]
then
    echo "Running zmprov..."
    cat commands.zmprov
else
    echo "Nothing to do!"
fi

zmprov < commands.zmprov

# Delete the file
rm -f commands.zmprov

# All is well, exit here
exit ${S_OK}
# ------- End Script -------

[techhelp]

Although this is a little old, I thought I would give it a shot. I have a new install Release 6.0.5_GA_2213.UBUNTU8 UBUNTU8 FOSS edition. I intend to have two or three domains setup and all seems to be working fine now although I am experiencing the autopolling issue from pop accounts.

After going through many of the posts, listed here also, I thought I would give the script a shot.

I get two errors at the end, and nothing changes.

I am fairly new to linux, but learning a bunch in the process of setting up Zimbra....
1. su root
2. vi a.sh (pasted contents)
3. chmod 755 a.sh
4. su zimbra
5. a.sh -d domain.here -i 3m
6. Processing intervals for domain akrc.on.ca - setting interval to 3m...
ERROR: service.INVALID_REQUEST (invalid request: can only be used with "zmprov -l/--ldap")
Nothing to do!
/bin/a.sh: line 92: commands.zmprov: No such file or directory


I have done the same for both the original and updated scripts. as this was written for v5, is there something I can fix for v6.

I plan on adding up to 100 users only on this server between the two domains, but would love to collect mail automagically. I am using this Zimbra as a secondary mail server by using my ISP as the primary.

Regards,

[gregg1ep00]

Sorry, I'm not familiar at all with v6, as we are still on v5. Hopefully one of the coders here is familiar with v6 and can post updated zmprov commands.

[uxbod]

When switching to the zimbra user please use

su - zimbra

this ensures the profile is sourced in correctly.

For ZCS V6 please change the following line in the script from

zmprov < commands.zmprov

to this

zmprov -l < commands.zmprov

.

[techhelp]

Originally Posted by uxbod

When switching to the zimbra user please use

su - zimbra

this ensures the profile is sourced in correctly.

For ZCS V6 please change the following line in the script from

zmprov < commands.zmprov

to this

zmprov -l < commands.zmprov

.

I still end up with the same error, I did a copy paste into vi of the original scripts, but is there a chance I have broken the script? I edited the file as su root. I am not using ldap for authentication, just internal authentication mechanism.

su - zimbra
zimbra@webmail:~$ automailcollect.sh -d mydomain.here -i 3m
Processing intervals for domain mydomain.here - setting interval to 3m...
ERROR: service.INVALID_REQUEST (invalid request: can only be used with "zmprov -l/--ldap")
Nothing to do!
/bin/automailcollect.sh: line 91: commands.zmprov: No such file or directory


BTW, thank you for superb response time.

[uxbod]

Just add in before the following code

# Make sure we have the domain name
if [ -z "${DOMAIN}" ]
then
    echo "${USAGE}"
    exit ${E_WRONGARGS}
fi

the following line as by default zimbra cannot write to /opt/zimbra

cd /tmp

[techhelp]

Originally Posted by uxbod

Just add in before the following code

# Make sure we have the domain name
if [ -z "${DOMAIN}" ]
then
    echo "${USAGE}"
    exit ${E_WRONGARGS}
fi

the following line as by default zimbra cannot write to /opt/zimbra

cd /tmp

I am running as su - zimbra
copied the script to /tmp
added all lines as best I understood and end up with

zimbra@webmail:/tmp$ automailcollect.sh -d mydomain.here -i 3m
Processing intervals for domain mydomain.here - setting interval to 3m...
ERROR: service.INVALID_REQUEST (invalid request: can only be used with "zmprov -l/--ldap")
Nothing to do!
/bin/automailcollect.sh: line 99: commands.zmprov: No such file or directory
zimbra@webmail:/tmp$

Below is the code I am currently working with, maybe I have some other issue? Possibly I added/deleted lines from the original script. Would you mind taking a look and let me know what you think, and if I can impose to have you correct any glaring mistakes added by myself. I feel so vulnerable trying to make this work. I am too M$ oriented and maybe I bit off too much with Zimbra... but I sure love my *nix box now

#!/bin/bash

# This is a script to go through all users of a domain
# and set the zimbraDataSourcePollingInterval attribute
# for all external accounts to the specified value.

# $1 = domain name
# $2 = polling interval in seconds

# Default Settings
DEFAULT_INTERVAL=60 # Default polling interval is 60 seconds

# Return codes

_OK=0
E_WRONGARGS=85 # Wrong arguments


USAGE="Usage: `basename ${0}` -d domain-name [-i interval]"
INTERVAL=${DEFAULT_INTERVAL}

# ------- Begin Script -------

# Parse command-line arguments
while [ $# -gt 0 ]; do
case "${1}" in
-d|--domain)
DOMAIN=${2}
shift
;;

-i|--interval)
INTERVAL=${2}
shift
;;

*)
echo "${USAGE}"
exit ${E_WRONGARGS}
esac

shift # Check next set of parameters
done

# Make sure we have the domain name
if [ -z "${DOMAIN}" ]
then
echo "${USAGE}"
exit ${E_WRONGARGS}
fi

# Make sure there are no .ds files
rm -f *.ds

# Make sure the commands.zmprov file is gone
if [ -e commands.zmprov ]
then
rm -f commands.zmprov
fi

echo "Processing intervals for domain ${DOMAIN} - setting interval to ${INTERVAL}..."

# Loop through all accounts in domain
for ACCOUNT in `zmprov gaa ${DOMAIN}`
do
echo "Searching account ${ACCOUNT}..."

# Create a list of data source names for this account
zmprov gds ${ACCOUNT} | grep zimbraDataSourceId | awk -F': ' '{print $2}' > ${ACCOUNT}.ds

# Loop through all data source names and set the interval
for DSID in `cat ${ACCOUNT}.ds`
do
echo "Found data source ${DSID}..."

# Set the interval
echo "mds ${ACCOUNT} ${DSID} zimbraDataSourcePollingInterval ${INTERVAL}" >> commands.zmprov
done

# When we're finished with the data source file, remove it
rm -f ${ACCOUNT}.ds
done

# Run the commands
if [ -e commands.zmprov ]
then
echo "Running zmprov..."
cat commands.zmprov
else
echo "Nothing to do!"
fi

# Make sure we have the domain name
if [ -z "${DOMAIN}" ]
then
echo "${USAGE}"
exit ${E_WRONGARGS}
fi

zmprov -l < commands.zmprov

cd /tmp

# Delete the file
rm -f commands.zmprov

# All is well, exit here
exit ${S_OK}
# ------- End Script -------