Zimbra-Upgrade and Server-Move inkl. Ubuntu-Dist-Upgrade

[karl.kowald]

Hey, me first (and hopefully not last) posting is to diskuss a Upgrade and Server-Move-Project.

The Situation:
Old:
A Ubuntu 08.04 LTS running (decidated server)
Installed zcs-5.0.11_GA_2695.UBUNTU8_64

New:
A Ubuntu 10.04 LTS virtual-Server (via kvm over a new decidated server)
Installing zcs-7.1.1_GA_3196.UBUNTU10_64.


My Sources:
Backup OpenSource Zimbra
Moving Server
OS Specific Upgrade


The Plan:
Step 1: Setting down the TTL-Times for all MX-Records of the Domains showing to Zimbra (about 600 Seconds should be enought)
Step 2: BackUp the Installation on the old Server (50GB data, needed around 2,5 - 3 hours)
Step 3: Upgrading to zcs-7.1.1_GA_3196.UBUNTU8_64

Version 1:
Step 4.1 Install zcs-7.1.1_GA_3196.UBUNTU10_64 on New Server
Step 5.1 Move /opt/zimbra from old Server via rsyn or nfs to new server
Step 6.1 Run zcs-7.1.1_GA_3196.UBUNTU10_64 upgrade again (cause old SErver data is zcs-7.1.1_GA_3196.UBUNTU8_64)

Version 2:
Step 4.2 Dist-Upgrade old Server to Ubuntu 10.04 LTS
Step 5.2 Run zcs-7.1.1_GA_3196.UBUNTU10_64 upgrade
Step 6.2 Move /opt/zimbra from old Server via rsyn or nfs to new server

Step 7 Changing DNS-MX-Records to new IP while the old server is down and the data is moving to the new server.

Because the old server would be shutdown after the servermove I like Version 1 little more , but I dont know if it will work correctly...

Some Questions:
- Which Version (1 or 2) is the saver way for my userdatas?

- I did the Upgrading-Process to Step 3 already, but had some troubles with logger and statistics. Loggerproblem is solved (uninstall syslog install rsyslog), sysstat-problem is not solved yet. Rest running correctly. Should I repair the stats-problem now or can I hope that the next steps would fix that problems?

- Are there some fails in my plan which I havent seen yet?
- Are there some more resources for my problem available i've not listed above which I should read before?


Thanks for your inputs

[bdial]

i've never done version 1 before but i think it would probably work

a few notes about version 2

on the new server, you'll want to do a install.sh -s which will just do the software installation but not configuration. This makes it so the appropriate package entries get put in the system for whatever ubuntu uses (rpm on suse) and that the files outside of /opt/zimbra get installed. there are a few things outside of that directory that zimbra modifies such as sudoers, crontab, logrotate and syslog

[karl.kowald]

Hello Again,

thanks to bdial, i installed zimbra on the new system mit -s...

I did my plan with Version 2...
first i thought i could do this without a reboot, so to don't have problems with a new kernel, but that didn't work and so i had to fix the old server for about 1 hour, cause the dist-upgrade didn't updated the new kernel to the grub-loader, instead it installed lilo *very strange*...
however...
after the server was back again, i transfered the opt/zimbra/ folder to the new server...
(after another 4 or 5 hours) the transfer was ok and i reinstalled zimbra on the new server again...

first it got problem with starting the zmconfigd and the zmlogwatch.. after a reboot of the hole server the zmconfigd started corretly (problem was an running instance on the port 7171 which blocks the zmconfigd)... but the zmlogwatch problem was again here... in the logs there was the info that there is no /var/log/zimbra-stats.log
so i created that file and gave the owner and group to zimbra (like the old file /var/log/zimbra.log)...
now zmcontrol star works fine and zmcontrol status says that everything is running...

but here is my trouble now:
If i login to the web-admin-interface, the Serverstatus shows all not working -> red crosses

and if i click into the serverstatistics there comes an error:

Mail: system failure: Unable to read logger stats Fehlercode: service.FAILURE Method: [unknown] Details:soap:Receiver

Perhaps someone has an idea how i could fix that? or some questions about the settings i should post here to help you to help me

thanks

[karl.kowald]

So, here are some outputs i should post if there are problems with the logger:

zmloggerctl status; echo $?; ps ax | grep zmlogger

0
 9811 pts/0    S+     0:00 grep zmlogger

/opt/zimbra/libexec/zmrrdfetch -f zmmtastats

DBD::SQLite::db prepare failed: no such table: rrds at /opt/zimbra/libexec/zmrrdfetch line 57.
Can't call method "bind_param" on an undefined value at /opt/zimbra/libexec/zmrrdfetch line 62.

/opt/zimbra/libexec/zmrrdfetch -c -f zmmtastats

DBD::SQLite::db prepare failed: no such table: rrds at /opt/zimbra/libexec/zmrrdfetch line 57.
Can't call method "bind_param" on an undefined value at /opt/zimbra/libexec/zmrrdfetch line 62.

tail /var/log/zimbra-stats.log is empty

zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats | head -20

ERROR: service.FAILURE (system failure: Unable to read logger stats)

zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats @limit=1 | head -50

ERROR: service.FAILURE (system failure: Unable to read logger stats)

thanks for your inputs

-----
edit
-----

Looks like i can not add a new posting.... so here is my new situation:

Ok, after another research via google i saw, that there was no /etc/rsyslog.d/60-zimbra.conf file created....
i created this one and restarted the rsyslog,
now i have other returns to the debug-informations:

zmloggerctl status; echo $?; ps ax | grep zmlogger

0
 1872 ?        S      0:00 /usr/bin/perl /opt/zimbra/libexec/zmlogger
 1907 ?        S      0:00 zmlogger: zmrrdfetch: server
 4214 pts/1    R+     0:00 grep zmlogger

for both:
/opt/zimbra/libexec/zmrrdfetch -f zmmtastats
/opt/zimbra/libexec/zmrrdfetch -c -f zmmtastats

no structure found for zmmtastats at /opt/zimbra/libexec/zmrrdfetch line 239.
zimbra@franziska:/root$ /opt/zimbra/lib

grep -w MTA /var/log/zimbra-stats.log
-> no output

tail /var/log/zimbra-stats.log

Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: zmstat mysql.csv: timestamp, Aborted_clients, Aborted_connects, Binlog_cache_disk_use, Binlog_cache_use, Bytes_received, Bytes_sent, Com_admin_commands, Com_assign_to_keycache, Com_alter_db, Com_alter_db_upgrade, Com_alter_event, Com_alter_function, Com_alter_procedure, Com_alter_server, Com_alter_table, Com_alter_tablespace, Com_analyze, Com_backup_table, Com_begin, Com_binlog, Com_call_procedure, Com_change_db, Com_change_master, Com_check, Com_checksum, Com_commit, Com_create_db, Com_create_event, Com_create_function, Com_create_index, Com_create_procedure, Com_create_server, Com_create_table, Com_create_trigger, Com_create_udf, Com_create_user, Com_create_view, Com_dealloc_sql, Com_delete, Com_delete_multi, Com_do, Com_drop_db, Com_drop_event, Com_drop_function, Com_drop_index, Com_dro:::DA8543CC-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA8543CC-9A72-11E0-BA3E-EC0036A25C8F:::p_procedure, Com_drop_server, Com_drop_table, Com_drop_trigger, Com_drop_user, Com_drop_view, Com_empty_query, Com_execute_sql, Com_flush, Com_grant, Com_ha_close, Com_ha_open, Com_ha_read, Com_help, Com_insert, Com_insert_select, Com_install_plugin, Com_kill, Com_load, Com_load_master_data, Com_load_master_table, Com_lock_tables, Com_optimize, Com_preload_keys, Com_prepare_sql, Com_purge, Com_purge_before_date, Com_release_savepoint, Com_rename_table, Com_rename_user, Com_repair, Com_replace, Com_replace_select, Com_reset, Com_restore_table, Com_revoke, Com_revoke_all, Com_rollback, Com_rollback_to_savepoint, Com_savepoint, Com_select, Com_set_option, Com_show_authors, Com_show_binlog_events, Com_show_binlogs, Com_show_charsets, Com_show_collations, Com_show_column_types, Com_show_contrib:::DA854DAE-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA854DAE-9A72-11E0-BA3E-EC0036A25C8F:::utors, Com_show_create_db, Com_show_create_event, Com_show_create_func, Com_show_create_proc, Com_show_create_table, Com_show_create_trigger, Com_show_databases, Com_show_engine_logs, Com_show_engine_mutex, Com_show_engine_status, Com_show_events, Com_show_errors, Com_show_fields, Com_show_function_status, Com_show_grants, Com_show_keys, Com_show_master_status, Com_show_new_master, Com_show_open_tables, Com_show_plugins, Com_show_privileges, Com_show_procedure_status, Com_show_processlist, Com_show_profile, Com_show_profiles, Com_show_slave_hosts, Com_show_slave_status, Com_show_status, Com_show_storage_engines, Com_show_table_status, Com_show_tables, Com_show_triggers, Com_show_variables, Com_show_warnings, Com_slave_start, Com_slave_stop, Com_stmt_close, Com_stmt_execute, Com_stmt_fetch,:::DA855196-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA855196-9A72-11E0-BA3E-EC0036A25C8F::: Com_stmt_prepare, Com_stmt_reprepare, Com_stmt_reset, Com_stmt_send_long_data, Com_truncate, Com_uninstall_plugin, Com_unlock_tables, Com_update, Com_update_multi, Com_xa_commit, Com_xa_end, Com_xa_prepare, Com_xa_recover, Com_xa_rollback, Com_xa_start, Compression, Connections, Created_tmp_disk_tables, Created_tmp_files, Created_tmp_tables, Delayed_errors, Delayed_insert_threads, Delayed_writes, Flush_commands, Handler_commit, Handler_delete, Handler_discover, Handler_prepare, Handler_read_first, Handler_read_key, Handler_read_next, Handler_read_prev, Handler_read_rnd, Handler_read_rnd_next, Handler_rollback, Handler_savepoint, Handler_savepoint_rollback, Handler_update, Handler_write, Innodb_buffer_pool_pages_data, Innodb_buffer_pool_pages_dirty, Innodb_buffer_pool_pages_flushed, Innodb:::DA855434-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA855434-9A72-11E0-BA3E-EC0036A25C8F:::_buffer_pool_pages_free, Innodb_buffer_pool_pages_misc, Innodb_buffer_pool_pages_total, Innodb_buffer_pool_read_ahead, Innodb_buffer_pool_read_ahead_evicted, Innodb_buffer_pool_read_requests, Innodb_buffer_pool_reads, Innodb_buffer_pool_wait_free, Innodb_buffer_pool_write_requests, Innodb_data_fsyncs, Innodb_data_pending_fsyncs, Innodb_data_pending_reads, Innodb_data_pending_writes, Innodb_data_read, Innodb_data_reads, Innodb_data_writes, Innodb_data_written, Innodb_dblwr_pages_written, Innodb_dblwr_writes, Innodb_have_atomic_builtins, Innodb_log_waits, Innodb_log_write_requests, Innodb_log_writes, Innodb_os_log_fsyncs, Innodb_os_log_pending_fsyncs, Innodb_os_log_pending_writes, Innodb_os_log_written, Innodb_page_size, Innodb_pages_created, Innodb_pages_read, Innodb_pages_written, Innodb_r:::DA85566E-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA85566E-9A72-11E0-BA3E-EC0036A25C8F:::ow_lock_current_waits, Innodb_row_lock_time, Innodb_row_lock_time_avg, Innodb_row_lock_time_max, Innodb_row_lock_waits, Innodb_rows_deleted, Innodb_rows_inserted, Innodb_rows_read, Innodb_rows_updated, Key_blocks_not_flushed, Key_blocks_unused, Key_blocks_used, Key_read_requests, Key_reads, Key_write_requests, Key_writes, Last_query_cost, Max_used_connections, Not_flushed_delayed_rows, Open_files, Open_streams, Open_table_definitions, Open_tables, Opened_files, Opened_table_definitions, Opened_tables, Prepared_stmt_count, Qcache_free_blocks, Qcache_free_memory, Qcache_hits, Qcache_inserts, Qcache_lowmem_prunes, Qcache_not_cached, Qcache_queries_in_cache, Qcache_total_blocks, Queries, Questions, Rpl_status, Select_full_join, Select_full_range_join, Select_range, Select_range_check, Select_s:::DA85588A-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA85588A-9A72-11E0-BA3E-EC0036A25C8F:::can, Slave_open_temp_tables, Slave_retried_transactions, Slave_running, Slow_launch_threads, Slow_queries, Sort_merge_passes, Sort_range, Sort_rows, Sort_scan, Ssl_accept_renegotiates, Ssl_accepts, Ssl_callback_cache_hits, Ssl_cipher, Ssl_cipher_list, Ssl_client_connects, Ssl_connect_renegotiates, Ssl_ctx_verify_depth, Ssl_ctx_verify_mode, Ssl_default_timeout, Ssl_finished_accepts, Ssl_finished_connects, Ssl_session_cache_hits, Ssl_session_cache_misses, Ssl_session_cache_mode, Ssl_session_cache_overflows, Ssl_session_cache_size, Ssl_session_cache_timeouts, Ssl_sessions_reused, Ssl_used_session_cache_entries, Ssl_verify_depth, Ssl_verify_mode, Ssl_version, Table_locks_immediate, Table_locks_waited, Tc_log_max_pages_used, Tc_log_page_size, Tc_log_page_waits, Threads_cached, Threads_connected:::DA855AB0-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: 2841:info: :::DA855AB0-9A72-11E0-BA3E-EC0036A25C8F:::, Threads_created, Threads_running, Uptime, Uptime_since_flush_status:: 06/19/2011 14:51:30,0,0,0,0,930364,4171765,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1763,0,0,0,0,0,0,0,0,0,0,0,0,156,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,0,0,0,1983,0,0,2345,5982,0,0,0,0,11,0,0,0,0,0,0,0,0,0,0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,45,0,0,0,0,11,0,0,0,0,0,0,0,0,0,0,0,0,0,358,0,0,0,0,0,0,0,OFF,58,0,5,52,0,0,0,1,4595,156,0,0,13,20466,15278,1535,61,10152,199,0,0,361,10085,4130,0,833,52409,37,56576,0,0,185593,4120,0,10725,322,0,0,0,69799936,4243,736,27686400,833,55,ON,0,782,91,126,0,0,372736,16384,4,4126,833,0,0,0,0,0,214,20,18531,361,0,6694,0,0,0,0,0,0.000000,12,0,31,0,115,119,172,115,126,0,0,0,0,0,0,0,0,0,12720,12720,NULL,0,0,142,0,57,0,0,OFF,0,6,0,4,7,0,0,0,0,,,:::DA855CCC-9A72-11E0-BA3E-EC0036A25C8F:::
Jun 19 14:51:30 franziska zimbramon[2841]: :::DA855CCC-9A72-11E0-BA3E-EC0036A25C8F:::0,0,0,0,0,0,0,0,0,NONE,0,0,0,0,0,0,0,,2873,0,0,0,0,0,12,12,1,881,881
Jun 19 14:51:30 franziska zimbramon[2843]: 2843:info: zmstat mtaqueue.csv: timestamp, KBytes, requests:: 06/19/2011 14:51:30, 2262, 8

zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats | head -20

<GetLoggerStatsResponse xmlns="urn:zimbraAdmin"/>

zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats @limit=1 | head -50

<GetLoggerStatsResponse xmlns="urn:zimbraAdmin"/>

any idea for a next step to solve my problem?

[karl.kowald]

ok, seems like i found the problem...

the zimbra-users crontab was empty... so i added all cronjobs and now everything is working fine...

[gruad23]

I know my reply is much to late, but probably someone will run into this thread with a similar problem:

I would have recommended version1, cause the old server is a running system and would be untouched until the new server is up and running. So during the whole process of migration you have a 100% functional backup in the background. Thats worth more than gold

I always use install.sh when migrating zimbra and I have the "feeling" it does good stuff. If install.sh finds existiting configuration in the ldap it will not force the configuration-dialog on you again. install.sh can do updates. Actually I think that whats is used for most.

Great to hear that you solved your problem.