A few months ago, I inherited a used server, and decided to install xen on it, and then create myself web and mail vm's. This would then replace my current web server.
I installed a web vm, which works nicely, a second web vm for a friend, and finally, a Zimbra VM. The Zimbra vm was troublesome. My hardware was old, it wasn't the best platform for virtualization, and zimbra seems unhappy in a xen VM.
After i finished migrating all of my data from my old web server to the new web vm, i decided to give zimbra a shot on metal. So i installed Zimbra on my old web server (after stripping out all of my web related accounts and services of course). Performance was better, and zimbra seemed happier. So i set about trying to find a nice clean way to move all of my zimbra data from the VM, to the new install.
Well, Open Source Edition does not include the nice backup features of Network Edition, which would have probably made this a lot easier. So I was left to find my own means. I found some talk on the zimbra forums about how this might be accomplished. Simply install zimbra on the target server, then copy /opt/zimbra from the source server, to the destination server. This depends on similar configurations on both ends though, and this did not end well for me. After hours of work (and hours of an scp copy) i ended up with a zimbra install that appeared to function, but postfix was hosed beyond belief. This seems at least partially due to the fact that my target system had a different hostname than the source system. This was almost a necessity for me, as my vm server, and my old web server are located in different datacenter's, many many miles apart.
So i decided to give something else a shot. Let's take zimbra's ldap, mysql, and store, and just tear it out of the source and import it into the target. Here's how i did it:
I started out with two servers. One the production server (source, "interchange") and one the newly setup server (target, "stimpson"). On the target, i installed zimbra, fresh and clean. I did a few customizations during the install such as time zone, and alert e-mails, but nothing fancy.
First, you'll want to stop your MTA on the source, so new mail coming in is queued at the sender. This way you're sure not to miss any mail.
Now, zimbra is still running, but it's not accepting mail. Any mesages sent to our server should be queued at the sender, and re-tried later. Don't leave your system in this state for too long however, as you have no idea how long the sending server will queue these messages. A few hours is probably a safe bet.
Now, backup all of your data.
ldap:
Zimbra, in zcs 6, includes a slapcat like script, for just this purpose.
This places an ldap.bak file in /tmp/foo. This is an ldif file, you can rename it. I called mine interchange.ldif
Mysql:
I had to create a script for mysqldump, this was included in earlier versions of zimbra, but they removed it. I'm not sure why.
I took ~zimbra/bin/mysql and copied it to ~zimbra/bin/mysqldump, and then modified it such:
Then, just dump mysql, as you would on any mysql server, you do not need to specify login credentials however, you'll see, that s included in the script.
Store:
The store is simple (this includes the index as well).
Now, however its convenient for you, get all of these files to the target server.
On the target, stop zimbra.
Now, i had to do some modification to my data files. because of hostname changes and whatnot.
If you want to make a backup of /opt/zimbra in case something goes wrong, and you need to start with a fresh install, now's the time to do it.
Ldap contains all of your account data, AND a lot of your configuration. I found it cleanest to strip all but the account data out. This was done by beheading the ldif of the first 1615 lines. You may need to look at your ldif and see if this is the right number for you, i found that an ldif i slapcatted from the target server (for comparison) had about 100 fewer lines that needed to be hacked off, so you'll have to look. I found in the ldif file where the COS definitions started, and hacked off every thing before that. This worked mostly well, i did run into one issue with one domain. I'll cover that later.
I also searched for any occurance of "interchange" and swapped it with "stimpson".
The sql file we dumped also needs to be updated. A simple global search/replace for your old hostname, to your new hostname is all you need. In my case, i searched for interchange, and replaced it with stimpson.
Now, the real fun begins.
Import ldap:
Be sure zimbra is stopped (see above).
Import your ldif with the following command:
I did not capture the output of this command, it returns a progress bar, and then a success message.
Mysql:
In order to import mysql, you'll need mysql running. It's safe to start up zimbra at this point, so go ahead.
You'll need to grab some information from your source, and target servers now. Namely, the mysql password. It will be different from one server to the next, and once you import your mysql dump, it will be reset (unless you go into the .sql file, and remove the piece which resets it.. up to you).
Yes, i've changed these passwords, theyre not actually bob and joe, i cant go giving my mysql password out to the world. What you'll have here is some long, random, password. Make note of both of them, and which servers they belong to.
Once zimbra is started, mysql will also be running, so now we can import our database.
Once its imported, you'll find that you can no longer access mysql.
This is because zimbra@localhost's password has changed. The simplest way around this is to take zimbra's mysql script, and hack it up again.
So i took ~zimbra/bin/mysql and copied it to ~zimbra/bin/mysql.mod then made the following change
Notice that --password says bob now.
Now connect to mysql, and issue the following commands. This is to reset zimbra's mysql password to what it was before the import.
Now mysql should be accessible to zimbra again.
Now, the store, this is the easy part.
Stop zimbra, you may not really need to do this, but it feels cleaner to me.
Depending on the path included in your tar archive (remember, mine was /opt/zimbra/store) you'll want to change to the appropriate directory. Since mine includes the absolute path, i'm moving to /, as root.
Your store will decompress, in my case to /opt/zimbra/store
Once it's complete, reset perms on /opt/zimbra store so that its owned by zimbra:zimbra.
Done! Now you should be set to go, start up zimbra!
Now you should be able to re-direct DNS, so that your MX for your zimbra domains are pointing to the proper place. Also, whatever your webmail addresses are should be updated.
For good measure, i also chkconfig off'd zimbra on the source server.
Issues i had:
I mentioned earlier that one of my domains didnt transfer over properly. I have about a half a dozen domains hosted. One of them, which happens to be the first domain i setup, didnt come over completely. All of the mailboxes were there, and i could login to them via webmail,. The only time i saw an issue was when i viewed the domains in the zimbra admin, or if a piece of e-mail came in for one of the addresses at the broken domain. They'd get bounced.
It seems to me that the ldap data i imported must have been incomplete. I was able to fix it rather easily though.
If i used zmprov to view all domains, the domain was listed.
But, if i tried to view detail on the broken domain, it returned an error stating that the domain didnt exist.
I assumed that zimbra added some sanity checking into their zmprov utility, and thought i'd give a simple modification to the domain a shot, and see if this fixed anything.
I renamed the domain via zmprov, and then renamed it back to its original name.
This resolved the issue.
Update:
See my thread on zimbra's forums for another method. Looks like I may have been able to accomplish this using the "Copy zimbra's home dir" method if i'd know about zmsetservername!
Ah well, i like my method better anyway.
Update:
I found that my zimbra documents were not working, probably due to the wiki account changing.
The fix is simple, once you track it down.
Find out what your global wiki account is, you can get it with the following command:
(domain.name) is your actual top level domain, of course. this is generally the fqdn of the first domain you setup, which is generally the fqdn of your zimbra server.
Now do the following:
I installed a web vm, which works nicely, a second web vm for a friend, and finally, a Zimbra VM. The Zimbra vm was troublesome. My hardware was old, it wasn't the best platform for virtualization, and zimbra seems unhappy in a xen VM.
After i finished migrating all of my data from my old web server to the new web vm, i decided to give zimbra a shot on metal. So i installed Zimbra on my old web server (after stripping out all of my web related accounts and services of course). Performance was better, and zimbra seemed happier. So i set about trying to find a nice clean way to move all of my zimbra data from the VM, to the new install.
Well, Open Source Edition does not include the nice backup features of Network Edition, which would have probably made this a lot easier. So I was left to find my own means. I found some talk on the zimbra forums about how this might be accomplished. Simply install zimbra on the target server, then copy /opt/zimbra from the source server, to the destination server. This depends on similar configurations on both ends though, and this did not end well for me. After hours of work (and hours of an scp copy) i ended up with a zimbra install that appeared to function, but postfix was hosed beyond belief. This seems at least partially due to the fact that my target system had a different hostname than the source system. This was almost a necessity for me, as my vm server, and my old web server are located in different datacenter's, many many miles apart.
So i decided to give something else a shot. Let's take zimbra's ldap, mysql, and store, and just tear it out of the source and import it into the target. Here's how i did it:
I started out with two servers. One the production server (source, "interchange") and one the newly setup server (target, "stimpson"). On the target, i installed zimbra, fresh and clean. I did a few customizations during the install such as time zone, and alert e-mails, but nothing fancy.
First, you'll want to stop your MTA on the source, so new mail coming in is queued at the sender. This way you're sure not to miss any mail.
[zimbra@interchange ~]$ zmmtactl stop
Now, zimbra is still running, but it's not accepting mail. Any mesages sent to our server should be queued at the sender, and re-tried later. Don't leave your system in this state for too long however, as you have no idea how long the sending server will queue these messages. A few hours is probably a safe bet.
Now, backup all of your data.
ldap:
Zimbra, in zcs 6, includes a slapcat like script, for just this purpose.
[zimbra@interchange ~]$ /opt/zimbra/libexec/zmslapcat /tmp/foo
This places an ldap.bak file in /tmp/foo. This is an ldif file, you can rename it. I called mine interchange.ldif
Mysql:
I had to create a script for mysqldump, this was included in earlier versions of zimbra, but they removed it. I'm not sure why.
I took ~zimbra/bin/mysql and copied it to ~zimbra/bin/mysqldump, and then modified it such:
[zimbra@interchange ~]$ cat ~zimbra/bin/mysqldump
#!/bin/bash
#
# **** BEGIN LICENSE BLOCK ****
#
# Zimbra Collaboration Suite Server
# Copyright (C) 2004, 2005, 2006, 2007 Zimbra, Inc.
#
# The contents of this file are subject to the Yahoo! Public License
# Version 1.0 ("License"); you may not use this file except in
# compliance with the License. You may obtain a copy of the License at
# http://www.zimbra.com/license.
#
# Software distributed under the License is distributed on an "AS IS"
# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.
#
# **** END LICENSE BLOCK ****
#
source `dirname $0`/zmshutil || exit 1
zmsetvars mysql_directory mysql_socket zimbra_mysql_user zimbra_mysql_password
exec ${mysql_directory}/bin/mysqldump -S ${mysql_socket} \
-u ${zimbra_mysql_user} --password=${zimbra_mysql_password} "$@"
Then, just dump mysql, as you would on any mysql server, you do not need to specify login credentials however, you'll see, that s included in the script.
[zimbra@interchange ~]$ mysqldump --all-databases > /tmp/foo/interchange.sql
Store:
The store is simple (this includes the index as well).
[zimbra@interchange ~]$ tar czf /tmp/foo/interchange_store.tar.gz /opt/zimbra/store /opt/zimbra/index
Now, however its convenient for you, get all of these files to the target server.
On the target, stop zimbra.
[zimbra@stimpson ~]$ zmcontrol stop
Host stimpson.undrground.org
Stopping stats...Done.
Stopping mta...Done.
Stopping spell...Done.
Stopping snmp...Done.
Stopping archiving...Done.
Stopping antivirus...Done.
Stopping antispam...Done.
Stopping imapproxy...Done.
Stopping memcached...Done.
Stopping mailbox...Done.
Stopping logger...Done.
Stopping ldap...Done.
Now, i had to do some modification to my data files. because of hostname changes and whatnot.
If you want to make a backup of /opt/zimbra in case something goes wrong, and you need to start with a fresh install, now's the time to do it.
[root@stimpson /opt]# cp -rp zimbra/ zimbra.fresh
Ldap contains all of your account data, AND a lot of your configuration. I found it cleanest to strip all but the account data out. This was done by beheading the ldif of the first 1615 lines. You may need to look at your ldif and see if this is the right number for you, i found that an ldif i slapcatted from the target server (for comparison) had about 100 fewer lines that needed to be hacked off, so you'll have to look. I found in the ldif file where the COS definitions started, and hacked off every thing before that. This worked mostly well, i did run into one issue with one domain. I'll cover that later.
I also searched for any occurance of "interchange" and swapped it with "stimpson".
The sql file we dumped also needs to be updated. A simple global search/replace for your old hostname, to your new hostname is all you need. In my case, i searched for interchange, and replaced it with stimpson.
Now, the real fun begins.
Import ldap:
Be sure zimbra is stopped (see above).
Import your ldif with the following command:
[zimbra@stimpson ~]$ ~zimbra/openldap-2.4.21.3z/sbin/slapadd -F ~zimbra/data/ldap/config -l interchange.ldif
I did not capture the output of this command, it returns a progress bar, and then a success message.
Mysql:
In order to import mysql, you'll need mysql running. It's safe to start up zimbra at this point, so go ahead.
[zimbra@stimpson ~]$ zmcontrol start
Host stimpson.undrground.org
Starting ldap...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting antispam...Done.
Starting antivirus...Done.
Starting snmp...Done.
Starting spell...Done.
Starting mta...Done.
Starting stats...Done.
You'll need to grab some information from your source, and target servers now. Namely, the mysql password. It will be different from one server to the next, and once you import your mysql dump, it will be reset (unless you go into the .sql file, and remove the piece which resets it.. up to you).
[zimbra@interchange ~]$ zmlocalconfig -s | grep zimbra_mysql_password
zimbra_mysql_password = bob
[zimbra@stimpson ~]$ zmlocalconfig -s | grep zimbra_mysql_password
zimbra_mysql_password = joe
Yes, i've changed these passwords, theyre not actually bob and joe, i cant go giving my mysql password out to the world. What you'll have here is some long, random, password. Make note of both of them, and which servers they belong to.
Once zimbra is started, mysql will also be running, so now we can import our database.
[zimbra@stimpson ~]$ mysql < interchange.sql
Once its imported, you'll find that you can no longer access mysql.
[zimbra@stimpson ~]$ mysql
ERROR 1045 (28000): Access denied for user 'zimbra'@'localhost' (using password: YES)
This is because zimbra@localhost's password has changed. The simplest way around this is to take zimbra's mysql script, and hack it up again.
So i took ~zimbra/bin/mysql and copied it to ~zimbra/bin/mysql.mod then made the following change
[[zimbra@stimpson ~]$ cat ~zimbra/bin/mysql.mod
#!/bin/bash
#
# **** BEGIN LICENSE BLOCK ****
# Zimbra Collaboration Suite Server
# Copyright (C) 2004, 2005, 2006, 2007, 2009, 2010 Zimbra, Inc.
#
# The contents of this file are subject to the Zimbra Public License
# Version 1.3 ("License"); you may not use this file except in
# compliance with the License. You may obtain a copy of the License at
# http://www.zimbra.com/license.
#
# Software distributed under the License is distributed on an "AS IS"
# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.
# **** END LICENSE BLOCK ****
#
source `dirname $0`/zmshutil || exit 1
zmsetvars mysql_directory mysql_socket zimbra_mysql_user zimbra_mysql_password
exec ${mysql_directory}/bin/mysql -S ${mysql_socket} \
-u ${zimbra_mysql_user} --password=bob "$@"
Notice that --password says bob now.
Now connect to mysql, and issue the following commands. This is to reset zimbra's mysql password to what it was before the import.
[zimbra@stimpson ~]$ mysql.mod
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 328
Server version: 5.0.90-log Source distribution
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> set password for 'zimbra'@'localhost' = password('joe');
Query OK, 0 rows affected (0.02 sec)
mysql> set password for 'zimbra'@'localhost.localdomain' = password('joe');
Query OK, 0 rows affected (0.00 sec)
mysql> set password for 'zimbra'@'%' = password('joe');
Query OK, 0 rows affected (0.00 sec)
mysql> quit
Bye
Now mysql should be accessible to zimbra again.
Now, the store, this is the easy part.
Stop zimbra, you may not really need to do this, but it feels cleaner to me.
Depending on the path included in your tar archive (remember, mine was /opt/zimbra/store) you'll want to change to the appropriate directory. Since mine includes the absolute path, i'm moving to /, as root.
[root@stimpson /]# tar xvzf interchange_store.tar.gz
Your store will decompress, in my case to /opt/zimbra/store
Once it's complete, reset perms on /opt/zimbra store so that its owned by zimbra:zimbra.
[root@stimpson /]# chown -R zimbra:zimbra /opt/zimbra/store /opt/zimbra/index
Done! Now you should be set to go, start up zimbra!
Now you should be able to re-direct DNS, so that your MX for your zimbra domains are pointing to the proper place. Also, whatever your webmail addresses are should be updated.
For good measure, i also chkconfig off'd zimbra on the source server.
Issues i had:
I mentioned earlier that one of my domains didnt transfer over properly. I have about a half a dozen domains hosted. One of them, which happens to be the first domain i setup, didnt come over completely. All of the mailboxes were there, and i could login to them via webmail,. The only time i saw an issue was when i viewed the domains in the zimbra admin, or if a piece of e-mail came in for one of the addresses at the broken domain. They'd get bounced.
It seems to me that the ldap data i imported must have been incomplete. I was able to fix it rather easily though.
If i used zmprov to view all domains, the domain was listed.
zmprov getalldomains
But, if i tried to view detail on the broken domain, it returned an error stating that the domain didnt exist.
zmprov getdomain brokendomain.com
I assumed that zimbra added some sanity checking into their zmprov utility, and thought i'd give a simple modification to the domain a shot, and see if this fixed anything.
I renamed the domain via zmprov, and then renamed it back to its original name.
zmprov -l renameDomain brokendomain.com brokeddomain.com
zmprov -l renameDomain borkeddomain.com brokendomain.com
This resolved the issue.
Update:
See my thread on zimbra's forums for another method. Looks like I may have been able to accomplish this using the "Copy zimbra's home dir" method if i'd know about zmsetservername!
Ah well, i like my method better anyway.
Update:
I found that my zimbra documents were not working, probably due to the wiki account changing.
The fix is simple, once you track it down.
Find out what your global wiki account is, you can get it with the following command:
[zimbra@stimpson ~]$ zmprov gacf | grep zimbraNotebookAccount
zimbraNotebookAccount: wiki@stimpson.(domain.name)
(domain.name) is your actual top level domain, of course. this is generally the fqdn of the first domain you setup, which is generally the fqdn of your zimbra server.
Now do the following:
[zimbra@stimpson ~]$ zmprov in wiki@stimpson.(domain.name)
Initializing folders
Creating folder Template
[zimbra@stimpson ~]$ zmprov ut -h stimpson.(domain.name) /opt/zimbra/wiki/Template/updating global wiki account wiki@stimpson.(domain.name)
Initializing...
Creating wiki page _PathBodyTemplate in folder Template
Creating wiki page _Index in folder Template
Creating wiki page _TocItemTemplate in folder Template
Creating wiki page _TitleBar in folder Template
Creating wiki page _PathItemTemplate in folder Template
Creating wiki page _VersionTitleBar in folder Template
Creating wiki page _Template in folder Template
Creating wiki page _VersionIndex in folder Template
Creating wiki page _TemplateStyles_unused in folder Template
Creating wiki page _SideBar in folder Template
Creating file document Logo.gif in folder Template
Creating wiki page _VersionTemplate in folder Template
Creating wiki page _TocVersionBodyTemplate in folder Template
Creating wiki page _TocVersionItemTemplate in folder Template
Creating wiki page _TemplateStyles in folder Template
Creating wiki page _Footer in folder Template
Creating wiki page _Header in folder Template
Creating wiki page _TocBodyTemplate in folder Template
Creating wiki page _PathSeparator in folder Template
