ldap database went from 97meg to 86gig

Printable View

Show 40 post(s) from this thread on one page

11-23-2012, 05:05 PM
ccelis5215

Quote:

Originally Posted by quanah

I would suggest you fix your backup strategy to what OpenLDAP supports. Which is to export the database, via slapcat, to an LDIF file.

Well, have to agree.

But, again , i'm just requesting a sizable parameter accordingly a vast base SMB users.

Thanks!

ccelis
11-23-2012, 05:12 PM
ccelis5215

ok but

Kevin, agree with your comments too, at least technical's to keep this discussions polite. I'm not a OpenLDAP expert but have to agree that using this new release improve performance and security.

It's supposed that Zimbra has to inform all users about changes and implications.

As we stated and demonstrated in this thread, there's a huge impact in our Zimbra deplyments and migration planning.

ccelis
11-23-2012, 05:37 PM
kevindods

yep, just want to get a good solution to this issue that works sensibly and to understand more about the reasons behind the changes that expose it.

For the last 6-7 years the rsync approach was de-facto and suddenly it is very significantly changed and in a way that seems to work against good storage management practice.

Presumably bad things will happen if the DB ever gets to 80GB and the storage space available is not there to write to. Strange that there is no requirement or auto-process to check for available drive space or indeed simple capacity on the partition or block device. If 80GB is a sensible size then surely the space should not just be allocated but protected and a specified requirement.

Surely if the DB is not able to grow dynamically and having potentially scalable capacity is this important it should be possible to have a process of export to LDIF, resize the DB allocation and import the LDIF data back in rather than have an arbitrary fixed high maximum for all cases.

Something seems a bit off with the process, I wonder if indeed it is not exhibiting the planned behaviour - Not sure where the benefit in having 80GB allocated in the FS is in this way.
11-23-2012, 05:40 PM
ccelis5215

Quote:

Originally Posted by kevindods

yep, just want to get a good solution to this issue that works sensibly and to understand more about the reasons behind the changes that expose it.

For the last 6-7 years the rsync approach was de-facto and suddenly it is very significantly changed and in a way that seems to work against good storage management practice.

Presumably bad things will happen if the DB ever gets to 80GB and the storage space available is not there to write to. Strange that there is no requirement or auto-process to check for available drive space or indeed simple capacity on the partition or block device. If 80GB is a sensible size then surely the space should not just be allocated but protected and a specified requirement.

Surely if the DB is not able to grow dynamically and having potentially scalable capacity is this important it should be possible to have a process of export to LDIF, resize the DB allocation and import the LDIF data back in rather than have an arbitrary fixed high maximum for all cases.

Something seems a bit off with the process, I wonder if indeed it is not exhibiting the planned behaviour - Not sure where the benefit in having 80GB allocated in the FS is in this way.

100% agreed!
11-23-2012, 06:06 PM
quanah

Please read https://bugzilla.zimbra.com/show_bug.cgi?id=78781#c4

I would also note that any person who "restored" their mdb.back database has completely undone the point of the 8.0.1 upgrade, which was to move the potentially corrupt LDAP database aside. If you have done this, you face a serious risk of ending up with a permanently corrupted LDAP database you will not be able to recover from.
11-23-2012, 07:39 PM
samgann

If the data base gets over 80 GIG? Mine is 86 GIG and I only have 550 email address and was 197meg. So does that mine my data base is going to be corrupt ? Why wast'nt this in the release note for 8.01?
Why didnt this happen when we upgraded to 8.0?
11-23-2012, 07:51 PM
quanah

It is not 86GB. It is whatever the actual usage on disk is. You need to understand the difference between allocation and usage. cd /opt/zimbra/data/ldap/mdb, then run du -c -h data.mdb. This will tell you the *actual* size of the database being used.
11-23-2012, 07:55 PM
samgann

The database file is 86 gig. The database could be using 197 meg of the 86 GIG but as far as disk space it's 86 GIG.
11-23-2012, 08:05 PM
samgann

Ok I ran du -c -h /opt/zimbra/data/ldap/mdb/db/data.mdb
5.6M /opt/zimbra/data/ldap/mdb/db/data.mdb
5.6M total
Ok the file size from the dir is was 197 meg. and now is 86GIG
Why would the database need to be 86Gig if I am only using 5.6meg ?
I am not trying to upset you. I am just trying to understand why?
11-23-2012, 08:22 PM
quanah

Clearly, the file is 5.6MB. It is not 86GB. I'm not sure where in the world you are getting 86GB from.

For example:

Code:

[zimbra@ldap01-zcs db]$ ls -l total 507216 -rw------- 1 zimbra zimbra 85899345920 Nov 23 20:19 data.mdb -rw------- 1 zimbra zimbra 8192 Nov 23 20:19 lock.mdb [zimbra@ldap01-zcs db]$ df -h . Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_zimbra-lv_zimbra 20G 3.9G 15G 21% /opt/zimbra [zimbra@ldap01-zcs db]$ du -c -h data.mdb 496M data.mdb 496M total

du -c -h clearly shows my database is 496MB in total size. ls -l clearly reports that the amount of space potentially allocated is 80GB. df -h clearly shows the full disk usage of the *entire* partition is 3.9GB out of 20GB total. Clearly, in no way, shape, or form, is 80GB being used anywhere. This is why you need to understand the difference between what is *allocated* as the *potential maximum used* versus what is *actually being used*. :)

Show 40 post(s) from this thread on one page