[SOLVED] Blob error, again.

[jeschulze]

Hi,

I recently upgraded our server from a HP desktop on the floor to a rackmount unit with a detached SCSI array. I used the backup / restore method to transfer all the files and everything went fine until we try and access our old emails. We get:

msg: No such blob: mailbox=5, item=16439, change=26151
code: mail.NO_SUCH_BLOB
method: SearchConvRequest
detail: soap:Sender
trace: btpool0-16:1210093712442:b14763c69c0d74eb

The only major difference is that I have told zimbra that it should have its index and stores on /mnt/str/zimbra/store and /mnt/str/zimbra/index. Other posts with this issue say I should move the store folder back under /opt where it used to be, but zimbra itself resides on a smaller, 40 gig Raid5, and I'd rather keep the store on the larger array. This seams to be a common issue, so hopefully there is a common answer I have missed.

this is Zimbra 5.0.5 Network (upgraded from 5.0.4 Net during transition).

Thank you for your help!

Edit:

I noticed a drop in SDA's hd useage that points to what I believe the restoration program placing the restored messages in their old location under /opt and not the new /mnt/str/zimbra locations. The messages are visible, so the SQL and index seam to be in the right place, and pointing to the new store location, but the blobs are NOT. Could this be the case? and if so, how do I move the blobs to the new location where they can be found?

[jeschulze]

messages indeed were restored to the old location under /opt/zimbra/store, primary store added pointing to that, but still no luck.

Tech support is helping, but thought i'd post here just in case someone else makes the same mistake, or if anyone has had this happen to them and can offer a solution.

[mmorse]

By-the-way as you're NE you could have used HSM with a low value like a day or something...anyways:

Originally Posted by jeschulze

Edit:
I noticed a drop in SDA's hd useage that points to what I believe the restoration program placing the restored messages in their old location under /opt and not the new /mnt/str/zimbra locations. The messages are visible, so the SQL and index seam to be in the right place, and pointing to the new store location, but the blobs are NOT. Could this be the case? and if so, how do I move the blobs to the new location where they can be found?

Originally Posted by jeschulze

messages indeed were restored to the old location under /opt/zimbra/store, primary store added pointing to that, but still no luck.

Tech support is helping, but thought i'd post here just in case someone else makes the same mistake, or if anyone has had this happen to them and can offer a solution.

Ok, it sounds like you can at least see all your old mail now correct? But possibly stuff is now spread out in 2 locations and also new messages aren't stored in /mnt/str/zimbra/store (did u move index back?)

Could you run zmvolume --list and zmvolume --displayCurrent so we can see what your setup is?

Disclaimer: Do NOT run the below yet - I'm currently trying to wrap my head around everything you've done to this point, and if you've got a support ticket open it probably best if you continue that route.

You may be able to update the volume_id column in the mail_item table in every mboxgroup database - something like: mysql> for (N = 1 through 100) {UPDATE mboxgroupN.mail_item SET volume_id = IDofNew WHERE volume_id = IDofOld;}

The first N is the mailbox id divided by 100, you definately need to read up on: Account mailbox database structure - Zimbra :: Wiki

If you wanted to poke around, some basic structure:
su - zimbra
mysql
show databases;
use mboxgroupN
show tables;
select * from mail_item where volume_id=1;
That's a BIG query btw! So probably better if you selected by mailbox_id or id - narrow this down and try stuff on a test account or better yet an individual msg before you run any update/set/alter commands across every account.

Basic commands: Pantz.org Technical Reference Site

[jeschulze]

output is as follows:

Volume id: 1
name: message1
type: primaryMessage
path: /mnt/str/zimbra/store
compressed: false
current: true

Volume id: 2
name: index1
type: index
path: /mnt/str/zimbra/index
compressed: false
current: true

Volume id: 3
name: message2
type: primaryMessage
path: /opt/zimbra/store
compressed: false
current: false

Volume id: 4
name: index2
type: index
path: /opt/zimbra/index
compressed: false
current: false

Yes, HSM. After the upgrade and setup I looked it up, and realized that HSM should have been the route to take. My new mail appears to be going to /mnt/str/zimbra/stores, according to the sizes of the stores. I can see the mail, i just get the blob error when it is selected. I have not moved anything, but as you can see, i created a reference to both the index and stores in /opt/zimbra there.

What I've done so far:

fresh install of 5.05 net nothing out of the ordinary there.
created directories on mounted Array of /mnt/str/zimbra/store and index
assigned the directories in the admin console to their respective index/store
imported old /opt/zimbra/backup dir
pointed at backup and issued 'restore all accounts' command
Tech support contacted, was told to create the /opt/zimbra/stores message store as primary, but not current.
and here we are. I've restarted the service a couple of times and added the /opt/zimbra/index.. index to see if that helps, and it does not.

Not that I'm going to try this, but what would happen if I copied the index and store directories to --> /mnt/str/zimbra/store and /index?

[mmorse]

Originally Posted by jeschulze

EDIT
What I've done so far:

fresh install of 5.05 net nothing out of the ordinary there.
created directories on mounted Array of /mnt/str/zimbra/store and index
assigned the directories in the admin console to their respective index/store
imported old /opt/zimbra/backup dir
pointed at backup and issued 'restore all accounts' command
Tech support contacted, was told to create the /opt/zimbra/stores message store as primary, but not current.
and here we are. I've restarted the service a couple of times and added the /opt/zimbra/index.. index to see if that helps, and it does not.

Not that I'm going to try this, but what would happen if I copied the index and store directories to --> /mnt/str/zimbra/store and /index?

So if your current /opt/zimbra/store has volume_id 3 (that's message2) then you created it as a NEW volume later.

Your mail_items in the database (with files currently sitting in /opt/zimbra/store message2) might still all have volume_id 1 because you just change the PATH of message1 from /opt/zimbra/store to /mnt/str/zimbra/store.

You didn't actually MOVE the data from /opt/zimbra/store > /mnt/str/zimbra/store correct?

Thus I think moving your current contents of /opt/zimbra/store to /mnt/str/zimbra/store should work. (Re-indexing might be needed.)

Don't touch the index settings yet - on your 'first step' did you actually move the data from your old /opt/zimbra/index to /mnt/str/zimbra/index or just change the path? I can't tell what you actually did with that index data or if it's split between 2 locations.

[mmorse]

Support is having a look at your box right now - so I would just let them handle it

[jeschulze]

Yes, the problem was I just moved the backup data to a temporary directory, and then pointed the restore program at it. The restore program put the data back in /opt/zimbra, where it had been on the old box. support had me merge the /opt/zimbra stores with /mnt/str/zimbra/stores, and reindex all mailboxes. this took the better half of the wee hours of the morning, but it is not working! Thanks for helping me understand this, Mmorse. I am marking this case solved.

[mmorse]

Originally Posted by jeschulze

This took the better half of the wee hours of the morning, but it is not working! Thanks for helping me understand this, Mmorse. I am marking this case solved.

Based on your last sentence I gather you meant to say"it is now working"? If so, be sure to reply to your support ticket email so they can close the case.

[jeschulze]

uhh.. yea, thats what 3 am in front of a SSH window will do to you. it _IS_ working. sorry there.