[SOLVED] Help with mail migration script

[The Fold]

Hi there,

I'm in the process of migrating users from Pegasus Mail to Zimbra.

My boss wrote a handy script which converted the not-quite-standard Pegasus mailbox format to MBOX which works great. He's also written a script to upload the MBOX files into Zimbra but keeps getting errors. I was wondering if anyone could shed some light on why this may not be working.

NOTE: Our zimbra server is just called 'zimbra'.

The script:

#!/usr/bin/perl

use strict;
use LWP::UserAgent;
use HTTP::Request::Common qw(POST);

if (!@ARGV) {
	die "usage: perl upload-mbox-to-zimbra.pl <user> <upload-user> <password> <mbox-files>";
}

my ($user, $pwd, $upload_user, @mbox_files) = @ARGV;

my $n = 0;
my $n_ok = 0;
my $n_errors = 0;

sub do_files {
	for my $file (@mbox_files) {
		do_file ($user, $pwd, $upload_user, $file);
	}
}

sub do_file {
	my ($user, $pwd, $upload_user, $file) = @_;
	if (!(open IN, '<', $file)) {
		print  "Couldn't open $file";
		return 0;
	}
	my $folder = $file;
	$folder =~ s|^.*/||;
	$folder =~ s/\.[A-Z]+$//i;
	$folder =~ s/[^\w-]//g;
	my $url = "http://zimbra/zimbra/home/$upload_user/$folder";
	print "Uploading $file to $folder ($url)\n";
	sub upload_message {
		my ($lines) = @_;
		do_message($user, $pwd, $url, $lines);
	}
	do_messages(\&upload_message);
	close IN;
}

sub do_messages {
	my ($handler) = @_;
	my @lines = ();
	while (my $line = <IN>) {
		if (@lines && ($line =~ m/^From /)) {
			&$handler(\@lines);
			@lines = ();
			print "DEBUGGING: Only doing 1 message\n";
			return 1;
		}
		push @lines, $line;
	}
	if ($#lines) {
		&$handler(\@lines);
	}
}

my $ua = LWP::UserAgent->new;

sub do_message {
	my ($user, $pwd, $url, $lines) = @_;
	$n++;
	print "\nMESSAGE $n: $#$lines lines\n";
	#print join('', @$lines) , "\n";
	
	my $req = HTTP::Request->new(POST => $url);
	$req->authorization_basic($user, $pwd);
	$req->content_type('application/x-www-form-urlencoded');
	$req->content(join('', @$lines));

	my $res = $ua->request($req);
	if ($res->as_string =~ /Client-Response-Num: (\d+)/) {
		print "Result: " , $1, "\n";
		if (!$1) {
			$n_ok++;
		} else {
			$n_errors++;
			my $html = $res->as_string;
			$html =~ s/.*\<body\>//is;
			$html =~ s/\<[^>]*\>/ /g;
			$html =~ s/\s\s+/ /g;
			print "DEBUGGING: " . $html . "\n";
		}
	} else {
		print $res->as_string, "\n";
		$n_errors++;
	}
}

do_files();

print "$n messages\n$n_ok OK\n$n_errors errors\n";

The error that occurs when running this script:

MESSAGE 1: 785521 lines
500 Server closed connection without sending any data back
Content-Type: text/plain
Client-Date: Tue, 15 Apr 2008 14:04:58 GMT
Client-Warning: Internal response

500 Server closed connection without sending any data back

1 messages
0 OK
1 errors

at one point there was also this error:

MESSAGE 1: 785521 lines
500 syswrite: Unknown error
Content-Type: text/plain
Client-Date: Tue, 15 Apr 2008 11:56:07 GMT
Client-Warning: Internal response

500 syswrite: Unknown error

1 messages
0 OK
1 errors

If someone could shed some light I'd be extremely greatful!

[uxbod]

Or you could try User Migration - Zimbra :: Wiki which have been used by many people.

[The Fold]

Right, so I tried the scripts as shown above. Turned out I the conversion script I was using wasn't making them into true mbox files. That's now been fixed.

The Python script seems to do the job nicely but when I try and perform a recursive upload I get the following output

Traceback (most recent call last):
  File "/tmp/mbox2imap.py", line 494, in <module>
    main()
  File "/tmp/mbox2imap.py", line 44, in main
    recursive_upload()
  File "/tmp/mbox2imap.py", line 138, in recursive_upload
    create_imap_mailboxes(target_list, server)
  File "/tmp/mbox2imap.py", line 299, in create_imap_mailboxes
    current_mailboxes = map(extract, current_mailboxes)
  File "/tmp/mbox2imap.py", line 297, in <lambda>
    extract = lambda x: re.search(r'^.*"\." "(.*)"', x).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Obviously I can upload each folder 1 at a time but that will be time consuming when I have 60 users to migrate.

Also is there a way of using the admin user to upload to other people's mailboxes?