Advice for email munging

[Limbic~Region]

All:
I am looking for a way to modify an email message in transit. In all cases I will be modifying the "to", "from", and some other headers. In some cases - I will be removing a string from the body of the message. In other cases - I will be adding a string to the body of the message.

I have been using Mail::Audit (which uses MIME::Entity if needed) as a starting point, and am going to throw some snippets to give you an idea (in no particular order):

my $mail = Mail::Audit->new(data => \@message);

$mail->bodyhandle;
my @body = $mail->bodyhandle->as_lines;
if ($msgtyp ne 'reply') {
    my $orig_from = $mail->get('Reply-To') ? $mail->get('Reply-To') : 
+$mail->get('From');
    unshift @body , "RPLYFROM: $replyfrom";
    unshift @body , "ORIGFROM: $origfrom";
    if (my $io = $mail->open("w")) {
        $io->print($_) foreach(@body);
        $io->close;
    }
}

$mail->delete_header('Reply-To') if ($mail->delete_header('Reply-To'))
+;
$mail->delete_header('Received'); 
$mail->delete_header('Message-Id'); 
$mail->replace_header('From', $orig);
$mail->replace_header('To', $recip);

$mail->resend($recip);

Ok - you get the idea - I munge the data, so what's the problem? I am not sure how to handle MIME body correctly. What I would like to say is:

Remove string x from the body and have it figure out how to handle the plain text and/or MIME

Add string x to the body and have it figure out how to handle the plain text and/or MIME

I thought I was pretty close until I encountered "Content-Type: multipart/alternative;". There are basically multiple pieces to the body (in my case plain text and HTML). The email client chooses the one it likes.

Now I am parsing HTML which I hadn't counted on (another module)

I have no idea how to make sure both pieces are updated so that it doesn't matter which body the client chooses.

I am grateful for any help that I can get, but what is most desireable is a working template. I can handle modifying all the headers - it is the body and specifically MIME body that is causing me grief.

Thanks in advance - L~R

[tedrek]

What I would do is find the first text/plain part in the message, munge that and use it as the sole body in the outgoing message. the reason for that is you can be almost certain that the mail will contain a text/plain part and by removing all other parts you don't have to worry about ppl seeing them. If you need to munge every single part you have your work cut out for you. You could theoretically have a alternative part which is sound, not to mention mixed inside alternative inside mixed. And I have seen text/rtf inside a multipart/alternative

As for a full strategy I'd probably go OO style and make a object for each mime type which has functions for each of the actions you need to perform eg $txt->replace_string('foo', 'bar'). Then container types would call the functions on whatever they contain. Any type you don't have a handler for gets dropped from the message when you parse it in. So now your munging is basically $body->replace_string('foo', 'bar') and that's all in one place which just works on all the different types you support.

[Limbic~Region]

tedrek,
I appreciate your advice and your reply. Unfortunately, I have no idea how to implement. I am sorry if I gave the false impression that I knew what I was doing with my code snippets - I am figuring this out as I go. That is why I was hoping for a working template. What I don't understand how to do (in code) is to figure out what pieces I can throw away, which pieces I need to keep, and how to make sure when I reconstruct the message I have done it properly. I have no issues with dropping the HTML piece (or RTF) as long as there is a plain text piece, but is it still mixed/alerternate at that point? What happens if it is MIME and the only thing I get is HTML?

I know this is my problem and not yours. I guess I have more questions than I have answers. I can make the code work if I get a specific type of message - I don't know how to say - DWIM to my code.

Thanks again, L~R

[tedrek]

Ok, I'm working with this under the assumption that it is safe, even desirable to throw away parts you can't change. eg sound, archives, rtf, or html. If that is the case the problem becomes 'How do I make a message with containing only text/plain from a message which may contain any type'. This can be broken down into 3 parts 'Find all the text/plain parts', 'Edit them', 'Put all edited parts back in the body'. I think you have the 'Edit them' part covered basically which leaves us with two things to do. so let's tackle the first:

There's two cases I see, a body containing one thing (hopefully text/plain) you already know how to handle this right? the other case is a body containing 'multipart/mixed' or 'multipart/alternative' which are pretty much interchangeable for us. In this case we can break it down to 'get each part inside the multipart', 'look at each part', and 'save for editing if part is text/plain'. I think this code should show my basic idea.

my @parts;

if ($msg->is_mime) {
    my @t_parts = $msg->parts();
    while (shift @t_parts) {
        if ($_->mime_type eq 'text/plain') {
             push @parts, $_;
             next;
        }
        if ($_->mime_type eq 'multipart/mixed' ||
            $_->mime_type eq 'multipart/alternative') {
             push @t_parts, $_->parts();
        }
    }
}

then you just edit each item in @parts. and assemble a new message with the headers from the old one and a new body containing whatever parts you have edited. If there is only one piece then the message content-type is 'text/plain' if there is more than one piece the type is probably 'multipart/mixed'. You will probably have to be aware of multipart/alternative with multiple text/plain inside, but that case isn't very likely. As for messages that don't contain any text/plain... it's probably spam or hotmail/yahoo. You may find that you need to capture the text/html parts, but that should be relatively easy to add.

an alternative strategy might be basically the same thing but instead of saving the text/plain parts delete anything that isn't text/plain or a multipart containing a text/plain. This way you don't need to worry about which multipart a text part was supposed to go into. I believe it is legal to have a multipart/alternative with only a single part inside it.

As for making sure the message is reconstructed properly.. that's why you are using Mail::Audit and MIME:Entity in the first place :). I hope that helps you some. and note I haven't tested any of the code. _{as a side note if you really want your code to say DWIM check out Acme::DWIM} :)

[allolex]

One thing that occurs to me is that if the MIME body contains HTML, you could strip the plain text message out completely, alter the HTML as you need it, and then insert a new plain text message derived from your altered HTML. I suggest this as an alternative to stripping the HTML because the more simple formats can be derived from the more complex ones with relatively little extra effort (it doesn't work in the other direction). I guess I'd suggest HTML::TokeParser or HTML::TokeParser::Simple for that job.

--
Allolex