Re: Subsetting text files containing e-mails

[GrandFather]

To an extent you can translate bash into Perl. The control structures (if and loops) translate without much trouble. Many of the system utilities that you'd be using in your bash script have Perl equivalents, although they aren't drop in replacements and you are right to guess there are better ways in Perl than the pipelined filter processing technique you likely used with bash.

With Perl you will tend more to parse through the input file essentially a line at a time to find the "interesting bits" and generate output as you go. The thing is to be able to recognise an interesting bit before you have moved on to the next line. Perl has a neat trick that makes that pretty easy in your case. I apologise in advance for spoilers - the following is most of the solution you need so more than you asked for:

#!/usr/bin/perl
use strict;
use warnings;

my $emailNum;

$/ = ''; # Set readline to "Paragraph mode"

while (<DATA>) {
    if (!$emailNum || /^From:/im) {
        ++$emailNum;

        print "---- Email $emailNum\n";
    }

    print;
}

__DATA__
From: here
To: there
Data:
I have a number of e-mail messages (envelope, headers, body, sometimes
+ including
base64-encoded attachments) which are spread out among an arbitrary nu
+mber of
text files. In most cases, there are multiple messages in each text fi
+le.

I want to subset each large file so that each e-mail is saved to its o
+wn file.
So, if the source file is "mails.txt" containing three messages, I wan
+t to
create (e.g.) mails_000001.txt, mails_000002.txt, and mails_000003.txt
+.

From: somewhere
To: elsewhere
Data:
A second email

Prints:

---- Email 1
From: here
To: there
Data:
I have a number of e-mail messages (envelope, headers, body, sometimes
+ including
base64-encoded attachments) which are spread out among an arbitrary nu
+mber of
text files. In most cases, there are multiple messages in each text fi
+le.

I want to subset each large file so that each e-mail is saved to its o
+wn file.
So, if the source file is "mails.txt" containing three messages, I wan
+t to
create (e.g.) mails_000001.txt, mails_000002.txt, and mails_000003.txt
+.

---- Email 2
From: somewhere
To: elsewhere
Data:
A second email

See perlvar for a description of what $/ is doing.