http://www.perlmonks.org?node_id=409615

Sclous has asked for the wisdom of the Perl Monks concerning the following question:

How to go about using Email via Perl to append to a changelog.txt, html or .xls file? The situation, we have a team of 12 people and have been tasked with updating a logfile when we make changes. Right now, despite my clever co-worker who knows Perl better than I, he has us manually opening a spreadsheet. Not surprisingly, very few people do this, and the log is useless. I've seen blogs that allow update via email, so I thought it would be faster (and a demonstration of the wisdom of the Perl Monks ) if I could find a script that would take the input of an email, read it, and append it to this text file. The file might also be HTML, or anything else. The process needs to get the content of the email, and post a time at the start of the log. I've found 2 possible scripts but they were 2 years old, and I felt sure there must be an easier way... Thanks!
  • Comment on using Email to Perl to append to a changelog.txt, html, or .xls?

Replies are listed 'Best First'.
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by toma (Vicar) on Nov 22, 2004 at 15:38 UTC
    I built a system like this once. It used a mailto: URL when a form was submitted. The mail was processed by procmail, which called a perl program to do the file update. The program used RCS so that the changes could be reversed if there was a problem.

    The trick is to get file locking to work properly, so that two pieces of mail don't have the file open for writing at the same time. In my case, procmail handled this for me automatically.

    On unix at least, mail is in some ways a better mechanism for web form submissions than HTTP POST or GET. It is more complicated to program, but it is more reliable since the clients do automatic retries when the server is down. It is best suited for jobs like yours, where you submit a single piece of data that needs to be saved, not an interactive interface.

    It should work perfectly the first time! - toma
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by DaWolf (Curate) on Nov 22, 2004 at 16:22 UTC
    In theory you have to do the following:
    1. Configure the server so when it receives an e-mail from specified addresses it should execute the Perl script passing the content of the e-mail as a param;
    2. Make the Perl script read this param, discard 'useless' information from it (like the mail headers) and append the rest to a file. If you're talking about an Excel spreadsheet, you'll probably consider using Win32::OLE for the append step.

    I'll have to do something similar probably a month from now, so if you still need assistance then we can work together, let me know.

    If you are still looking for something that was already made, keep googling. I'm sure that there's a lot of stuff out there.

    Regards,
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by fglock (Vicar) on Nov 22, 2004 at 15:22 UTC

    How about using CVS or SVN (control version systems) ?
    There are many Perl support modules available in CPAN.

      good stuff, granted, but overkill. we want it to be as easy as, "I just changed the firmware on all the switches to version X" send email. Then when the network has crashed, we can remind the guy to send the email first, then crash the network :)
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by Xenograg (Scribe) on Nov 22, 2004 at 17:04 UTC
    I avoided (but did not solve) the file lock issue on a project by coding a script to access a POP3 mailbox, and using cron to control execution. The mailbox was checked once every 30 minutes with a script that ran for 30 seconds, so simultaneous file access was high unlikely--but not impossible.
      The mailbox was checked once every 30 minutes with a script that ran for 30 seconds, so simultaneous file access was high unlikely--but not impossible.
      Hmmm. If you are 'reading' a mailbox, as a mail client might, then it seems you wouldn't run into the need to lock the file.

      I'd would approach the problem like so:

      my $mboxfhandle; # open $mboxfhandle for readonly my @mbox = <$mboxfhandle>; # Let's use lots of memory # rest of the process ...
      Seems like this would catch each new message, and not conflict with any incoming messages.

      ( on further reflections, I imagine that a mail client must be able to mark messages as read, which would involve a write ... )

      -------------------------------------
      Nothing is too wonderful to be true
      -- Michael Faraday

        I meant that simultaneous file access upon the *output file* was high unlikely--but not impossible--because the script should run for only 30 seconds every 30 minutes.
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by TomDLux (Vicar) on Nov 22, 2004 at 15:25 UTC

    If you're using Unix, you don't even need a perl Script, you should be able to do it within the configuration file, 'aliases.txt'.

    --
    TTTATCGGTCGTTATATAGATGTTTGCA

      we need to update a html file on windows file server. :( long story that.
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by DaWolf (Curate) on Nov 22, 2004 at 22:06 UTC
    ++ Xenograg. I've completely fogot about this approach.

    Just to expand the idea a little bit, you (the OP) could create an e-mail address specific for this task (assuming the mail server resides on the same machine as the Perl log script*). For an example system_log@your_domain.com.

    Then you can, as Xenograg wisely suggested, make a script that checks this e-mail account and do the append stuff. This approach has at least two advantages from my original post:
    1. You don't have to make a "valid e-mail addresses" list, since this account will only receive e-mails from inside the company and therefore won't be "exposed" (if I've understood correctly).
    2. You can even treat information that was or not logged by looking if the e-mail message was already read or not.

    * I've stated this for the sake of performance and to the application to be practical. If the mail server is in another machine I think this could get a little messy.

    Best regards,
      is there a free standing perl email reciever, like blat is to sending mail? the email server is on a different box.

      The Net::POP3 module can access a POP3 mail account on any mail server accessible via TCP/IP.

      The Perl script will need direct file access to the (HTML) output file.

      Agreed, this would be my choice as well. While my first inclination would be to configure the mail server to launch a perl script when a particular address received mail, having a client read the mail gives you a lot of flexibility regarding where and when the perl script is executed.

      Of course, it's up to you to determine things like the "correct" subject header and proper formatting of the message. But other than that it seems like a relatively simple deal.

      PS & FYI, I love Log::Dispatch for writing messages.
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by Qiang (Friar) on Nov 24, 2004 at 00:19 UTC
    another approach - set up a local wiki site and have everyone work on that.

    It has great potential for document collection and other collaboration. changelog is also well fit.

    kwiki and twiki are both written in Perl.

Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by hsinclai (Deacon) on Nov 24, 2004 at 06:03 UTC
    I believe I am reading your OP correctly and your intention, so, here is a mail responder code I wrote (which you might/could use as a template, altering it of course for your specific needs).

    To use it, you must control the receiving email server. Invent an aliased user name to which any changes should be emailed, and map the script name in the /etc/mail/aliases file so that mail sent to this new alias goes to the script:
    aliasname: "|/path/to/script.pl"

    The script will receive the entire mail, then respond to the sender, with the short message of your choosing, bcc'ing you if so desired, but also save the entire email into a disk textfile, using a unique name (in your case you would change that filename to incorporate the sender's name - should be easy using a variable already there from the headers capture).

    Now that you've preserved the email on disk (and also in an array in the script) you can do any number of things like separate the body out and send it onward to .txt, .html, or .XLS. IMO that is a separate task and should be treated as such unless you're doing a quick one off.

    Sorry I would have altered the script more to your needs but really there are just too many variables not knowing your exact circumstances.

    But hopefully this demonstrates that it is easy to capture the email and do what you want with it. The hardest part is going to be to get the users to send the email in the first place. Here's the code:

    Forgetting any Perl code momentarily, I'd like to mention that if you're trying to do any change control or tracking, this sure is a hard way to do it. Even if your team sends the requisite email, they surely won't ever be including the level of detail you're envisioning, or they might not even send the email until significantly later. One thing though, if you send an email, it is nice to get an automated response confirming receipt and processing.
Re: using Email to Perl to append to a changelog.txt, html, or .xls?
by bsdz (Friar) on Nov 24, 2004 at 12:14 UTC
    A suggestion for a MS windows solution. I haven't implemented this yet but I do intend to in the near future (I have only done preliminary research so far). One can install Microsoft's IIS SMTP service provided you have W2K (or above). Then you can use the "Managed Sinks" feature that waits for a TriggerServerEvent. I admit it may need a little XS code if there isn't an ActiveX interface and it's not as simple as an /etc/aliases.