deadkarma has asked for the wisdom of the Perl Monks concerning the following question:

I am thinking about writing a perl script to act as a simple sort of MTA and store/process all emails into a MySQL database.
The fields will be something like:

Date
From
To
Subject
Body
Attachment(s)

What I'm concerned about is if a perl script can perform well under high loads, and also if it's possible to create some type of 'watcher' process to monitor the perl daemon and restart it if it dies.

Or alternatively, (and maybe off-topic) is there something like this already written that is stable enough to use? The closest I've found was dbmail which doesn't seem quite ready for primetime just yet.

Replies are listed 'Best First'.
Re: perl scripts as daemons
by busunsl (Vicar) on Dec 07, 2001 at 01:47 UTC
    I think best you can do is buy 'network programming' by Lincoln Stein. Read it cover to cover and then start writing your daemon.

    It covers everything from basic network programming to multiplexing, threading and forking for high loads.

    It also covers 'daemonizing', logging and other stuff you probably want to do.

    In short: A perl program can perform well under high loads, it can even perform better than stuff written in other languages.
    I'm longing to hear Marc Lehmans talk at German Perl Workshop, why he wrote his own web server for very high loads that couldn't be served by apache and thttp.

[reply]
      I picked up a copy at lunch since my next project involves a networked perl daemon. I'm only a couple hours into the book, but am quite impressed so far.

      On a side note, merlyn benchmarked a trivial http-redirector written in perl at nearly 10x the throughput of apache. See his comments in this thread for details.

      -Blake

[reply]
      Absolutely true. This book walks through this exact topic in detail and refines it to the point where you just use the modules you have written for daemons. Not only that but you will understand the modules quite well.

      "Nothing is sure but death and taxes" I say combine the two and its death to all taxes!
[reply]
Re: perl scripts as daemons
by IlyaM (Parson) on Dec 07, 2001 at 01:42 UTC
[reply]
      I forgot to add that I would not write MTA which listens to SMTP port. I don't trust myself to do it in proper, secure, DOS resistant way. Instead I would plug script written with usage of pperl into some "real" MTA like postfix or sendmail. Such script still is subject for security compromises but their changes are lower since your script would be simplier (thus lower probability of security holes) and it would handle only portion of possible attacks.

      --
      Ilya Martynov (http://martynov.org/)

[reply]
        I would whole-heartedly concur with this approach - In addition to minimising the work of the script author to write a robust and ruggedly network process, this approach would allow the program to be called 'as-needed' rather than as a daemon process.

        Written as a plug-in to a more thoroughly tested and widely used MTA, the script could be called through an /etc/mail/aliases type of pipe such as:

        test:          "|/usr/local/bin/myscript"

        [download]

        With the corresponding script able to process mail received via reading STDIN. eg.

        #!/usr/bin/perl -T

use strict;

undef $/;
my $message = <>;
my ($headers, $body) = split(/\n\n/, $message, 2);
$headers =~ s/\n\s+/ /g;

foreach (split(/\n/, $headers)) {
    # ... process headers ...
}

        [download]

        An excellent suggestion IlyaM++

         

        perl -e 's&&rob@cowsnet.com.au&&&split/[@.]/&&s&.com.&_&&&print'

[reply]
[d/l]
[select]
Re: perl scripts as daemons
by sparkyichi (Deacon) on Dec 07, 2001 at 02:03 UTC
    I have written some Perl scripts that manage other Perl scripts. I used Win32::Process module to start and manage the threads. 
    use Win32::Process;
Win32::Process::Create($Process1, 
            "/myprog.exe",
            "myprog",
            0,
            DETACHED_PROCESS,
            "/") || die "Fatality: $!\n";
}

    [download]


    I would then use log files and marker files to make sure that the processes are doing what they are intended to do. Your manager script can then monitor the processes by checking those files. Win32::Process module also has tools you can use to monitor the performance and completion of the threads that it started. You should check out the module documentation and see if it fits your needs. There are other modules that do the same type of think, but I have had the best luck with this one.

    Sparky
[reply]
[d/l]
Re: perl scripts as daemons
by perrin (Chancellor) on Dec 07, 2001 at 01:52 UTC
    You might try using PerlEx, or Mail::Audit for this, rather than a separate daemon.
[reply]
Re: perl scripts as daemons
by eformat (Initiate) on Dec 07, 2001 at 20:23 UTC
    Hi,
    I don't know if this is the sort of reply you're after, but out of interest here is a small daemonizing subroutine you could use:
    use POSIX qw/setsid/;
sub daemonise
{
    my ($self) = @_;
    chdir('/') or die "Can't chdir to /: $!";
    open STDIN, "</dev/null";
    defined(my $pid = fork) or die "Can't fork: $!";
    exit if $pid;
    setsid();
    $SIG{HUP} = \&hup_handler($self); # you need to define this
   # we store our pid's for shutdown
    if ( $self->{pidfile} ) {
     open PIDFILE, "> " . $self->{pidfile};
     print PIDFILE "$$\n";
     close PIDFILE;
    }
}

    [download]
    Mike
[reply]
[d/l]

Back to Seekers of Perl Wisdom