Triggering Perl Scripts

[Perl Newby]

I have a sports web site where I am going to post the box scores of pro sports games. I have already written script that will parse the data and put it into an HTML format. My problem is that I want to be able to set up some form of a trigger that will automatically fire when I receive a new file. This is the scenario I want to use: 1.) I receive the box score via FTP in a text file format, ex. mlb_boxscore.txt 2.) As soon as the file is FTP'd I would like a trigger to set off the code I have written that parses and formats the data 3) After the code runs, the web site is updated. Any suggestions on how I would write a trigger that could accomplish this?

[stephen]

Probably the best you can do is have a CRON job that runs every few minutes that checks for the existence of a new file. If you do this, I'd suggest that you have the job
1. Spot the existence of the new file.
2. Wait for a bit, then see if the file's size has changed.
3. Only parse and format the data when the file size has stabilized.
Otherwise, you're in danger of parsing the file before it's fully uploaded.

stephen

[AgentM]

A couple of options:
• set up a cron job to scan the directory for files or new files and fire off the perl parsing script. This is the most wasteful option because it polls, but is most likely to be supported on your system.
• Install tripwire. You may be subject to licensing fees, but this will most definitely get the job done.
• Hack/modify/add a hook to the FTP server to fire off the perl script whenever a file is put into a specific directory. If the FTP server is used for nothing but this uploading thang (sic), then use some CPAN modules to throw together your own FTP server or modify a Perl FTP server that you find online. This is the least secure of your options- that is, if you're not pedantic about your UN*X programming.
It looks like Option #2 is the winner!


AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

[arturo]

Here's another option, akin to AgentM's suggestion that you might write a Perl FTP server and handle the whole thing through that. Handle the uploads via CGI scripts, and have the results submitted via LWP (or whatever makes sense -- as long as you submit the files via HTTP and have a CGI handle them).

Once the upload is complete, the CGI can check the file and update the site.

HTH

[premchai21]

Have the file e-mailed to a special address that pipes the mail to your script.

[arhuman]

To my mind, the CRON solution is the better one,
But If you can't (or dont want to) use it
(only root's allowed cron, Windoze boxes without cron, not used to CRON syntax...)
You can simply write a perl script that'll check for the existence of a file, process it (and move/delete it)
if it's present, and then sleep 5 mins (in an infinite loop).

Quick and dirty but simple...<br

"Only Bad Coders Badly Code In Perl" (OBC2IP)

[suaveant]

One way is to have the script on a cron, and every five minutes or whatever check the directory, if the file is there (and complete) then you can move it to a temporary spot and parse it. Of course, then you need a way to make sure the file is finished uploading.
                - Ant

[diarmuid]

Put your perl script in a crontab job to check the ftp site and run it every minute or whatever. Get it to check the modification date and only update the file if it has changed.

I tried using a mkfifo but that was more messing than needed

[asiufy]

Instead of using the CRON, you can have your own parsing/formatting script poll for the new files in the specific directory. Something like this:


while (1<2) {
     opendir (DIR, $dir) or die "cannot opendir $dir";
     my @only_files = grep {-f "$dir/$_"} readdir(DIR);
     readdir DIR;
     closedir DIR;
     foreach $file (@only_files) {
        &process_file ($file);
     }
sleep(10);
}


This will poll $dir every 10 seconds (set in the sleep line) for any new files.

If your files are big, you might want to add some of the verification mentioned earlier, regarding the file size, etc.