http://www.perlmonks.org?node_id=180325


in reply to Anti-Virus Signature Updates

Finding the intersection of two arrays is described in perlfaq 4. The short version: use a hash. So incorporating fruitures notes from above, here is my (untested, since I do not have access to a suitable ftp server) try:
#!/usr/bin/perl -w use strict; use Net::FTP; my $host = 'www.ftp.com'; my $user = 'anonymous'; my $pass = 'updatepuller@ftp.com'; my $remote_dir = '/pub/antivirus/NAV/signatures'; my $destination_dir = 'current-sig';
Here I read the contents of lastupdate.txt into the array @lupd. If the file does not exist, @lupd will be left empty (no need for a special flag).
my @lupd = (); if(-e "./lastupdate.txt") { open(LASTUPDATE,"./lastupdate.txt") or die "Error while opening lastupdate.txt ($!)"; chomp( @lupd = <LASTUPDATE> ); close(LASTUPDATE); }
The following construct is called a hash slice. It is described in more detail in this page
my %haveit; @haveit{ @lupd } = (1) x @lupd; my $ftp = Net::FTP->new($host, Debug => 1); $ftp->login($user,$pass); $ftp->type("I"); my @listing = grep /x86\.exe$/i, $ftp->ls($remote_dir); my @newupdate; foreach my $file (@listing) {
The hash slice above produced a hash which maps every known filename to a true value. So if the file has already been downloaded, we can simply skip the download. When the download is unsuccessful, the file is skipped as well.
next if $haveit{ $file }; $ftp->get($file) or next; push @newupdate, $file; } $ftp->quit; open(NEWUPDATE,">>./lastupdate.txt") or die "Couldn't open lastupdate.txt for appending ($!)"; print NEWUPDATE map "$_\n", @newupdate; close(NEWUPDATE) or die "Closing lastupdate.txt failed ($!)";
You might also want to think about locking if there is any chance that two instances of this program run at the same time.

Replies are listed 'Best First'.
Re: Re: Anti-Virus Signature Updates
by linebacker (Scribe) on Jul 08, 2002 at 21:41 UTC
    This is great, and really helpful with the hash slice info.

    Unfortunately, your version of the script does not appear to compare the file (lastupdate.txt) with the listing on the ftp server. In other words, the program downloads all the files ending in x86.exe and appends the names of the files to the lastupdate.txt file. So...the net gain so far is nada.

    I really appreciate your time. Thanks very much!

    -Mike
      This is strange - I just tested it once again using dummy data and it works for me. Note that the comparison used to determine whether a given file has already been downloaded is (case sensitive) string equality, so preparing lastupdate.txt by hand is likely to fail - it shouldn't ignore the files it added itself though. If this is a problem, you might want to add some kind of canonicalisation step like
      chomp( @lupd = map lc, <LASTUPDATE> ); ... next if $haveit{ lc $file }
      to ignore case.

      In any case, I'd recommend you to step it through with the debugger, checking the exact contents of the data structures at any time. This should give enough information to get things working.

Re: Re: Anti-Virus Signature Updates
by linebacker (Scribe) on Jul 10, 2002 at 14:22 UTC