Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Anti-Virus Signature Updates

by linebacker (Scribe)
on Jul 08, 2002 at 19:35 UTC ( #180289=perlquestion: print w/replies, xml ) Need Help??

linebacker has asked for the wisdom of the Perl Monks concerning the following question:

This is a continuation of the same program I have been struggling with. I think it is about 95% there, but the last part still eludes me.

The goal is to update AV signatures if you have not 'seen' them before. I use a flat file containing update signature file names. I want to iterate through that file and compare it to the files found on the ftp server. If I have never seen the file (ending in x86.exe) then 'get' it.

I can't seem to get the arrays to compare with each other. Pointers are appreciated. Thanks to all those who already helped, I will pay it forward.

1 #!/usr/bin/perl -w 2 #use strict; 3 use Net::FTP; 4 my $host = 'www.ftp.com'; 5 my $user = 'anonymous'; 6 my $pass = 'updatepuller@ftp.com'; 7 my $remote_dir = '/pub/antivirus/NAV/signatures'; 8 my $destination_dir= "current-sig"; 9 $ftp = Net::FTP->new($host, Debug => 1); 10 $ftp->login($user,$pass); 11 my @listing = $ftp->ls("$remote_dir"); 12 $lflag = 0; 13 if(-e "./lastupdate.txt") { 14 open(LASTUPDATE,"./lastupdate.txt"); 15 @lupd = (<LASTUPDATE>); 16 close(LASTUPDATE); 17 $lflag = 1; 18 } else { 19 $lupd = ""; 20 } 21 foreach $line (@listing) { 22 chomp($line); 23 $lline = lc($line); 24 if ($lline =~ "x86.exe") { 25 foreach $haveit (@lupd) { 26 chomp($haveit); 27 if (($lflag == 0) || ($haveit ne $line)) { 28 $ftp->type("I"); 29 $ftp->get("$line"); 30 } 31 } 32 $newupdate .= "$line\n"; 33 } 34 } 35 $ftp->quit; 36 #@lupd = split(/\n/,$newupdate); 37 @nupd = ($newupdate); 38 open(NEWUPDATE,">>./lastupdate.txt"); 39 print NEWUPDATE @nupd; 40 close(NEWUPDATE);
Contents of lastupdate.txt after a run
/pub/antivirus/NAV/signatures/0708x86.exe/pub/antivirus/NAV/signatures/0703x86.exe

Replies are listed 'Best First'.
Re: Anti-Virus Signature Updates
by rincew (Novice) on Jul 08, 2002 at 21:08 UTC
    Finding the intersection of two arrays is described in perlfaq 4. The short version: use a hash. So incorporating fruitures notes from above, here is my (untested, since I do not have access to a suitable ftp server) try:
    #!/usr/bin/perl -w use strict; use Net::FTP; my $host = 'www.ftp.com'; my $user = 'anonymous'; my $pass = 'updatepuller@ftp.com'; my $remote_dir = '/pub/antivirus/NAV/signatures'; my $destination_dir = 'current-sig';
    Here I read the contents of lastupdate.txt into the array @lupd. If the file does not exist, @lupd will be left empty (no need for a special flag).
    my @lupd = (); if(-e "./lastupdate.txt") { open(LASTUPDATE,"./lastupdate.txt") or die "Error while opening lastupdate.txt ($!)"; chomp( @lupd = <LASTUPDATE> ); close(LASTUPDATE); }
    The following construct is called a hash slice. It is described in more detail in this page
    my %haveit; @haveit{ @lupd } = (1) x @lupd; my $ftp = Net::FTP->new($host, Debug => 1); $ftp->login($user,$pass); $ftp->type("I"); my @listing = grep /x86\.exe$/i, $ftp->ls($remote_dir); my @newupdate; foreach my $file (@listing) {
    The hash slice above produced a hash which maps every known filename to a true value. So if the file has already been downloaded, we can simply skip the download. When the download is unsuccessful, the file is skipped as well.
    next if $haveit{ $file }; $ftp->get($file) or next; push @newupdate, $file; } $ftp->quit; open(NEWUPDATE,">>./lastupdate.txt") or die "Couldn't open lastupdate.txt for appending ($!)"; print NEWUPDATE map "$_\n", @newupdate; close(NEWUPDATE) or die "Closing lastupdate.txt failed ($!)";
    You might also want to think about locking if there is any chance that two instances of this program run at the same time.
      This is great, and really helpful with the hash slice info.

      Unfortunately, your version of the script does not appear to compare the file (lastupdate.txt) with the listing on the ftp server. In other words, the program downloads all the files ending in x86.exe and appends the names of the files to the lastupdate.txt file. So...the net gain so far is nada.

      I really appreciate your time. Thanks very much!

      -Mike
        This is strange - I just tested it once again using dummy data and it works for me. Note that the comparison used to determine whether a given file has already been downloaded is (case sensitive) string equality, so preparing lastupdate.txt by hand is likely to fail - it shouldn't ignore the files it added itself though. If this is a problem, you might want to add some kind of canonicalisation step like
        chomp( @lupd = map lc, <LASTUPDATE> ); ... next if $haveit{ lc $file }
        to ignore case.

        In any case, I'd recommend you to step it through with the debugger, checking the exact contents of the data structures at any time. This should give enough information to get things working.

Re: Anti-Virus Signature Updates
by fruiture (Curate) on Jul 08, 2002 at 20:34 UTC

    Some pointers:

    • use strict; it helps!
    • `perldoc -q 'quoting.*vars'` (!)
    • $lupd is nonsense
    • @nupd is nonsense
    • $lline is nonsense if you match case-insensitive , see perlop
    • And perlre knows your mistake. '$line =~ /x86\.exe/'
    • Always, (really always), check the return value of open()! Check close() when you have written to a file.
    HTH

    --
    http://fruiture.de
Re: Anti-Virus Signature Updates
by Anonymous Monk on Jul 09, 2002 at 00:14 UTC
    I think this is what you are looking for. I didnt get the chance to test it (sorry, Im way too tired and my eyes are crawling shut). Please share your code with the rest of us when you get it working! btw, You really should add error checking for the ftp transactions!
    #!/usr/bin/perl -w use Strict; use Net::FTP; my $host = 'www.ftp.com'; my $user = 'anonymous'; my $pass = 'updatepuller@ftp.com'; my $remote_dir = '/pub/antivirus/NAV/signatures'; my $destination_dir = 'current-sig'; my $UPDATE_TRACKER = './lastupdate.txt'; # You should add error checking to make sure you actually make # a connection to the server my $ftp = Net::FTP->new($host, Debug => 1); $ftp->login($user,$pass); my @listing = $ftp->ls($remote_dir); foreach (@listing) { $_=~s/\r|\n//g; if ( /x86\.exe/gi ) { my ($response,$errmsg) = do_we_already_have_it($_); if ( $response != 1 ) { # Either we have it or we have an error if ($response == -1) { # We have an error print "Error - $errmsg\n"; quit_now(); }; } else { # We dont have it - grab it ($response,$errmsg) = grab_it($_); if ( $response == -1 ) { # We have an error print "Error - $errmsg\n"; quit_now(); }; }; }; }; quit_now(); sub quit_now { $ftp->quit; exit; }; sub grab_it { # returns 1 if we get the file # returns -1 if we dont get the file # in its current state it will only return -1 if we cant update # the tracker db my ($remote_file) = @_; $ftp->type("I"); $ftp->get($remote_file); # You should add error checking here to make sure you actually # get the file !! ( I would but im too tired ) if ( open(NEWUPDATE,">>$UPDATE_TRACKER") ) { print NEWUPDATE "$remote_file\n"; close(NEWUPDATE); return(1); } else { # Something went wrong return -1 and the error msg return(-1,$!); }; }; sub do_we_already_have_it { # returns 1 if we already have it # returns 0 if we dont have it # returns -1,error msg if something goes wrong (like permissions) my ($check_file) = @_; $check_file=~s/\r|\n//g; if ( open(LASTUPDATE,$UPDATE_TRACKER) ) { while(<LASTUPDATE>) { $_=~s/\r|\n//g; if ($_ eq $check_file) { close(LASTUPDATE); return(1); }; }; close(LASTUPDATE); return(0); } else { # something went wrong - check if the file exists # If the file exists then we have some sort of other error # that we should probably report back to the client # otherwise we simply dont have it yet ! if(-e $UPDATE_TRACKER) { return(-1,$!) }; return(0); }; };
      First occurance of
      if ( $response != 1 ) {
      should have been
      if ( $response != 0 ) {
      I blame it on the heat and lack of sleep!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://180289]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2019-10-19 23:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?