Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

RFC - PERL SCRIPT TO SSH MULTIPLE SERVER NODES, CHECKS OF SSL CERTIFICATES EXPIRY AND EMAIL TO ROOT

by Zeeshan (Initiate)
on May 08, 2012 at 21:21 UTC ( [id://969539]=perlmeditation: print w/replies, xml ) Need Help??

A set of nodes are managed using ssh only (call them "deva", "devb", "devc", "devd"). Each node runs a service that supports certificates, and the certificate expiration date must be checked. Each node can have multiple certificates, and they are all returned by running the shell command "get x509 cert" on the node itself. The command outputs one cert per line, with the following format:

<CERT_NAME> hash<CERT_HASH> exp<DATE:YYYY-MM-DD> <TIME:hh:mm:ss>

Create a script that will ssh to each node and check for certificate expiration. The script should also send e-mail five days before expiration, with node name and certificate details, and another e-mail when cert expires in less than 2 days. Please notice, that it should be easy to add another node to the list. Also, another warning mail should be sent if the node is unavailable. A summary log after the work is done, would be a nice add-on.

#!/usr/bin/perl use Net::SSH::Expect; use Time::localtime; use Date::Parse; use Term::ReadKey; #Days to check validity for my $minimumExpDays = 5; my $minimumExpDays2 = 2; #Email of root my $rootEmail = "root\@localhost"; my $emailMsg = "Expiration of Certificates"; #log file location my $logfile = "/var/log/script-log.txt"; #change it to your desired lo +cation # Define servers array - List as many servers as you want, use the ful +l FQDN or IP address @servers = ("deva, “devb", “devc", “devd"); #Name of the certificates file you want to moinitor. @certfiles = ("cert.pem","cert2.pem", “cert3.pem”, “cert4.pem”); foreach (@servers) { print " $_ \n"; } print "Please enter your username for above systems? \n"; $user = <>; #A sudo user can also be hardcoded for a Cron job. chomp $user; print "\nPlease enter your password for the above systems? \n"; ReadMode 2; #Stop echoing to screen $pass = <>; chomp $pass; ReadMode 0; #Return tty to default; $to="5"; # Loop through the servers array and connect to one server (box) at a +time. foreach $box (@servers) { # Print to screen what server you are connecting to print "Connecting to $box... \n"; $ssh = Net::SSH::Expect->new ( host => "$box", password => "$pass", user => "$user", raw_pty => 1 ); undef $login_output; eval { $login_output = $ssh->login(15); }; # If login output is empty try again while ($login_output eq "") { sleep(2); $login_output = $ssh->login(15); } if ($login_output =~ m/Last login/) { print "Login Successful... \n\n"; } else { print "Login has failed! - Please check your username/password and ca +ps lock. \n\n"; next; } # RUN COMMANDS AS USER print "Running command....\n"; foreach $certfile (@certfiles) { $ssh->send("openssl x509 -in /etc/pki/tls/$certfile -noout -subject - +hash -enddate | awk \'{print \$1, \$2, \$3, \$4, \$5,\$6, \$7}\'"); while ( defined ($output = $ssh->read_line($to)) ) { # Send output of command to output array for printing when script is +complete push (@outputs, "$box: $output"); } }#while loop }#end of foreach certfile my $tmp = 3; foreach(@certfiles){ # print $_ ."\n"; my $enddate = $outputs[$tmp]; my $tmpdate; if($enddate =~ m/notAfter=(.*)/){ $tmpdate = $1; } my $end = str2time($tmpdate); my $daysleft = ($end -time())/86400; print "Days left := $daysleft .\n"; $tmp = $tmp + 4; if(($daysleft < $minimumExpDays2) || ($daysleft < $minimumExpDays)) { $emailMsg = "Your certificates will be expired in " .int($daysleft)." + days\n Please check your log file at $logfile \n"; print $emailMsg; system("echo \"$emailMsg\" | mail -s \"Certificate Expiration Warning +\" $rootEmail"); }#end if }#end offoreach print "\n ********Printing Report******** \n"; #Opening file stream, make sure you have the appropritate permission o +n the file system. open FILE, ">>$logfile" or die $!; foreach (@outputs) { #Printing a summary on console. print $_ . "\n"; #Writing into the log file. print FILE $_ . "\n"; }#end of foreach close FILE; #### END SCRIPT

Replies are listed 'Best First'.
Re: RFC - PERL SCRIPT TO SSH MULTIPLE SERVER NODES, CHECKS OF SSL CERTIFICATES EXPIRY AND EMAIL TO ROOT
by choroba (Cardinal) on May 09, 2012 at 07:21 UTC
    Just a tip on formating: Using double line spacing makes the code longer, but not more readable. On the contrary, you can skim smaller portions of the code in one page, which means it might be harder to spot a bug. On the other hand, using more than one space for indentation (some people use as much as four (4) spaces!) can make the code more easy to read and lets you delete all the #end of foreach comments.
Re: RFC - PERL SCRIPT TO SSH MULTIPLE SERVER NODES, CHECKS OF SSL CERTIFICATES EXPIRY AND EMAIL TO ROOT
by salva (Canon) on May 09, 2012 at 08:04 UTC
    for my $box (@servers) { my $ssh = Net::OpenSSH->new($box, user => $user, password => $password); if ($ssh->error) { warn "unable to connect to $box"; next; } for my $certfile (@certfiles) { my $output = $ssh->capture('openssl', 'x509', '-in', "/etc/pki/tls/$certfile", '-noout', '-subject', '-hash', '-enddate'); unless ($output =~ /notAfter=(\S+)/) { warn "expiration data not found in $box:$certfile" next; } my $notAfter = $1; ... } }
Re: RFC - PERL SCRIPT TO SSH MULTIPLE SERVER NODES, CHECKS OF SSL CERTIFICATES EXPIRY AND EMAIL TO ROOT
by Tanktalus (Canon) on May 28, 2012 at 23:10 UTC

    In no particular order, some additional comments:

    • Set up passwordless ssh. Or at least set up ssh keys. With truly passwordless (ssh keys where the private key doesn't have a password), you have to guard the private key, but you can set this up as a cron job with no user input. That means you don't have to remember to run it, which makes it incredibly more useful.
    • If you have a lot of systems to check, consider threading/job queue. Personally, I use AnyEvent + AnyEvent::Util::run_cmd to run a dozen or two ssh's in parallel (no job queue - this isn't enough ssh's to matter, but if you have 40+ systems to check, you may want a job queue). If you take the above advice and then put it in a cron job, this may not matter as you can schedule it for a time when it can take a long time and when the network is otherwise not in significant use (e.g., 4AM). If all the systems are on a local Gbps network, this will also not matter. But if some systems are located in other physical locations, possibly with lower bandwidth and/or higher latency, being able to parallelise them can really speed things up. Again, speaking of my personal usage of this pattern, all the machines I ssh to are on a private 10+Gbps network, but the commands can take a long time to run, so I've used AE to parallelise them. In the future, I will likely switch to Coro + Coro::Channel/Coro::PrioChannel for job queuing, though that still leaves AE handling the parallel ssh calls.

      This really depends on what you're doing and how it affects you. If you run it at the command line, as you must right now, you may want to minimise the amount of time it takes. If it's in a cron job in the middle of the night, it may matter less. If it takes 15 seconds, you may not care. If it takes 30 minutes for all the nodes, you may care. All I'm really suggesting is considering it. It's not actually that hard to do with the right modules.

    • Send a single message with a list of servers (and their expirations). Rather than flood an inbox, put them all together. Also will generally make it easier to deal with all nodes at the same time. This will require that you save the server and expiration date in an array, then, when you're finished looking at all the servers, you check if you have anything in your array. If so, send the email, otherwise you're done.
    • Consider sending the logfile as an attachment. Email::MIME might help here. Rather than having to look around for the log file, just send it. This works even better if you don't send to root, but send to an actual corporate email address where the sysadmin(s) can read it in their normal email program (Thunderbird, right?) with everything else.
    Just random thoughts, in addition to others already given.

Re: RFC - PERL SCRIPT TO SSH MULTIPLE SERVER NODES, CHECKS OF SSL CERTIFICATES EXPIRY AND EMAIL TO ROOT
by naikonta (Curate) on May 19, 2012 at 06:04 UTC

      I'd say CUFP. It's our catch-all code section.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://969539]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2024-03-19 11:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found