http://www.perlmonks.org?node_id=1013245

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have script that runs as a damon that writes all data from a tcp socket to a file. I need to close and reopen a new file every day without restarting the script. Here is what I have so far and it works but it does not close and reopen a new file at the time listed.
#!/usr/bin/perl -w use strict; use IO::Socket; use MIME::Lite; use Time::Out qw(timeout); $SIG{PIPE} = "IGNORE"; $| = 1; my $nb_secs = 10; my $buf = ""; my $sock = new IO::Socket::INET (PeerAddr => '192.168.173.9', PeerPort => 4002, Proto => 'tcp', Type => SOCK_STREAM, ); die "cannot open socket" unless ($sock); my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(t +ime); my $hrmin = sprintf ("%02d%02d",$hour,$min); my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$mday,$ +hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open FILE, ">$filename" || die("Couldn't open file"); while (my $line = <$sock>) { # Close existing file and reopen a new one at midnight. if($hrmin == 1114) { sleep 1; close FILE; my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$md +ay,$hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open FILE, ">$filename" || die("Couldn't open file"); } if ($line =~ m/^(ALARM: )/) { ....... }; #Print current line to open file. print FILE $line; } sleep 10; # allow sometime to exit close FILE;

Replies are listed 'Best First'.
Re: Close a file and reopen a new one at midnight
by MidLifeXis (Monsignor) on Jan 14, 2013 at 17:59 UTC

    Is there sufficient log activity for the $hrmin check to have a large enough resolution, or is it possibly skipping over your check time?

    If the check is only for the date, why not do a comparison to see if the last date matches the current date?

    I also note that you are never recalculating the datetime, so $hrmin and $ymd never change during the lifetime of your while (<$sock>) loop.

    Update: Be careful if this restarts. Instead of appending to the current date's log file, you will clobber it and truncate it at the beginning with your open statement. Consider using '>>' instead of '>'.

    --MidLifeXis

Re: Close a file and reopen a new one at midnight
by flexvault (Monsignor) on Jan 14, 2013 at 18:01 UTC

    AM,

    Two things after a glace at your code;

    • $hrmin is a string, and you're testing as a number.
      if ( $hrmin eq '1114' ) {
    • '1114' is before noon, '2359' is before midnight. You also should be concerned about seconds. A lot can happen in 60 seconds.

    You may want to buffer the end of the day, which may give you a more accurate end-of-day and start-of-day.

    Good Luck!

    "Well done is better than well said." - Benjamin Franklin

      This is what I changed it to. The time is not 0000 for testing. It still does not work.
      #!/usr/bin/perl -w use strict; use IO::Socket; use MIME::Lite; use Time::Out qw(timeout); $SIG{PIPE} = "IGNORE"; $| = 1; my $nb_secs = 10; my $buf = ""; my $file; my $sock = new IO::Socket::INET (PeerAddr => '192.168.173.9', PeerPort => 4002, Proto => 'tcp', Type => SOCK_STREAM, ); die "cannot open socket" unless ($sock); my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(t +ime); my $hrmin = sprintf ("%02d%02d",$hour,$min); my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$mday,$ +hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open $file, ">$filename" || die("Couldn't open $file"); while (my $line = <$sock>) { # Close existing $file and reopen a new one at time listed. if($hrmin eq 1228) { sleep 1; close $file; my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localti +me(time); my $hrmin = sprintf ("%02d%02d",$hour,$min); my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$md +ay,$hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open $file, ">$filename" || die("Couldn't open $file"); } if ($line =~ m/^(ALARM: )/) {

        • You now should have updated the $hrmin and $ymd variables.
        • You still have a problem if no data is received at '1228'.
        • You still have a problem overwriting a file if this gets restarted.
        • There are also options out there that handle this for you (one that is log4j-like, one that is tied, listed below by clueless_newbie, and probably others). See FileRotate for some options.

        --MidLifeXis

      I believe that this time it was put to test.
      However, if for one minute data are not received, this condition is not met. if($hrmin == 1114)
        update: MidLifeXis' explanation got thru my thick skull... and seems plausible. Apologies for wasting the effort of so many electrons and Monks but I do think the last graf still stands... and, FTR, a comment or narrative explanation of "1144" by the OP would have been, IMO, highly desireable.

        Monk 0day: Would you kindly explain your statements? I don't see that either has any relevance.

        "1144" remains a representation of an hour and minute that are not even close to midnight but it's being used as the code for the question "Is it almost midnite, and time to close the old log file and open a new one?"

        and NJBTW, any update of $ymd " after sleep 1 at 11:44AM is not going to be any different than the old one, so while there will be a new file created, it will have the same name as (and overwrite) the previous one. The test MUST occur much closer to midnight and with an appropriate sleep to ensure the calendar turns to another day.

Re: Close a file and reopen a new one at midnight
by clueless newbie (Curate) on Jan 14, 2013 at 18:47 UTC

    Being clueless, I was wondering why there's no file tie module that performs this task ...

    Here you go ... Tie::Handle::FileWriteRotate

    use Tie::Handle::FileWriteRotate; tie *FH, 'Tie::Handle::FileWriteRotate', dir=>'/some/dir', prefix=>'myapp', print FH "Logging a line\n"; print FH "Logging another line\n";
      Here is the program without email info.
      #!/usr/bin/perl -w use strict; use IO::Socket; use MIME::Lite; use Time::Out qw(timeout); $SIG{PIPE} = "IGNORE"; $| = 1; my $nb_secs = 10; my $buf = ""; my $file; my $sock = new IO::Socket::INET (PeerAddr => '192.168.173.9', PeerPort => 4002, Proto => 'tcp', Type => SOCK_STREAM, ); die "cannot open socket" unless ($sock); my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(t +ime); my $hrmin = sprintf ("%02d%02d",$hour,$min); my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$mday,$ +hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open $file, ">$filename" || die("Couldn't open $file"); Email startup message my $msg = MIME::Lite->new( From => To => Subject => Type =>'TEXT', Data => 'Started.' , ); $msg->send(); while (my $line = <$sock>) { # Close existing $file and reopen a new one at midnight. if($hrmin eq 0000) { sleep 1; close $file; my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localti +me(time); my $hrmin = sprintf ("%02d%02d",$hour,$min); my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$md +ay,$hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open $file, ">$filename" || die("Couldn't open $file"); } if ($line =~ m/^(ALARM: )/) { timeout $nb_secs => sub { my $count = 0; until ($count == 5 ) #Read the next xx lines { $buf .= $sock->getline(); $sock->flush(); $count++; } }; if ($@){ my $msg = MIME::Lite->new( From => To => Cc => Subject =>'ALARM message from house', Type =>'TEXT', Data =>$buf , ); $msg->send(); #print $line; #print $buf; $line = ""; $buf = ""; } } if ($line =~ m/^(INFO: )/) { timeout $nb_secs => sub { my $count = 0; until ($count == 5 ) #Read the next xx lines { $buf .= $sock->getline(); $sock->flush(); $count++; } }; if ($@){ my $msg = MIME::Lite->new( From => To => Cc => Subject =>'INFO message from house', Type =>'TEXT', Data =>$buf , ); $msg->send(); #print $line; #print $buf; $line = ""; $buf = ""; } } if ($line =~ m/^HISTORY: /) { timeout $nb_secs => sub { my $count = 0; until ($count == 40 ) #Read the next xx lines { $buf .= $sock->getline(); $count++; } }; if ($@){ my $msg = MIME::Lite->new( From => To => Subject =>'History Page', Type =>'TEXT', Data =>$buf , ); $msg->send(); #print $line; #print $buf; $line = ""; $buf = ""; } } if ($line =~ m/^ALARM-JONN: /) { timeout $nb_secs => sub { my $count = 0; until ($count == 40 ) #Read the next xx lines { $buf .= $sock->getline(); $count++; } }; if ($@){ my $msg = MIME::Lite->new( From => To => Subject => Type =>'TEXT', Data =>$buf , ); $msg->send(); #print $line; #print $buf; $line = ""; $buf = ""; } } $SIG{TERM} = sub { my $msg = MIME::Lite->new( From => To => Subject => Type =>'TEXT', Data => 'Terminated.' , ); $msg->send(); die "\nKilling program.\n"; close $sock or die "close: $!"; }; $SIG{INT} = sub { my $msg = MIME::Lite->new( From => To => Subject => Type =>'TEXT', Data => 'Terminated.' , ); $msg->send();# die "\nKilling program.\n"; close $sock or die "close: $!"; }; #Print current line to open $file. print $file $line; } sleep 10; # allow sometime to exit close $file;
        Ok here is how I got it to work. I used a signal handler to close and reopen the file.
        while (1) { $SIG{USR1} = sub { close $file; my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localti +me(time); my $hrmin = sprintf ("%02d%02d",$hour,$min); my $ymd = sprintf("%04d%02d%02d%02d%02d%02d",$year+1900,$mon+1,$md +ay,$hour,$min,$sec); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open $file, ">$filename" || die("Couldn't open $file"); }; while (my $line = <$sock>) { if ($line =~ m/^(ALARM: )/) {
        Now I can just use the log rotate system to send a killall -USR1 to start a new file.

        "compiled" but not tested!

        Set the date of last write to "000000" and close and append to a new file when the date changes ie date of last write != today. Thus if nothing happens for a week no logs are created for those dead days.

        #!/usr/bin/perl -w use strict; use IO::Socket; use MIME::Lite; my $nb_secs = 10; my $buf = ""; my $file; my $sock = new IO::Socket::INET( PeerAddr => '192.168.173.9', PeerPort => 4002, Proto => 'tcp', Type => SOCK_STREAM, ); die "cannot open socket" unless ($sock); my ($lastwrite,$today)="000000"; while ( my $line = <$sock> ) { if ( $lastwrite ne ($today=sprintf("%02d%02d%02d",(localtime())[5, +4,3]))) { # Close existing $file and reopen a new one at when the date c +hanges close $file; my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isd +st ) = localtime(time); my $ymd = sprintf( "%04d%02d%02d%02d%02d%02d", $year + 1900, $ +mon + 1, $mday, $hour, $min, $sec ); my $filename = "/tmp/$ymd.txt"; print $filename, "\n"; open $file, ">>","$filename" || die("Couldn't open $file"); $lastwrite=$today; } #Print current line to open $file. print $file $line; } close $file;
Re: Close a file and reopen a new one at midnight
by aitap (Curate) on Jan 14, 2013 at 18:44 UTC
    It looks like you are running a UNIX-like system. You may want to install a signal handler to reopen the filehandle and either alarm at midnight (and catch the ALRM signal) or send some signal (USR1, for example) via cron daemon and handle it. You can even automatically compress and/or remove old files using logrotate if you want.
    Sorry if my advice was wrong.
      my $timeout = 24*60*60 - ($hour*3600 + $min*60 + $sec); while(1) { eval { local $SIG{ALRM} = sub { die; }; alarm($timeout); while (my $line = <$sock>) { ................. } alarm(0); }; $timeout = 24*60*60; }
Re: Close a file and reopen a new one at midnight
by 0day (Sexton) on Jan 14, 2013 at 17:59 UTC
    You have to use a global variable of an open file:
    my $file; open $file, ">$filename" || die("Couldn't open file"); ...

    In addition, are you assured that at this time there is data on the socket for reading?

      That, however stylish it may be, using lexical file handles will not resolve the root cause of the problem. The file handle is being closed and reopened in a logically correct location.

      Update: Updated statement to clarify.

      --MidLifeXis