Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Monitor new file

by colox (Sexton)
on Dec 05, 2017 at 12:16 UTC ( #1204952=perlquestion: print w/replies, xml ) Need Help??
colox has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks, From my other posts/inquiries, I am using File::Find to search through a directory for files new than a timestamp I saved into a hash file. However, I realized this is very inefficient as it has to comb through the whole directory & check each & every file in it to determine if the file timestamp is newer than my reference (hash). Any recommendation (module) that would do the task more efficient? Thank you as usual for your valuable inputs & advise.

Replies are listed 'Best First'.
Re: Monitor new file
by Corion (Pope) on Dec 05, 2017 at 12:19 UTC

    You could look at Inotify, Win32::FileNotify or some other notification mechanism for your OS.

    Other than that, there is no mechanism other than getting the full list of directory entries and looking for the changes.

Re: Monitor new file
by Laurent_R (Canon) on Dec 05, 2017 at 13:32 UTC
    Hi colox,

    are you monitoring all the files in the directory tree or are you interested in tracking only a limited number of them? I am asking because if you're monitoring only a smaller number of files (and not each and every file in the directory tree), then it might possibly be better (faster) not to comb through the whole directory with File::Find, but rather to start from the list of files in your hash and check only these files' age with the relevant file test operator.

Re: Monitor new file
by karlgoethebier (Monsignor) on Dec 05, 2017 at 17:36 UTC

    May be you actually want a "hotfolder".

    On Linux (which i don't have anymore) there is File::Hotfolder.

    Here is a sketch written in a hurry and not thoroughly tested for the Mac using Mac::FSEvents:

    #!/usr/bin/env perl use strict; use warnings; use Data::Dump; use Mac::FSEvents; use IO::Select; use feature qw(say); my $dir = q(/Users/karl/Desktop/somewhere; my $fs = Mac::FSEvents->new( path => $dir, file_events => 1, ); my $fh = $fs->watch; my $select = IO::Select->new($fh); while ( $select->can_read ) { my @events = $fs->read_events; # dd \@events; for my $event (@events) { say $event->path; say for ( stat( $event->path ) )[ 7, 9 ]; } } __END__

    With File::Hotfolder something similar should be possible.

    Best regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

    perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help

Re: Monitor new file
by thanos1983 (Vicar) on Dec 05, 2017 at 16:26 UTC

    Hello colox

    In the past "something similar" was asked from another fellow Monk see Method to verify an environment with changes.

    You might be looking for something like:

    #!/usr/bin/perl use strict; use warnings; use File::stat; use Data::Dumper; use Time::localtime; use File::Find::Rule; sub get_files { my @dirs = ('/home/user/Monks' , '/home/user/Monks/mySubDir'); # a +dd more my $level = shift // 2; # level to dig into my @files = File::Find::Rule->file() ->name('*.xml', '*.txt') # add your file(s) extension(s) ->maxdepth($level) ->in(@dirs); return @files; } my @files = get_files(); my %hash; for (@files) { $hash{$_} = ctime(stat($_)->mtime); } # or # for (@files) { $hash{ctime(stat($_)->mtime)} = $_; } print Dumper \%hash; __END__ $ perl $VAR1 = { '/home/user/Monks/ArabicCharacters/original.txt' => 'Fri Jul + 28 13:21:04 2017', '/home/user/Monks/out.txt' => 'Thu Jul 20 13:29:41 2017', '/home/user/Monks/filter.txt' => 'Thu Jul 13 11:03:53 2017', . . . };

    Although you said that you have implemented something similar, I would combine it with git CPAN module(s). At the end with the help of git you can check for the file(s) when they where last updated based on the date and then check what has changed and by whom?

    Hope this helps, BR.

    Seeking for Perl wisdom...on the process of learning...not there...yet!

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1204952]
Front-paged by haukex
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2018-10-19 00:31 GMT
Find Nodes?
    Voting Booth?
    When I need money for a bigger acquisition, I usually ...

    Results (106 votes). Check out past polls.