Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

Re^2: Reading every file in a directory

by spookyjack123 (Initiate)
on Sep 29, 2012 at 01:35 UTC ( #996318=note: print w/replies, xml ) Need Help??

in reply to Re: Reading every file in a directory
in thread Reading every file in a directory

Every file is the same format (.yml) as well as the same format, I need a system like a glob that is super light for the job, and short. one that opens all files in the directory regardless of format type, just something where a directory is assigned, and it opens the files. Would this serve that, or is there an even simpler method ?

  • Comment on Re^2: Reading every file in a directory

Replies are listed 'Best First'.
Re^3: Reading every file in a directory
by Kenosis (Priest) on Sep 29, 2012 at 01:49 UTC

    Here's an option:

    use strict; use warnings; my $directory = '.'; for my $YMLfile (<"$directory/*.yml">) { open my $fh, '<', $YMLfile or die $!; while (<$fh>) { # process the file's contents } close $fh; }

    Update: Perhaps I've misunderstood your issue. In one place you say that you want the files " be opened so the timestamp may be read...," as if the timestamp may be a string within the file. Yet, at the beginning, you mention a script that can "...check the age of over 20,000 YML files..." In the latter case files are not opened. If it's the latter case, the following may assist you:

    use strict; use warnings; use File::stat; my $directory = '.'; my $currentTime = time; for my $YMLfile (<"$directory/*.yml">) { my $mtime = stat($YMLfile)->mtime; print "$YMLfile: " . ( $currentTime - $mtime ) . " seconds old\n"; }

    You can also try the -M operator on the files, which returns the file's age in days:

    for my $YMLfile (<"$directory/*.yml">) { print -M $YMLfile, "\n"; }

    And the following PM node may be helpful: How do I find and delete files based on age?.

    I strongly suggest running your file-deleting script on a practice directory.

      Oh 1 note I just discovered, I believe the sample here that you posted:

      use strict; use warnings; my $directory = '.'; for my $YMLfile (<"$directory/*.yml">) { open my $fh, '<', $YMLfile or die $!; while (<$fh>) { # process the file's contents } close $fh; }

      That code is good for opening the directory, I'm super grateful :) ! now reading and parsing the files is the next main issue for this script, the login line in the comment I last posted needs to be assigned to a value that can be read, then the login: followed by a space, need to be removed so it only shows the timestamp (the parsing part) then comparing will be super easy. Any ideas there ?

        Hello spookyjack123, and welcome to the Monastery!

        I think the following (untested) code will do what you want:

        #! perl use strict; use warnings; my $current_time = time; my $old_time = $current_time - (60 * 60 * 24 * 5); printf "Current time is %d, old time is %d\n", $current_time, $old_tim +e; my $directory = '/home/gac3/plugins/Essentials/userdata'; my @to_delete; for my $YMLfile (<"$directory/*.yml">) { open(my $fh, '<', $YMLfile) or die "Cannot open file '$YMLfile' fo +r reading: $!"; while (<$fh>) { if (/ ^ \s* login: \s+ ( \d+ ) $ /x) { push @to_delete, $YMLfile if int($1 / 1_000) < $old_time; last; } } close $fh or die "Cannot close file '$YMLfile': +$!"; } unlink @to_delete;

        Note that the timestamps in your sample data are 3 digits longer than the output of Perl’s time function. I have simply truncated the timestamp; you may need to adjust this part of the script.

        Hope that helps,

        Update: Fixed the call to unlink. Thanks to Kenosis for the heads-up!

        Athanasius <°(((><contra mundum

        Try the following on a scratch directory that contains some scratch .yml files:

        use strict; use warnings; my $directory = './yml'; my $fiveDays = 5 * 86400; my $currentTime = time; my ( $fileLine, $timeStamp ); for my $YMLfile (<"$directory/*.yml">) { open my $fh, '<', $YMLfile or die $!; while ( $fileLine = <$fh> ) { last if ($timeStamp) = $fileLine =~ /login: (\d+)/; } close $fh; # Divide 13 digit TS by 1000 to convert to 10 digit TS if ( ( $currentTime - $timeStamp / 1000 ) > $fiveDays ) { print "Login TS in $YMLfile is greater than five days old.\n"; } }

        Verify that the files shown by the routine have a login timestamp (TS) greater than five days. If the script is working as needed/expected, try the following on your scratch dir and files (place it on the next line after the print statement, so files are conditionally unlinked):

        unlink $YMLfile or warn "Could not unlink $YMLfile: $!";

        Disclaimer: Be very careful with this file-deleting script, as you're assuming all responsibility for the use of it.

      Sorry about my vagueness, that code helps, but let me explain more, I'm totally lost as a noob. basically, there is 20,000 or so .yml files in the directory /home/gac3/plugins/Essentials/userdata/ these files contain a line labled login, that record the last login of a user, I need to take that timestamp, compare it to the current time, minus 5 days, so if the origin file has a user which has not logged in after 5 days, the .yml file is deleted. here is an example yml file.

      timestamps: login: 1348876288937 logout: 1348876601161 lastteleport: 1348876478686 ipAddress: lastlocation: world: Main_Flat2 x: -10.627933608068007 y: 38.9038268486379 z: -20.417177146511058 yaw: -192.60002 pitch: 24.150005 nickname: žlJack powertools: {} afk: false money: 0.0 homes: gf: world: Main_Flat x: -81.00634258469903 y: 50.0 z: 80.56448372272854 yaw: 238.24374 pitch: 78.49189

      That line labled login, is what needs to be read, the script needs to read that line in every single file in the directory, all of which are the same format, and have that as a value which can then be compared. Any ideas ?

Re^3: Reading every file in a directory
by Anonymous Monk on Sep 29, 2012 at 06:38 UTC

    Would this serve that, or is there an even simpler method ?

    You tell me? Try It To See?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://996318]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (10)
As of 2017-12-11 14:00 GMT
Find Nodes?
    Voting Booth?
    What programming language do you hate the most?

    Results (295 votes). Check out past polls.