Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: reading through dated directories

by choroba (Archbishop)
on Jun 17, 2021 at 20:03 UTC ( #11133972=note: print w/replies, xml ) Need Help??


in reply to reading through dated directories

I created the following Makefile to simulate your situation:
SHELL = /bin/bash .PHONY: run run: compare $(MAKE) glob > /dev/null $(MAKE) readdir > /dev/null $(MAKE) path > /dev/null .PHONY: prepare prepare: gen.sh .PHONY: clean clean: rm -rf {100..200} .PHONY: glob glob: time glob.pl .PHONY: readdir readdir: time readdir.pl .PHONY: path path: time path.pl .PHONY: compare compare: diff <(glob.pl | sort) <(readdir.pl | sort) diff <(glob.pl | sort) <(path.pl | sort)

The gen.sh is used to generate the directories:

#! /bin/bash now=$(date +%s) yday=$(date -d yesterday +%s) tmrw=$(date -d tomorrow +%s) mkdir {100..200} for d in {100..200} ; do for r in {1..250} ; do mkdir $d/$(date +%Y%m%d -d @$((yday-10000*RANDOM))) done 2>/dev/null mkdir $d/$(date +%Y%m%d -d @$((tmrw+10*RANDOM))) for r in {1..250} ; do mkdir $d/$(date +%Y%m%d -d @$((yday-10000*RANDOM))) done 2>/dev/null echo $d >&2 done

glob.pl uses glob. It returns the entries sorted, so the last one is the future one, as dates sort alphabetically in the YYYYMMDD form (at least for the nearest future).

#!/usr/bin/perl use warnings; use strict; use feature qw{ say }; use Time::Piece; my $now = localtime->ymd(""); for my $dir (glob '???/') { say substr +(glob "$dir*/")[-1], 0, -1; }

path.pl uses Path::Tiny.

#!/usr/bin/perl use warnings; use strict; use feature qw{ say }; use Time::Piece; use Path::Tiny qw{ path }; my $now = localtime->ymd(""); for my $dir (grep $_->is_dir, path('.')->children) { for my $date (grep $_->is_dir, $dir->children) { say $date if $date->basename > $now; } }

readdir.pl uses opendir and readdir.

#!/usr/bin/perl use warnings; use strict; use feature qw{ say }; use Time::Piece; my $now = localtime->ymd(""); opendir my $dir, '.' or die $!; while (my $pdir = readdir $dir) { next if $pdir =~ /^\.{1,2}$/ || ! -d $pdir; opendir my $ddir, $pdir or die $!; while (my $date = readdir $ddir) { next if $date =~ /^\.{1,2}$/ || ! -d $dir; say "$pdir/$date" if $date > $now; } }

Run make compare to verify all three scripts give the same output.

The results on my machine were

make glob > /dev/null real 0m0.109s user 0m0.054s sys 0m0.053s make readdir > /dev/null real 0m0.074s user 0m0.045s sys 0m0.029s make path > /dev/null real 0m0.440s user 0m0.380s sys 0m0.060s

You can see, readdir.pl has the most verbose code as it's the most low-level, but it's the fastest one. Glob is a bit slower, but still pretty good. Path::Tiny didn't really shine, but maybe its code can be improved.

map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

Replies are listed 'Best First'.
Re^2: reading through dated directories
by Anonymous Monk on Jun 17, 2021 at 21:02 UTC
    Thank you thank you for the amazing examples & the benchmarks! Wow. You monks never cease to amaze me!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11133972]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (2)
As of 2021-09-24 01:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?