Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked


by GotToBTru (Vicar)
on Jun 15, 2010 at 13:25 UTC ( #844862=user: print w/ replies, xml ) Need Help??

Learned Perl

around 2001 when an EDI trading partner insisted on using FTP instead of a VAN. I needed a quick way to send and receive that could be embedded in a scheduler, and I knew "Perl does that." 4 hours later, we were up and running.

I use Perl to examine data files and system logs, to produce automated emails and database updates, to perform file transformations, and to perform automated file transfers (moves and/or ftp).

The name GotToBTru is from the title of a song by Christian singer/songwriter Steven Curtis Chapman, and marked his first foray into something approximating rap music. He uses the lyrics to poke fun at himself (and also his listeners). Two things I like: despite his "superstar" status he doesn't take himself too seriously, and the reminder in the lyrics that nobody will believe my words unless my behavior backs them up.


“There is nothing noble in being superior to your fellow man; true nobility is being superior to your former self.” ~ Ernest Hemingway

The plural of 'anecdote' is not 'data'.

Social Media

LinkedIn Profile


Blog (infrequent updates!)

Other interests:

Amateur (ham) radio (callsign AE0Z), Toastmasters(CC,ALB), vocal and instrumental music, Christian apologetics, Torah observance for Christians


is a Yaesu FT-101ZD ham radio transceiver. This was the last of several models of this radio made in the 1970s and 80s. Note the stylish LCD display. My own (not pictured) is almost as old as Linux.

Well, that about sums it up for evolution:

Life has never been observed to come from non-life.

There is no known observable process by which new genetic information can be added to the genetic code of an organism.

And, some of the reasons to investigate Intelligent Design seriously:

Three Scientific Evidences that point to a designed universe by Dr. Walter L. Bradley

Useful Links:

last hour of cbfullpage chat
Re: Real Life Perl ExercisesDBI recipes
Basic debugging checklistLevels of Monks
Linking ShortcutsMarkup in the Monastery
Writeup Formatting TipsHow do I post a question effectively?
Keyboard ShortcutsReferences quick reference
Regex TesterData Structures Cookbook
Module StyleExcellent Hash Reference
Approval Log

Posts I'm proud of:

Although, if I take too much (any really) pride in my posts here, I need only notice that there is a strong inverse correlation between rating and how much I type!


679th Saint in the Book December 10th, 2014.

Fun Milestones

8/31/15 - You have 1248 points until level 16 - Parson.


Version: 3.1
GCS d-- s:+ a+ C++$ ULA$ P++$ L++>$ !E--- W++ !N !o !K--
w+ !O !M V+>$ PS- PE !Y PGP>$ t+ !5 !X !R tv b++ DI++++ D+
G-- e++ h---- r+++ y?

Style Code
I2Osi0S<+>++<gt>B0L1C2P0N>R1vl(en)c1a1p(0)r1d0Hsw main-sub

Posts by GotToBTru
Searching over multiple directories using unusual logic in Seekers of Perl Wisdom
1 direct reply — Read more / Contribute
by GotToBTru
on Oct 27, 2015 at 13:24

    I am looking for either suggestions for improvement, or ways to use existing modules like File::Find::*.

    I wrote a utility to search through file archives organized according to a direction/date/topic structure. I usually know which direction and topic to search, but the transaction may have been archived on a range of days. I wrote my own very limited File::Find (code below) in order to implement this search.

    For instance, I am looking for a transaction we sent containing the string "12345678", I know it's for CustomerD, and I'm pretty sure we sent it this week, so it could be in:

    outbound/20151027/CustomerD outbound/20151026/CustomerD ... outbound/20151021/CustomerD

    outbound/2015nnnn/ will have many subdirectories, and some of them will have hundreds of files. As a result, if I can't supply the topic, I run the search in the background and work on something else. But if I can, the response is quick enough.

    So why explore modules if I have a working solution? Learning what's in CPAN, and how to better use it, is to my benefit.

    Source code:

    Dum Spiro Spero
Free Space Usage Report in Cool Uses for Perl
No replies — Read more | Post response
by GotToBTru
on Aug 04, 2015 at 09:22

    The following is the latest incarnation of something I wrote a few years back, and have refined as I have learned new tricks. The output of the df -k command is filtered for a list of directories. If the free space has changed since I last ran the command, it is stored. I also store a timestamp along with it, in case I want to look at historical trends. Right now, it keeps only the latest 8 reports.

    #!/usr/bin/perl # List free space left use strict; use warnings; use Storable; my $storable_file = '/home/edi/howard/dfk.storable'; my (@reports); @reports = @{retrieve($storable_file)} if ( -e $storable_file); my %new = map {(split /\s+/,$_)[6,3]} grep {/home|archive|edi_store/} +`df -k`; $new{'timestamp'} = time; unshift @reports, \%new; if (print_report(@reports[0,1])) { pop @reports if (scalar @reports > 7); store(\@reports,$storable_file); } sub print_report { my ($nhr,$ohr) = @_; my $change_flag = defined $ohr ? 0 : 1; foreach my $key ( keys %{$nhr}) { next if $key eq 'timestamp'; if (defined $ohr->{$key} && $ohr->{$key} ne $nhr->{$key}) { printf "%-17s was: %3s now: %3s\n", $key, $ohr->{$key}, $nhr->{$ +key}; $change_flag = 1; } else { printf "%-17s: %3s\n",$key,$nhr->{$key} } } return $change_flag; }
    Dum Spiro Spero
Distribute the leftovers in Cool Uses for Perl
No replies — Read more | Post response
by GotToBTru
on Mar 20, 2015 at 12:46

    We get the quantity of a product shipped from the database, and a list of individual carton labels from an external file. The actual contents of each carton are unknown. Working assumption would be qty per carton = shipped qty / carton count. Easy enough. But what if the division doesn't come out clean?

    use POSIX qw/ceil/; ... $sq = $shipped_quantity; $cc = $carton_count; foreach $label(@label_list) { if ($cc == 0) { print "Qty per carton math error"; next OUTER; } $per_container = ceil($sq/$cc); $sq -= $per_container; $cc -= 1; ... }

    If shipped quantity is 8 and carton count is 5, first 3 cartons will have 2 each, last 2 have 1.

    Yeah, almost too simple to mention. But this came up today, and I remember little code bits like this better when I write them down somewhere.

    Update: s/box/carton/

    Dum Spiro Spero
Archive by month and extension in Cool Uses for Perl
No replies — Read more | Post response
by GotToBTru
on Nov 24, 2014 at 16:01

    Created the following to archive data from our applications. We archive by month, and by file extension, so those are built in assumptions in this program.

    Source Code:


    1 Peter 4:10
File test in grep not excluding current directory in Seekers of Perl Wisdom
4 direct replies — Read more / Contribute
by GotToBTru
on Jul 01, 2014 at 09:22

    I am not sure if it is File::Find or grep that is responsible for the behavior I see.

    use strict; use warnings; use File::Find; my $dir='/home/edi/wlsedi/howard/temp'; find({preprocess => sub { return grep { -M $_ < 1 } @_ }, wanted => sub { printf "%s\n",$_ if (-f $_) } }, $dir);

    Output(as expected):

    file1 file2 file3

    I would rather have the directory case handled in preprocess than wanted.

    use strict; use warnings; use File::Find; my $dir='/home/edi/wlsedi/data_backup/univfiledrop'; find({preprocess => sub { return grep { -f $_ && -M $_ < 1 } @_ }, wanted => sub { printf "%s\n",$_ } }, $dir);


    . file1 file2 file3

    Why is . included?


    File::Find calls the wanted function with the directory name before it performs the readdir() on the directory. The preprocess routine is not called for this invocation.

    use strict; use warnings; use File::Find; my $p = 0; my $dir='/home/edi/wlsedi/howard/temp'; find({preprocess => sub { printf "p %d %s\n",$p++,$_; return @_ }, wanted => sub { printf "w %d %s\n",$p++,$_ } }, $dir);
    ls -e /home/edi/wlsedi/howard/temp total 0 drwxr-xr-x- 2 wlsedi wlsedi 256 Jul 01 09:33 dirhere -rw-r--r--- 1 wlsedi wlsedi 0 Jul 01 08:14 file1 -rw-r--r--- 1 wlsedi wlsedi 0 Jul 01 08:14 file2 -rw-r--r--- 1 wlsedi wlsedi 0 Jul 01 08:14 file3


    w 0 . p 1 . w 2 file1 w 3 file2 w 4 file3 w 5 dirhere p 6 dirhere
    1 Peter 4:10
Storing Experience for Posterity in Cool Uses for Perl
4 direct replies — Read more / Contribute
by GotToBTru
on Jun 12, 2014 at 10:34

    I put the following together to scrape experience, level and writeups off my profile page and store it in a text file on my computer, to record my progress through the Monastery. A scheduled task runs this once a day.

    use strict; use warnings; use LWP::Simple; use URI::URL; my $date=`ECHO %DATE:~10,4%%DATE:~4,2%%DATE:~7,2%`; # YYYYMMDD my $url = url(''); my $content = get($url); $content =~ s/\cJ//g; $content =~ s/\cM//g; my ($experience, $level, $posts) = ($content =~ /Experience:\D+(\d+).+ Level:.+([A-Z][a-z]+\s+\(\d+\)).+ Writeups:.+>(\d+)</x); open my $ofh, '>>','perl_xp.dat'; printf $ofh "%d,%d,%d,%s\n",$date,$experience,$posts,$level; close($ofh);

    There is probably a way to do this in Javascript that could be included in the Free Nodelet, but that's beyond my skill level.


    Improved version here.

    1 Peter 4:10
Debugger Questions - Variable Scope in Seekers of Perl Wisdom
4 direct replies — Read more / Contribute
by GotToBTru
on May 15, 2014 at 18:43

    If I run the following code in the debugger:

    use strict; my $t = 14; blah(); print "$t\n"; sub blah { my $t = 42; print "$t\n"; }

    and I place a watch on $t, it applies to both the main program variable and the subroutine variable, even though they are distinct. When the assignment statement in the subroutine executes, it reports that the value changed from 14 to 42, which is not true. Perl is not confused, the values of the variables stay distinct, but the debugger can't keep the namespaces distinct. Or perhaps it can -- is there a way to tell the debugger I want to watch only the variable in the subroutine?

    Also, if I make calls to modules, variables in my code go out of scope. If I am watching them, the debugger halts twice, once to tell me the variable now has no value, and once again to tell me it has reappeared! Is there a way to avoid these halts?

    Neither of these issues is more than an annoyance, but just in case I'm missing something ...

    1 Peter 4:10
Remove RTF Formatting in Cool Uses for Perl
2 direct replies — Read more / Contribute
by GotToBTru
on Feb 22, 2011 at 14:38
    We have clients sending us (in many ways) documents (in many formats) which we process automatically instead of having somebody key the data in. Applications like this are basically why my group exists at work.

    Latest challenge: file is in Rich Text Format. I don't need to parse it, or convert it into HTML. I just want the data, absent any formatting.

    Here is my end result. The regexes could no doubt be tightened up.

    use strict; use warnings; my ($wholefile,$line); my $outfilename='orderdata.out'; {open INFILE, '<', "orderform.rtf"; local $/; $wholefile=<INFILE>; close(INFILE); $wholefile =~ s/^\s*{.*\n//g; # remove lines starting wi +th braces $wholefile =~ s/\\[\w-]+\b//g; # remove RTF commands $wholefile =~ s/(\n)\s+(\S+)/$1$2/g; # remove extra spaces $wholefile =~ s/([^\\\n]+)\\(\n)/$1$2/g; # get rid of \ at the ends + of lines $wholefile =~ s/\\\n//g; # get rid of lines with a +\ only open OUTFILE,'>',$outfilename or die "Can't open $outfilename!\n"; print OUTFILE $wholefile; close(OUTFILE); }
    Update: applied suggestions to regexes #1, #2
Recognize X12 Interchanges in Cool Uses for Perl
1 direct reply — Read more / Contribute
by GotToBTru
on Aug 05, 2010 at 14:47

    The question came up on a list dedicated to EDI (electronic data interchange) about how to recognize EDI data in a file. Interchanges formatted in accordance with ANSI X12 standards are made up of labeled, delimited records called segments.

    A pair of segments form the outermost "envelope" around the data, the ISA and IEA segments. The ISA segment is 106 characters long, including the segment terminator character. The terminator tells you how to break the file into segments and another character, the element delimiter, tells how to break the segments into elements. This last is the 4th character in the ISA segment. These two characters, the segment terminator and element delimiter, are not specified in the standards and companies/industries use a variety of choices.

    Ever since I first learned about Perl I figured it would be ideal for dealing with EDI files. I worked out a regexp for identifying an EDI file by the presence of this fixed length ISA segment. The result is the ability to recognize an X12 file, without knowing ahead of time which delimiter and terminator your trading partner chose, and it delivers them up to you as a side effect of recognition.

    ($elemdelim,$segterm) = ( $contents =~ /ISA(.)..\1.{10}\1..\1.{10}\1..\1.{15}\1..\1.{15}\1.{6}\1. +{4}\1.\1.{5}\1.?.{9}\1.\1.\1.(.)/);

    It is said among X12 experts that the ISA is not fixed length. Element 14 is a 9 digit control number, which could be negative (according to the data type definition of the element but in complete contravention to the practical use of control numbers), in which case the ISA would be 107 characters long. This is silly, in my opinion, but is also easily dealt with using ".?".

    I have also opted to use the . instead of \d or \w. That's just because I can't be bothered to type two characters when one will do.

Log In?

What's my password?
Create A New User
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2015-11-30 07:17 GMT
Find Nodes?
    Voting Booth?

    What would be the most significant thing to happen if a rope (or wire) tied the Earth and the Moon together?

    Results (763 votes), past polls