Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Meditations

( #480=superdoc: print w/replies, xml ) Need Help??

If you've discovered something amazing about Perl that you just need to share with everyone, this is the right place.

This section is also used for non-question discussions about Perl, and for any discussions that are not specifically programming related. For example, if you want to share or discuss opinions on hacker culture, the job market, or Perl 6 development, this is the place. (Note, however, that discussions about the PerlMonks web site belong in PerlMonks Discussion.)

Meditations is sometimes used as a sounding-board — a place to post initial drafts of perl tutorials, code modules, book reviews, articles, quizzes, etc. — so that the author can benefit from the collective insight of the monks before publishing the finished item to its proper place (be it Tutorials, Cool Uses for Perl, Reviews, or whatever). If you do this, it is generally considered appropriate to prefix your node title with "RFC:" (for "request for comments").

User Meditations
Forgetfulness and 6-months-from-now-you
1 direct reply — Read more / Contribute
by oakbox
on Jan 12, 2018 at 05:51
    Can anyone explain to me WHY I must constantly forget and relearn that:
    my $sth = $dbh->prepare("SELECT * FROM Something"); $sth->execute(); if($sth->err()){ die $sth->errstr(); }
    is functionally equivalent to
    my $sth = $dbh->prepare("SELECT * FROM Something"); $sth->execute(); if($dbh->err()){ die $dbh->errstr(); }
    ???

    Because every couple of years I will become frustrated with typing out the error check on the various statement handles.
    And then look into how I can make that easier.
    And RE-LEARN that I could have been checking the database handle and just copy/pasting that over and over.

    So frustrating, a real smack-to-the-forehead kind of moment.

    And I think it ties in with why I write code the way I do. I WRITE IT OUT. I take the time to format and indent and line up the code because I have learned 6-months-in-the-future-me will really appreciate it and I don't want him cussing at me, retroactively.

    Just needed to vent a little bit, please return to your regularly scheduled activities.

Repeating a substitution
4 direct replies — Read more / Contribute
by choroba
on Jan 09, 2018 at 05:39
    Inspired by Stack Overflow, again.

    A user asked for a (awk or similar) one-liner that would replace a separator by a different one in a file, but only the first N separators should be replaced.

    For small Ns, it's easiest to repeat the substitution:

    perl -pe 's/,/|/;s/,/|/;s/,/|/'

    But, what should one do when they want to replace the first 10 separators?

    My first idea was to use a for loop:

    perl -pe 's/,/|/ for 1 .. 10' # Oops!

    Unfortunately, it doesn't work, as the for creates another local $_ and the substitution happens to the numbers, not the input.

    So, my next idea was to use a counter with /e:

    perl -pe 's/,/$i++<10 ? "|" : ","/ge'

    It works, but is ugly and hard to explain to someone not familiar with Perl.

    Another way is to tie the two $_ variables together by aliasing the outer $_ by the inner one:

    perl -pe 's/,/|/ for $_, $_'
    Again, this works only for small number of substitutions.

    But, we can generalize a list of the same things: we can use the x operator in list context! It's short, readable, and follows the DRY principle:

    perl -pe 's/,/|/ for ($_) x 10'

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
To glob or not to glob
7 direct replies — Read more / Contribute
by haukex
on Jan 07, 2018 at 07:54

    I am often torn when it comes to recommending Perl's glob (aka the <...> operator, when it isn't readline). On the one hand, it's built in and often shortens code, on the other, it has several caveats one should be aware of.

    1. glob does not list filenames beginning with a dot by default. For someone coming from a unixish shell, this might make perfect sense, but for someone coming from, for example, a readdir implementation, this might be surprising, and so it should at least be mentioned.

    2. Probably the biggest problem I see with glob is variables interpolated into the pattern. The default glob splits its argument on whitespace, which means that, for example, glob("$dir/*.log") is a problem when $dir is 'c:/program files/foo'. This can be avoided by doing use File::Glob ':bsd_glob'; (Update: except on Perls before v5.16, please see Tux's reply and below for alternatives), but that doesn't help with the next problem:

    3. If a variable interpolated into the pattern contains glob metacharacters (\[]{}*?~), this will cause unexpected results for anyone not aware of this list and expecting the characters to be taken literally.

    4. Lastly, File::Glob can override glob globally. If, for example, you use it in a module, and someone else overrides the default glob, then suddenly your code might not behave the way you expected.

    5. <update> glob in scalar context with a variable pattern also suffers from surprising behavior, as choroba pointed out in his reply - thank you! (additional info) </update>

    That's why I think advising the use of glob without mentioning the caveats is potentially problematic. Perhaps one wants to create a backup of a folder and don't want to miss any files, say for example, .htaccess? And I also often see things like glob("$dir/*") going without comment.

    Personally I find readdir, in combination with some of the functions from File::Spec, to be a decent, if slightly complicated, tool (one example). One better alternative among several is children from Path::Class::Dir, or methods from one of the other modules like Path::Tiny. (Modules like File::Find::Rule often get mentioned as alternatives, except that those of course recurse into subdirectories by default.)

    use Path::Class qw/dir/; my @files = dir('foo', 'bar quz', 'baz')->children; # @files includes .dot files, but not . and .. # and its elements are Path::Class objects print "<$_>\n" for @files;

    Now, of course this isn't to say glob is all bad, I've certainly used and recommended it plenty of times. If one has read all of its documentation, including File::Glob, and is aware of all the caveats, and especially if one is using fixed strings for the patterns, it can be perfectly fine. But I still think it should not be blindly used or recommended.

Using constants as hash keys
3 direct replies — Read more / Contribute
by choroba
on Jan 04, 2018 at 14:03
    Context: a StackOverflow question on how to use constants as hash keys.

    Note that the module itself mentions the following:

    For example, you can't say $hash{CONSTANT} because CONSTANT will be interpreted as a string. Use $hash{CONSTANT()} or $hash{+CONSTANT} to prevent the bareword quoting mechanism from kicking in. Similarly, since the => operator quotes a bareword immediately to its left, you have to say CONSTANT() => 'value' (or simply use a comma in place of the big arrow) instead of CONSTANT => 'value'.

    The OP used &CONSTANT => 'value' which works but doesn't inline the constant (i.e. expand it during compile time).

    ikegami pointed me to a different way which I found more pleasing than CONSTANT() which, as he rightly noted, leaks the internal implementation of constants.

    use constant A => 12; my %hash = ( (A) => 'twelve' ); # beautiful

    I wanted to verify it behaves exactly the same, so I tried running it through B::Deparse, B::Terse, and B::Concise.

    m=Deparse diff <(perl -MO=$m -e 'use constant A => 12; my %hash = ( A() => "twel +ve")') \ <(perl -MO=$m -e 'use constant A => 12; my %hash = ( (A) => "twel +ve")')
    says the code is identical.

    Terse needs some tweaking to skip the pointers:

    m=Terse diff <(perl -MO=$m -e 'use constant A => 12; my %hash = ( A() => "twel +ve")' \ | perl -pe 's/0x\w+/X/g') \ <(perl -MO=$m -e 'use constant A => 12; my %hash = ( (A) => "twel +ve")' \ | perl -pe 's/0x\w+/X/g')

    But Concise shows a slight difference:

    m=Concise diff <(perl -MO=$m -e 'use constant A => 12; my %hash = ( A() => "twel +ve")') \ <(perl -MO=$m -e 'use constant A => 12; my %hash = ( (A) => "twel +ve")') 7c7 < 4 <$> const[IV 12] s*/FOLD ->5 --- > 4 <$> const[IV 12] sP*/FOLD ->5

    From the documentation it seems the P just means that A was parenthesized. I can imagine this information could be valuable to Perl (e.g. in the LHS of an assignment, but the structures of scalar versus list assignments are much more different); but probably not in this case.

    Update: Topics for meditation include other possible syntaxes (e.g. A ,=> 12), personal preferences, explanation of the Concise's output, etc.

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
Now released: Assert::Refute - A unified testing and assertion tool
1 direct reply — Read more / Contribute
by Dallaylaen
on Jan 02, 2018 at 04:55

    Hello dear esteemed monks,

    More than once I felt an urge to put a piece of a unit test script into production code to see what's actually happening there.

    Now there is excellent Test::More that provides a very terse, recognizable, and convenient language to build unit test. Unfortunately, it is not quite useful for production code. There are also multiple runtime assertion solutions, but they mostly focus on optimizing themselves out.

    My new module Assert::Refute is here to try and bridge the gap. The usage is as follows:

    use My::Module; use Assert::Refute qw(:all), {on_fail => 'carp'}; use Assert::Refute::T::Numeric; my $foo = My::Module->bloated_untestable_method; refute_these { like $foo->{bar}, qr/f?o?r?m?a?t/; can_ok $foo->{baz}, qw(do_this do_that frobnicate); is_between $foo->{price}, 10, 1000, "Price is reasonable"; };

    And this can be copied-and-pasted verbatim into a unit testing script. So why bother with runtime assertions? Possible reasons include:

    • Testing the method requires many preconditions/dependencies;
    • The method includes side effects and outside world interactions that are not easily replaced with mocks;
    • The method only misbehaves sporadically, doing what it should do most of the time, and the exact conditions required are not known;
    • The method needs to be refactored to be properly tested, and needs test coverage to be refactored.

    Main features include:

    • refute and subcontract calls allow to build arbitrarily complex checks from simple ones;
    • refute_these {...} block function to perform runtime assertions;
    • Prototyped functions mirroring those in Test::More to allow for moving checks between runtime and test scripts for optimal speed/accuracy tradeoff;
    • Object-oriented interface to allow for keeping the namespace clean;
    • Simple building and testing of custom checks that will run happily under Test::More as well as Assert::Refute;
    • Reasonably fast, at around 250-300K refutations/second on a 2.7GHz processor.

    This project continues some of my previous posts. Despite humble 0.07 version and documentation mentioning it's alpha, I think it is ready to be shown to the public now.

Programming Concepts
5 direct replies — Read more / Contribute
by aartist
on Dec 29, 2017 at 10:59
    Before a long time, I had taken a course, 'Programming Languages'. It discussed the various feature of programming languages and how it is implmented in a particular language. That could be the most interesting subject considering that I have spent few years in Programming. Unfortuanately much of that learning is lost in learning the syntax of the languages. Now, that I have programmed in various languages, I like to go to the basics again for my understanding.

    I like to know "what are the various programming concepts on which various languages have been designed", "Where do they differ" etc. So that, I can undertand or pickup any language based on concepts and pickup implementation of that concept later in my learning easily. I l like to see this in terms of Perl, Python, Java, Javascript and "your favorite language".

    Also now that, there are variety of framework for each language, I like to gain similar knowledge as on which concepts these framework are designed. Each frame work could have been designed based on different concepts.

    Any references would be useful.

    Thanks.

Verifying your distribution's revdeps still work after a change
1 direct reply — Read more / Contribute
by stevieb
on Dec 28, 2017 at 15:48

    In one of my pieces of software, Mock::Sub, I recently found that I wanted to add a new feature, and as always, I wrote a test file for the new work before modifying code. After making the first round of changes, I stumbled upon a previously unknown bug, so I opened an issue, and decided to tackle that before adding the new feature, to ensure all previous tests would run.

    The new feature is quite minor and actually requires a specific parameter to be sent in to change existing behavour (ie. no existing code that uses the distribution should have been affected), but the bug was a little more complex, and did change things internally.

    After I got the bug and new feature added, and before just blindly uploading it to the CPAN, I of course wanted to know whether the reverse dependencies (down-river distributions) would not be adversely affected, which would cascade Testers failure emails to the poor souls who's distributions I broke (most are mine in this case, but I digress).

    A long time ago, I went about writing a completely autonomous testing platform to do extensive testing on my repos against all Perlbrew/Berrybrew installations installed (it can dispatch out to remote systems as well). This distribution is Test::BrewBuild One of the core features I built into this software, is to automatically perform unit tests on all reverse dependencies as they currently sit on the CPAN against the changed code in the local distribution.

    I'll get right to it; it's pretty straightforward:

    First, ensure you're in your repository directory, and ensure all tests pass on the distribution you've just updated:

    ~/devel/repos/mock-sub$ make test t/00-load.t .................... ok t/01-called.t .................. ok t/02-called_count.t ............ ok t/03-instantiate.t ............. ok t/04-return_value.t ............ ok t/05-side_effect.t ............. ok t/06-reset.t ................... ok t/07-name.t .................... ok t/08-called_with.t ............. ok t/09-void_context.t ............ ok t/10-unmock.t .................. ok t/11-state.t ................... ok t/12-mocked_subs.t ............. ok t/13-mocked_objects.t .......... ok t/14-core_subs.t ............... ok t/15-remock.t .................. ok t/16-non_exist_warn.t .......... ok t/17-no_warnings.t ............. ok t/18-bug_25-retval_override.t .. ok t/19-return_params.t ........... ok t/manifest.t ................... skipped: Author tests not required fo +r installation t/pod-coverage.t ............... skipped: Author tests not required fo +r installation t/pod.t ........................ skipped: Author tests not required fo +r installation All tests successful. Files=23, Tests=243, 1 wallclock secs ( 0.05 usr 0.02 sys + 0.72 cu +sr 0.07 csys = 0.86 CPU) Result: PASS

    So far, so good (of course, I had already ensured the "skipped" tests pass as well). Now, after installing Test::BrewBuild, and ensuring you've got at least one instance of Perlbrew/Berrybrew installed, simply run the brewbuild binary, with the -R or --revdep flag. In the example below, for brevity, I've limited the testing against only the version of Perl I'm currently using. If I had tested against more versions (or left off the -o or --on flag it tests against all installed versions by default), each version would be listed under each revdep with the PASS or FAIL status:

    ~/devel/repos/mock-sub$ brewbuild -R -o 5.24.1 reverse dependencies: App::RPi::EnvUI, RPi::DigiPot::MCP4XXXX, Devel::Examine::Subs, PSGI::Hector, File::Edit::Portable, Devel::Trace +::Subs App::RPi::EnvUI 5.24.1 :: PASS RPi::DigiPot::MCP4XXXX 5.24.1 :: PASS Devel::Examine::Subs 5.24.1 :: PASS PSGI::Hector 5.24.1 :: PASS File::Edit::Portable 5.24.1 :: PASS Devel::Trace::Subs 5.24.1 :: PASS

    That's all there is to it. Now I am confident that my changes will absolutely not break any of the down-river distributions that require this one.

    Note: If there had of been failures, a bblog directory will be created, and the full test output of that distribution located into its own file for easy review as to what went wrong. The file contains everything related to the test run that you'd normally see output by the cpanm command. Example:

    ~/devel/repos/mock-sub$ ll bblog drwx------ 2 steve steve 4096 Dec 28 10:53 . drwxrwxr-x 8 steve steve 4096 Dec 28 10:53 .. -rw-rw-r-- 1 steve steve 8128 Dec 28 10:26 App-RPi-EnvUI-5.24.1-FAIL.b +blog

    Note 2: If you want to get an understanding of most of the stuff brewbuild is doing, simply throw in a -d 7 to enable full debug logging to stdout.

    Update: Here's an example with multiple versions of Perl installed:

    App::RPi::EnvUI 5.24.1 :: PASS 5.18.4 :: FAIL 5.24.0 :: FAIL RPi::DigiPot::MCP4XXXX 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS Devel::Examine::Subs 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS PSGI::Hector 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS File::Edit::Portable 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS Devel::Trace::Subs 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS
    ... and the resulting bblog directory entries:

    -rw-rw-r-- 1 steve steve 1085954 Dec 28 13:20 App-RPi-EnvUI-5.18.4-FAI +L.bblog -rw-rw-r-- 1 steve steve 1078097 Dec 28 13:20 App-RPi-EnvUI-5.24.0-FAI +L.bblog

    The output in the FAIL logs show that one of my dependencies for the distribution is behind a version on those two versions of Perl, so all I have to do is bump it in the Makefile.PL and re-run the tests. It has nothing to do with the Mock::Sub distribution at all, but did point out a different problem entirely solely with that dist. So I suppose that this tool is handy for warning about other issues outside of the current dist you're updating.

Read file text and find fibonacci series
3 direct replies — Read more / Contribute
by darkblackblue
on Dec 28, 2017 at 09:37

    Hi My goal is read only line from file , and find fibonacci series that has minimun 3 element.I want to control numbers from 1 to 6 digit. Fibonacci series starts 1,1,2,3,5,8,13,21,34,55,89, ...... .For example;File text document

    49693463452312343388645

    I division string 1 to 6 digit. Check 4 is fibonacci number , no , go ahaed 49 is fibonacci number , no, next 496 is fibonacci ,no, 4969 after 44693 ,after 496934 no.There isnt any fibonacci , go to next digit and do it again 9 , 96,969,9693,96934.

    use 5.010; use strict; use warnings; open(FILE, "<:encoding(UTF-8)", "aa.txt") or die "Could not open file: + $!"; my $numbers; while (<FILE>) { $numbers="$_" ; print "$_"; } chomp $numbers; print "\n$numbers"; my $len=length($numbers); print "\n$len\n"; my $i; my $abc; foreach my $i (0..$len){ foreach my $j (1..6) { print "$j----->"; $abc = substr($numbers,$i,$j); print "$abc\n"; } print "***********************************\n"; } close FILE;

    I read to find fibonacci series with PERFECT SQUARE. I didn't use it. C code example

    // C++ program to check if x is a perfect square #include <iostream> #include <math.h> using namespace std; // A utility function that returns true if x is perfect square bool isPerfectSquare(int x) { int s = sqrt(x); return (s*s == x); } // Returns true if n is a Fibinacci Number, else false bool isFibonacci(int n) { // n is Fibinacci if one of 5*n*n + 4 or 5*n*n - 4 or both // is a perferct square return isPerfectSquare(5*n*n + 4) || isPerfectSquare(5*n*n - 4); } // A utility function to test above functions int main() { for (int i = 1; i <= 10; i++) isFibonacci(i)? cout << i << " is a Fibonacci Number \n": cout << i << " is a not Fibonacci Number \n" ; + return 0; }

    2017-12-29 Athanasius restored original content

Efforts to modernize CPAN interface?
8 direct replies — Read more / Contribute
by nysus
on Dec 19, 2017 at 15:46

    CPAN works flawlessly, and as every PerlMonk knows, it is one of the Seven Programming Wonders of the World. But I'm curious to know if there have been any efforts to overhaul and modernize the CPAN interface, particularly PAUSE and the bug reporting site at rt.cpan.org. Is anything in the works? Has this been a topic of discussion amongst Larry Wall or any of the other lesser Perl Gods who oversee it?

    Personally, I would love to see these great resources brought up to date with a more modern interface. I know, I get it, these sites probably work without javascript and on every browser since Mosaic. But there must be a way to keep both the purists and developers who are into the superficialities happy.

    I'm curious to know what the seasoned Monks think. Does CPAN need a facelift?

    Downvotes and nasty ad homs welcomed and encouraged for entertainment purposes. More thoughtful comments are welcome as well.

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

database without mysql
2 direct replies — Read more / Contribute
by darkblackblue
on Dec 16, 2017 at 09:23
    SOLVED

    2017-12-30 Original content restored by Athanasius:

    hi , I was asked "how to create database without mysql or cvs" befeor a few week ago. Thanks for answers. I tried and create a program. Maybe you want to add new properties for good program. (there aren't some properties. example delete record from table , or update data in table , ....)
    #!/use/perl/bin print " #create database: CREATE TABLE dbismi\n #create database with +table: CREATE TABLE dbname tablename\n #veritaban&#305; olu&#351;turm +a ve table ekleme : CREATE TABLE dbname tablename colonname colonname + int chartext #delete database : DELETE dbname \n #delete table: DELE +TE dbname tabloname \n #insert values : INSERT INTO dbname tablename +colonname value colonname value colonname value \n #show database : F +IND ALL_DATABASES \n #show all tables in databases : FIND dbismi \n # +show values in table in database : SHOW dbname tablename \n #search v +alue in table and show record: SEARCH dbname tablename key_word\n" ; use utf8; binmode(STDOUT,":utf8"); if((uc(@ARGV[0]) eq 'CREATE') && uc(@ARGV[1]) eq 'TABLE' ){ chomp(my $dir = @ARGV[2]); my $dir = "../Desktop/$dir"; mkdir( $dir ,0755); print "Directory created successfully\n"; if(@ARGV[3]){ my $d_p = @ARGV[2]; my $file_name = @ARGV[3].".txt"; my $file = "../Desktop/$d_p/$file_name"; unless(open FILE, '>'.$file) { die "\nUnable to create $file\n"; } } my $numArgs = $#ARGV + 1; foreach my $argnum (4 .. (4+($#ARGV-4)*1/2)) { print FILE "$ARGV[$argnum]\t"; } print FILE "\n"; close FILE; } if((uc(@ARGV[0]) eq 'DELETE') ){ my $numArgs = @ARGV ; if($numArgs == 2 ){ my $del= @ARGV[1].'/*.txt'; my $deleted = unlink glob $del; print "$deleted files were removed\n"; my $dir = '../Desktop/'.@ARGV[1].''; rmdir( $dir ) or die "Couldn't remove $dir directory, $!"; print "Directory removed successfully\n"; } if($numArgs eq 3 ){ my $del= @ARGV[1].'/'.@ARGV[2].'.txt'; my $deleted = unlink $del; } } if((uc(@ARGV[0]) eq 'INSERT') and uc(@ARGV[1]) eq 'INTO' ){ my $filename = @ARGV[2].'/'.@ARGV[3].'.txt'; open(my $fh, '>>', $filename) or die "Could not open file '$filena +me' $!"; my $numArgs = $#ARGV + 1; foreach my $argnum (5 .. $#ARGV) { if($argnum%2 eq 0){ } else{ print $fh "$ARGV[$argnum]\t"; } } print $fh "\n"; close $fh; #print "done\n"; } if((uc(@ARGV[0]) eq 'FIND') ){ if(uc(@ARGV[1]) eq 'ALL_DATABASES' ){ my $path = '../Desktop'; die "Please specify which directory to search" unless -d $path +; my $num =1; opendir( my $DIR, $path ); while ( my $entry = readdir $DIR ) { next unless -d $path . '/' . $entry; next if $entry eq '.' or $entry eq '..'; print "$num ) $entry\n"; $num++; } closedir $DIR; } if(uc(@ARGV[1]) ne 'ALL_DATABASES' ){ my $dir = '../Desktop/'.@ARGV[1].''; opendir DIR,$dir; my @dir = readdir(DIR); close DIR; my $temp=1; foreach(@dir){ if (-f $dir . "/" . $_ ){ print "$temp ) Table : ".$_."\n"; } } } } if(uc(@ARGV[0]) eq 'SHOW'){ my $file =@ARGV[1].'/'.@ARGV[2].'.txt'; open my $info, $file or die "Could not open $file: $!"; while( my $line = <$info>) { print $line; } close $info; } if((uc(@ARGV[0]) eq 'SEARCH') ){ my $file =@ARGV[1].'/'.@ARGV[2].'.txt'; my $str =@ARGV[3]; chomp($str); open(my $file, '<', $file) or die "Could not open file '$filen' $!"; while(my $line = <$file>){ if($line=~/$str/){ print $line; } } close ($file); }
RTFM!!!!! (but if you didn't, no biggie)
5 direct replies — Read more / Contribute
by nysus
on Dec 15, 2017 at 08:12

    I was recently involved in a debate with marto. On the whole it was mild and though it started to go in circles, I think productive. I have no interest in rehashing the particulars of that debate. It's over. And this post is not directed specifically at marto and I thank him for his time and consideration. But I did want to add a few thoughts to this very age old, and probably tired, discussion about RTFM.

    I'll first say that yes, I agree, "RTFM" is very good advice. It was advice that was pounded into me probably within days of me joining the PM community 17 years ago. It is advice that I reflect upon quite often. One interpretation of it boils down to: "Be self sufficient. Don't expect others to do your work for you. Be very respectful of others' time and effort and don't expect an answer if you can't be bothered to put some work in for yourself."

    But like all rules of thumb, "RTFM" is open to wide interpretation. And so I offer another interpretation: "Learn how to figure things out for yourself, you'll be a better programmer for it. But if you feel like you are in over your head and you're feeling frustrated, go ahead and ask. It can't hurt. We'll try to be helpful, within reasonable limits, and you'll get the benefit of the doubt that you put in some effort into trying to figure it out."

    I would like to encourage Monks to provide that benefit of the doubt. It's most people's instinct to want to reciprocate when they see others being generous with help. And so I think erring on the side of generosity and helpfulness does a lot to enhance and build the reputation of Perl. Sure, there will be some takers who don't reciprocate or who constantly ask questions that can be answered with just a cursory read of the manual or google search. But we should be careful not to be so guarded against them that they prevent us from helping those who genuinely need a helping hand and that will be very appreciative of any specific guidance they receive.

    It's not easy advice to follow. We live in a dog-eat-dog, kill-or-be-killed kind of society that values hyper efficiency. Going out of your way to look something up for someone that strikes you as a lame brain does not seem like a very good use of your time. But I think, in the end, having faith in the good will of others gets us further down the road of progress. And I think that spirit of gift giving and reciprocation is what makes PM such a great community and Perl such a great programming language. I will do my absolute best to carry on the tradition.

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

New meaning to SmokeTesters?
9 direct replies — Read more / Contribute
by VinsWorldcom
on Dec 13, 2017 at 09:50

    What the heck is this:

    Perl Vape

    They even talk about "PODs" and "MODs" - it's too much!

The problem of "the" default shell
5 direct replies — Read more / Contribute
by afoken
on Dec 09, 2017 at 08:17

    I've got a little bit tired of searching my "avoid the default shell" postings over and over again, so I wrote this meditation to sum it up.

    What is wrong with the default shell?

    In an ideal world, nothing. The default shell /bin/sh would have a consistent, well-defined behaviour across all platforms, including quoting and escaping rules. It would be quite easy and unproblematic to use.

    But this is the real world. Different platforms have different default shells, and they change the default shell over time. Also, shell behaviour changed over time. Remember that the Unix family of operating systems has evolved since the 1970s, and of course, this includes the shells. Have a look at "Various system shells" to get a first impression. Don't even assume that operating systems keep using the same shell as default shell.

    And yes, there is more than just the huge Unix family. MS-DOS copied concepts from CP/M and also a very little bit of Unix. OS/2 and the Windows NT family (including 2000, XP, Vista, 7, 10) copied from MS-DOS. Windows 1-3, 9x, ME still ran on top of DOS. From this tree of operating systems, we got command.com and cmd.exe.

    By the way: Modern MacOS variants (since MacOS X) are part of the Unix family, and so is Android (after all, it's just a heavily customized Linux).

    Some ugly details:

    And when it comes to Windows (and DOS, OS/2), legacy becomes really ugly.

    So, to sum it up, there is no thing like "the" default shell. There are a lot of default shells, all with more or less different behaviour. You can't even hope that the default shell resembles a well-known family of shells, like bourne. So there is much potential for nasty surprises.

    Why and how does that affect Perl?

    Perl has several ways to execute external commands, some more obvious, some less. In the very basic form, you pass a string to perl that roughly ressembles what you would type into your favorite shell:

    • system('echo hello');
    • exec('echo hello');
    • open my $pipe,'echo hello |' or die "Can't open pipe: $!"; my $hello=do { local $/; <$pipe> }; close $pipe;
    • my $hello=qx(echo hello);
    • my $hello=`echo hello`;

    Looks pretty innocent, doesn't it? And it is, until you want to start doing real-world things, like passing arguments containing quotes, dollar signs, or backslashes to an external program. You need to know the quoting rule of whatever shell happens to be the default shell.

    For those cases, perl is expected to pass the string to /bin/sh for execution. Except that in this innocent case, and several other cases, perl does not invoke the default shell at all. Burried deep in the perl sources, there is some heuristics happening. If perl thinks that it can start the executable on its own, because the command does not contain what is documented as "shell metacharacters", perl splits the command on its own and can avoid invoking the default shell.

    Why? Because perl can easily figure out what the shell would do, and do it by itself instead. This avoids a lot of overhead and so is faster and does not use as much memory as invoking the shell would.

    Unfortunately, the documentation is a little bit short on details. See "Perl guessing" in Re^2: Improve pipe open? (redirect hook): From the code of Perl_do_exec3() in doio.c (perl 5.24.1), it seems that the word "exec" inside the command string triggers a different handling, and some of the logic also depends on how perl was compiled (preprocessor symbol CSH).

    If you don't need support from the default shell, you can help perl by passing system, exec, and open a list of arguments instead of a string. This "multi-argument" or "list form" of the commands always avoids the shell, and it completely avoids any need to quote.

    (Well, at least on Unix. Windows is a completely different beast. See Re^3: Perl Rename and Re^3: Having to manually escape quote character in args to "system"?. It should be safe to pretend that you are on Unix even if you are on Windows. Perl should do the right thing with the "list form".)

    So our examples now look like this:

    • system('echo','hello','here','is','a','dollar:','$');
    • exec('echo','hello','here','is','a','dollar:','$');
    • open my $pipe,'-|','echo','hello','here','is','a','dollar:','$' or die "Can't open pipe: $!"; my $hello=do { local $/; <$pipe> }; close $pipe;

    Did you notice that qx() and its shorter alias `` don't support a list form? That sucks, but we can work around that by using open instead. Writing a small function that wraps open is quite easy. See "Safe pipe opens" in perlipc.

    Edge cases

    OK, let's assume I've convinced you to use the list forms of system, exec, and open. You want to start a program named "foo bar", and it needs an argument "baz". Yes, the program has a space in its name. This is unusual but legal in the Unix family, and quite common on Windows.

    • system('foo bar','baz');
    • exec('foo bar','baz');
    • open my $pipe,'-|','foo bar','baz' or die ...

    or even:

    my @command=('foo bar','baz'); and one of:

    • system @command;
    • exec @command;
    • open my $pipe,'-|',@command or die ...

    All is well. Perl does what you expect, no default shell is ever involved.

    Now, "foo bar" get's an update, and you no longer have to pass the "baz" argument. In fact, you must not pass the "baz" argument at all. Should be easy, right?

    • system 'foo bar';
    • exec 'foo bar';
    • open my $pipe,'-|','foo bar' or die ...

    or:

    my @command=('foo bar'); and one of:

    • system @command;
    • exec @command;
    • open my $pipe,'-|',@command or die ...

    Wrong! system, exec, and even open in the three-argument form now see a single scalar value as the command, and start once again guessing what you want. And they will wrongly guess that you want to start "foo" with an argument of "bar".

    The solution for system and exec is hidden in the documentation of exec: Pass the executable name using indirect object syntax to system or exec, and perl will treat the single-argument list as list, and not a single command string.

    • system { 'foo bar' } 'foo bar';
    • exec { 'foo bar' } 'foo bar';

    or:

    my @command=('foo bar'); and one of:

    • system { $command[0] } @command;
    • exec { $command[0] } @command;

    If the command list is not guaranteed to contain at least two arguments (e.g. because arguments come from the user or the network), you should always use the indirect object notation to avoid this trap.

    Did you notice that we lost another way of invoking external commands here? There is (currently) no way in perl to use pipe open with a single-element command list without triggering the default shell heuristics. That's why I wrote Improve pipe open?. Yes, you can work around by using the code shown in "Safe pipe opens" in perlipc and using exec with indirect object notation in the child process. But that takes 10 to 20 lines of code just because perl tries to be smart instead of being secure.

    Avoiding external programs

    Why do you want to run external programs? Perl can easily replace most of the basic Unix utilities, by using internal functions or existing modules. And as an additional extra, you don't depend on the external programs. This makes your code more portable. For example, Windows does not have ls, grep, awk, sed, test, cat, head, or tail out of the box, and find is not find, but a poor excuse for grep. If you use perl functions and modules, that does not matter at all. Likewise, not all members of the Unix family have the GNU variant of those utilities. Again, if you use perl functions and modules, it does not matter.

    ToolPerl replacement
    echoprint, say
    rmunlink
    rm -rFile::Path
    mkdirmkdir
    mkdir -pFile::Path
    rmdirrmdir
    grepgrep (note: you need to open and read files manually)
    awka2p
    seds2p
    ls, findFile::Find, glob, stat, lstat, opendir, readdir, closedir
    test, [, [[stat, lstat, -X, File::stat
    cat, head, tailopen, readline, print, say, close, seek, tell
    lnlink, symlink
    chmodchmod
    chownchown
    touchutime
    curl, wget, ftpLWP::UserAgent and friends
    ftpNet::FTP
    sshNet::SSH2, Net::OpenSSH

    Note: The table above is far from being complete.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Better names for SCRIPT_NAME/PATH_INFO in a web framework?
2 direct replies — Read more / Contribute
by Dallaylaen
on Dec 02, 2017 at 13:42

    Hello dear esteemed monks,

    My toy framework called MVC::Neaf has crawled to 0.20 milestone. EDIT And it's got some misleading method names in it which I would like to correct.

    One thing I'm struggling to grasp is how to call requested path's fragments - the part that matched the current controller and whatever follows that path. The current convention is as follows:

    • script_name is the part of the path that matched current route, not the name of the script or the raw stuff from PSGI request.
    • path_info is anything that follows that matching part, for instance a wiki article name.
    • path_info_split is a newer version that takes regular expression capture groups into account.

    The overall /EDIT syntax, though still evolving, looks like follows now:

    use strict; use warnings; use MVC::Neaf qw(:sugar); get + post '/some/path' => sub { my $req = shift; my $foo = $req->param( foo ); my $bar = $req->param( bar => '\w+' ) # no params w/o validation or die 404; # render a "not found" page my ($from, $to) = $req->path_info_split; $req->script_name; # '/some/path' # frobnicate my $data = frobnicate( $bar, $from .. $to ); return { result => $data, foo => $foo, }; }, default => { -view => 'TT', # JS is the default which generates JSON -template => 'some_file.tt', title => 'My mega new application', version => '0.42', }, param_regex => { foo => '\d+' }, path_info_regex => '(\d+)/(\d+)'; # Hitting /some/path would trigger roughly something like follows # (assuming the controller doesn't die/redirect/stop otherwise) # $my_template->process( 'some_file.tt', { # title => ..., # version => ..., # result => ..., # # whatever else controller returned # } ); neaf->run;

    Some working examples exist in the distro.

    I know everyone else is using routes of form '/foo/:bar' these days but I don't really like it (although support may be added in the end). The reason is there are a number of formats (:foo, *bar, #baz) and they still don't cover all needed cases ([0-9]+ being the most obvious one).

    Now I feel like like these SCRIPT_NAME and PATH_INFO dating back into past century are awkward. My proposal is to rename this stuff completely:

    • $req->route() stands for the route that matched (Dancer does like that already).
    • $req->suffix() returns the URI capture groups if any were defined.
    • suffix => qr/(...)and(...)/ is the corresponding route parameter.
    • get '/article/(\d+)/(\d\d)/(\d\d)/(.*)' => ... - I'm also thinking about adding capture groups right into the path (w/o suffix parameter) but that'd be hard to undo so I'm holding it back.
    • param => \%spec is the predefined regular expression hash for $req->param( "name" ); (Neaf explicitly forbids unvalidated params and cookies, much like perl -T does).

    This scheme looks consistent and clear to me, but maybe I'm missing something. Does that look like a syntax you'd like to try out? What would you like to be added/removed? What is causing surprise and awkwardness here?

    Thank you,

contest problem about shortest Regular expression
2 direct replies — Read more / Contribute
by rsFalse
on Nov 28, 2017 at 16:11
    Hello.

    A week ago I met an interesting problem about regular expressions. It was from the OpenCup contest held in 2017 november 19. Here I will post a statement, and I ask to discuss how to solve this problem, what strategies and tools to use. I recommend to try to solve this problem by yourselves, and if you would like, you can copy pdf statements with this link, problem number 7 (You can download statement, while the link works, but you can't access and upload your solution for a testing system if you are not participating with your team from your high school). I used perlbrew switch v5.14.4 to match a version of testing system, and before I was failing with v5.18 because of some newer features which I used. At the bottom I will write my approaches and code.

    Statement: Here I will write ideas about solving.
    Firstly, I was thinking about two approaches: 1) to generate regex from an input data, e.g. to make a regex from the first input line, and later modify (expand) it when analysing next lines; 2) to generate all possible regexes, then sort them by length, then try to match all input lines against shortest regex, if fail then match against longer regexes.
    The first approach seems more sophisticated and I haven't found any ideas how to solve by that way. Can you suggest something? Second approach seems easier, and I successfully used it.
    Secondly, I realized that there are a regular expression which can match any possible line composed by only 4 distinct letters. So, I need to generate all regular expressions, which are shorter than all-matching-regex. Thirdly, I was thinking how to generate possible regexes. There were one idea at first. Later I got another. First idea was to generate all possible permutations of letters, and later add all possible permutations of other symbols into permutations of letters. This approach take much time for me, and it wasn't successful. Of course I tried to find some logics and avoid generating regular expressions which are longer than their shorter equivalents. Second idea was to generate regular expressions by joining smallest ones - only letters (say "atoms") - to the bigger and bigger ones. That expanding was performed by binary joining or by enveloping regex with Kleene star.
    Further are more ideas about solution, and the code.

    ... I was thinking that the type of "abstract" regexes which have backreferences (<named> for a comfort) can be useful. But I am not sure. I tried unsuccessfully.
    Maybe this problem can be solved with more advanced techniques: recursive regexes, eval-groups, and other?

    Related topic: Perl in programming contests and problem solving
    #regexes #qr #golf #problem #efficiency
    CODE: INPUT OUTPUT: OUTPUT verbose ($debug = 1;):

Add your Meditation
Title:
Meditation:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others browsing the Monastery: (6)
    As of 2018-01-20 16:10 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?
      How did you see in the new year?










      Results (227 votes). Check out past polls.

      Notices?