Nothing to read during next holidays? ;=)

I wrote this tutorial and I'll really appreciate your comments and corrections.

The following tutorial is in part the fruit of a learn by teaching process, so before pointing newbies to my work I need some confirmations.

The tutorial is a step by step journey into perl module development with tests, documentation and git integration. It seemed to me the very minimal approach in late 2018.

Being a long post (over the 64kb perlmonks constraint) the second part is in a reply to this node and because of this, the table of content links are broken for the second part (I'll fix them sooner or later).

This material is already on its github repository with a long name (I hope can be easier to find). Also the code generated in this tutorial has its own archived repository.

I'll gladly accept comments here or as pull request (see contributing), as you wish, about:

• the testing part: i've managed only the basics of testing but i'm quite new in this; please review
• errors in the overall discussion of the matter
• english errors or misuses of terms (I tend to construct phrases in a latin way.. sorry ;)
• git related errors

The module code presented is.. fairly semplicistic and I do not plan to change it: the tutorial is about all but coding: tests, documentation, distribution and revision control are the points of this guide and I tried to keep everything as small as possible. If you really cannot resist to rewrite the code of the module, rewrite it all and I can add a TIMTOWTDI section, just for amusement.

By other hand the day eight: other module techniques has room for improvements and additions: if you want to share your own tecquiques about testing, makefile hacking, automating distribution I think this is the place. I choosed module-starter to sketch out the module as it seemed to me simple and complete, but it has some quirks. Other tools examples can be worth another day of tutorial, but keep it simple.

When you have commented this tutorial I'll remove the [RFC] in the title and I'll point newcomers to this guide (or better if it will be reposted in other section?), if you judged it is worth to read.

Thanks!

L*

UPDATE 20 Dec. Added a readmore tag around the below content. The online repository is receiving some pull requests ;) so I added a version number to the doc. Tux is very busy but pointed me to Release::Checklist and I'l add it to the tutorial.

https://www.codementor.io/sumit12dec/python-tricks-for-beginners-du107t193, '10 Neat Python Tricks' is the title of this link. My perl solutions to these tricks can be seen below. Interestingly in many cases perl solution looks more obvious than python solution. Any comment is welcome.

Update: Confusingly edited IX and X are corrected.

Update 2: X fixed

Update 3: I updated

Probably most people know about the "Eskimo greeting" "secret operator". I'm not sure if the following trick is common knowledge, but I just saw it for the first time in this blog post by Yary:

$ perl -MO=Deparse -M'5;print "foo"' -ne '}{print "bar"'

sub BEGIN {
    require 5;
    ()
}
print 'foo';
LINE: while (defined($_ = readline ARGV)) {
    ();
}
{
    print 'bar';
}
[download]

A neat little trick :-)

If we have two sets, it is considered a solved problem to get their union or intersection or symmetric difference, there is even an entry in perlfaq4 about it. The situation is slightly more complicated if we have more than two input sets, because then it's a valid question to ask to e.g. get the set of elements that are in the first or second set but not in the third and fourth etc. The number of combinations grows rapidly with the number of input sets, and just writing ad hoc solutions to each little problem becomes infeasible. So a more general solution is needed - the hard part is designing the user interface so that it is able to express all the possible combinations of selections in a general and flexible, yet efficient and understandable manner. A cursory search of CPAN brought up several (abandoned?) modules in the Set::* namespace, but none of them was exactly what I needed.

I have the outline of an attempted solution. It's an OO module that has a constructor to which the input sets can be fed, and one method called select, which accepts a selector string argument and emits (an arrayref of) the elements that match the selector. If we have 3 input sets, then the '110' selector string selects all elements that are in the first and second sets but not in the third.

#!/usr/bin/perl

{
package Set::Select;

use strict;
use warnings;

sub new {
    my ($class, $args, @sets) = @_;
    my $attr;
    $attr = $args->{key} if (ref $args eq 'HASH' and exists $args->{ke
+y});
    my $self;
    for my $i (0..$#sets) {
        for my $elem (@{$sets[$i]}) {
            my $key = defined $attr && ref $elem eq 'HASH' ? $elem->{$
+attr} : $elem;
            $self->{$key}->[1] //= $elem;
            $self->{$key}->[0] //= '0' x @sets;
            vec($self->{$key}->[0], $i, 8) = 0x31;
        }
    }
    bless $self, $class;
}

sub select {
    my ($self, $bits) = @_;
    return [map { $self->{$_}->[1] } grep { $self->{$_}->[0] =~ $bits 
+} keys %$self];
}

}

package main;

use strict;
use warnings;
use Data::Dumper;

my $x = Set::Select->new({}, [1, 3, 5, 7], [2, 3, 6, 7], [4, 5, 6, 7])
+;

print Dumper $x->select($_) for qw/100 101 111 10. ... /;

my $y = Set::Select->new({key => 'id' },
    [{id => 1, value => 1}, {id => 3, value => 1}, {id => 5, value => 
+1}, {id => 7, value => 1}],
    [{id => 2, value => 2}, {id => 3, value => 2}, {id => 6, value => 
+2}, {id => 7, value => 2}],
    [{id => 4, value => 3}, {id => 5, value => 3}, {id => 6, value => 
+3}, {id => 7, value => 3}],
);

print Dumper $y->select($_) for qw/100 101 111 10. ... /;
[download]

A Venn diagram that may or may not make the intent clearer:

         .---.
        /  1  \
       |       |
    .--+--. .--+--.
   /   | 3 X 5 |   \
  |    |  / \  |    |
  |  2  \/ 7 \/  4  |
  |     |`---'|     |
  |      \ 6 /      |
   \      \ /      /
    `------^------'

I think these selector strings as the primary (and only) user interface are better than the possible alternatives that come to mind: a verbose, ad hoc query language would have to be explained at length in the documention, tested carefully in the source, and parsed painfully at runtime, while a forest of arbitrarily named methods to select this or that subset would bloat the code needlessly and make it harder to use.

Using regular expressions opens the door to abuse, but it also allows convenient and terse selector strings, makes the implementation efficient, and it's something people already know.

If the elements are hashrefs (representing a record or object or something), there is a mode to use not the elements themselves but a named key inside them as the basis of selection, as the second example shows. This mode can be considered buggy as it is now, because only one version of a record with the same key is stored (in the example, some values are discarded. I don't have a good solution for this problem yet, partly because it would make the implementation slower and more complicated, partly because I don't know what would be the right thing to do.

Questions:

• Is this useful to anyone?
• How to make it better?
• What would be a good name if this were to become a module? I've tentatively chosen Set::Select but it may be too generic.

I've been using Perl for several years mostly for small to medium sized programs of sysadmim type (automation, gluing, data transformation, log searching). Recently I started to learn Go. I wanted to write something in both languages and compare. Here goes.

The Perl code is more than 2 times smaller:

$ ls -l x.* | perl -lanE 'say "$F[8]\t$F[4] bytes"'
x.go    694 bytes
x.pl    294 bytes
[download]

Perl code is more than 4 times slower when run ...

$ time go run x.go > /dev/null 

real    0m1.222s
user    0m1.097s
sys    0m0.220s

$ time perl x.pl > /dev/null 

real    0m5.358s
user    0m4.778s
sys    0m0.497s
[download]

... and more than 5 times slower when I built the Go code:

$ go build x.go
$ time ./x > /dev/null 

real    0m0.947s
user    0m0.890s
sys    0m0.126s
[download]

The code generates 10 million random integers from 0 to 9. Than it counts the occurrence of each generated integer and prints it.

$ cat x.go
package main

import (
    "fmt"
    "math/rand"
    "time"
)

func main() {
    // Seed the random number generator
    seed := rand.NewSource(time.Now().UnixNano())
    r1 := rand.New(seed)

    // Generate random integers
    var ints []int
    for i := 0; i < 10000000; i++ {
        n := r1.Intn(10)
        ints = append(ints, n)
    }

    // Count ints occurrence
    count := make(map[int]int)
    for _, n := range ints {
        count[n]++
    }

    // Sort ints
    var intsSorted []int
    for n := range count {
        intsSorted = append(intsSorted, n)
    }

    // Print out ints occurrence
    for n := range intsSorted {
        fmt.Printf("%d\t%d\n", n, count[n])
    }
}

$ cat x.pl
#!/usr/bin/perl
use warnings;
use strict;

# Generate random integers
my @ints;
push @ints, int rand 10 for 1 .. 10_000_000;

# Count ints occurrence
my %count;
$count{$_}++ for @ints;

# Print out ints occurrence
for my $int ( sort keys %count ) {
    printf "%d\t%d\n", $int, $count{$int};
}
[download]

In conclusion I must say that I like both languages. I like beer too :-).

Say, for example, I get eaten by a bear, roll my truck off into the lake, get crushed by falling mountain boulders or otherwise burn in a fire, I'm wondering what will happen with my CPAN distributions.

I mean, I won't care if I'm dead, but if they are being used (from what I can tell, they are in a 'minimalistic' sense (I don't know numbers)), what happens? By default after years of trying to understand, they'll fall into a state of disrepair and then go through the normal channel of adaption.

I'm wondering what my fellow Monks think about this.

My question here, is would it be worth working it up the chain to have a "will" of sorts; someone you could "dedicate" your distributions to, within the Makefile (or whatever dist thingy one uses). A new attribute, effective across all build platforms and accepted by CPAN, that acknowledges who you want to oversee what you've written.

I'm not talking about co-auth here. I'm talking about someone who may not even care about one's work. I'm talking about someone who cares about Perl enough that one would feel comfortable with rightfully distributing one's distributions accordingly, because they are somewhat familiar with the Perl ecosystem.

This is totally off the wall, but I've been through so much in the last 24 months, that I'm trying to think of everything.

Would a IF_I_DIE flag within a Makefile.PL that is easily searchable be a good idea, or an idea of a madman who keeps buying sensors to write Perl around?

tired of coding print-statements for debugging purposes again and again while being too lazy to use the debugger I recently wrote a module Debug::Filter::PrintExpr that - acting as a Perl filter - transforms some special comments into debugging print statements. As an example, this piece of code

use Debug::Filter::PrintExpr;
...
my $some_var = 'some content';
my @array = qw(this is an array);
my %hash = (key1 => 'value1', key2 => 'value2');

#${$some_var}
#@{@array}
#${$array[3]}
#%{custom_label: %hash}
[download]

line 28: $some_var = 'some content'
line 29: @array = ('this', 'is', 'an', 'array')
line 30: $array[3] = 'array'
custom_label: %hash = ('key2' => 'value2', 'key1' => 'value1')
[download]

Hello Monks

I've been deeply ensconced in getting some relatively-minor and low-level details squared away, but I find it hugely technical and unpredictable as I try to grasp what is happening at my terminal. In order to share it properly, it has to be pasted into a text document. Under such circumstances, I like having pairs of code tags at the ready. One can edit them later on. I seem to do better when I start the write-up early, especially with mystifying and hard-to-replicate output.

Likewise, a handful of p tags are necessary, and why not line them all up in shiny rows, so that we can see their completeness and symmetry. A short interview with this program gets you the tags you need. How many times have you created the tags from scratch and bolloxed it up? Machines can do it better.

Output then source:

The output doesn't show on perlmonks. It does in the file that is formed from the munged time. This becomes the prep file for the write-up. Tags are arrayed in order, with nothing between open and close tags, so that's hardly surprising. I wanted to write this utility before I did anything more, to turn my character defect of laziness into something keystroke-saving in the long haul.

The source file for Text::Template object is:

$ cat 1.monk.tmpl
<{$symbol}></{$symbol}>
$ 
[download]

I realize that there are much smaller ways to do what I have achieved here and would solicit them. I believe that Path::Tiny will create the directories it needs with the touch method. My pride will not be hurt to see this shortened by great lengths. This isn't going to be the only thing I use Text::Template for in the near term, so I wanted to roll it out with the full framework of Path::Tiny.

Are there any projects officially making an attempt at this?

I'm getting bored of notating different docs between the two markup/down languages. I do have a couple of translation scripts, but they aren't even public as they are inconsistent and often the resulting doc requires some manual work.

If there are worthy near-prod-consistent efforts, I'll join and help. If not, I'll create something that the community can assist with.

Hi Monastery !

I was reading a post about the future of Perl. I don't know if it is the most appropriate place to provide this feedback. I've seen the statistics in TIOBE on how Perl is loosing places in the rank of the most performed languages. I freshly finished my IT studies and i discovered Perl language now in my company. I wanted to raise the fact that the rank of the language doesn't matter at all. The most important thing is the maintain activity. Why ? Because by default , Perl is a more powerful language ( not only according to me ) than Python or Ruby or others. Perl is not so friendly let's say or common but Perl is doing the job better, because the Perl community is actively "better" and because the language has in his own genetic heritage the basis to be essential. The black dot is coming of lobbying which is not serving Perl interest. And how the language is taught at school. As we are in a monastery, i would like to suggest this precept: If someone need to go somewhere he will need oxygen. Perl is like oxygen ( in the air ) and there is only one thing to do , making this oxygen available ( meaning having a strong Perl community ). And another black dot and not the least: the availability of Perl in growning country. I had discussion with some foreigners, IT engineers and software developers. As suprising as it might seem, some of them don't even know what is Perl language !! It could be difficult to teach Perl in Universities in Western countries from a political point of view due to these lobbies and pressures but it could be different in growning countries. It's important to make partnerships with them ( with their Univeristies and so on..) A good idea ,maybe, would be to organize the Perl conference or any big related Perl event in one of these countries.

Thanks for reading and the sorry for the spam. I wanted to share my feeling

PHP, Python, Ruby, Javascript all start by copying Perl's worst practices, according to computer scientists, to become immensely popular, with no strict and globals everywhere.

But the fun never lasts because they eventually succumb to aspersions of computer scientists to add all sorts of cruft to enforce austerities that satisfy obsessively compartmentalized minds.

I think the reason is this: Languages like Perl force computers to think like people, rather than forcing people to think like computers.

Don't get me wrong, we need the scientists to build and maintain the playgound so we can play, but we also need them to get the heck out of our way, and to stay away!

Hard Fork Perl with a trendy cool name and make sure the batteries are included by throwing in a kitchen sink of about 1000 of the most awesome CPAN modules in a way that will do everything and run everywhere and you may have (another) winner.

Perl 6, seriously, this is sad:

$input.close or die $!;
close($output);

I'm proposing we try out the Five Whys process at work. The primary goal is to determine the root cause of a problem by repeating the question "Why?". Each answer forms the basis of the next question. The "Five" should not be taken literally, keep asking why until you uncover the root cause.

The method provides no hard and fast rules about what lines of questions to explore, or how long to continue the search for additional root causes. Even when the method is closely followed, the outcome still depends upon the knowledge and persistence of the people involved.

It's important to note that the purpose of Five Whys isn't to place blame, but to uncover the root cause of the problem. To create small, incremental steps so that the same issue doesn't happen again. Our bias as developers is to over-focus on the technical part of the problem; Five Whys tends to counteract that tendency. What started as a technical problem often turns out to be a human and process problem. Sometimes a solution may cut across several departments ... requiring the support of someone with authority to change process at that level.

I'd like to learn from your experiences with Five Whys. Has Five Whys been tried in the Perl or Open Source world? (for example, on the Perl 6 project). Have you tried it at work? How did it go? What did you learn?

Proposed Five Whys Process

As advised by Eric Ries (see References below), at least for a trial period, all Five Whys meetings will be run in the same way, by the same "Five Whys Master". We'll continuously adapt our use of Five Whys, based on our experiences. Note that Five Whys is a technique for continuous improvement, and can therefore be applied to itself.

Loosely based on Ries' advice, this is the proposed Five Whys process:

• Get everyone who was involved in the problem in a room together.
• It's not a crisis, it's a learning opportunity; not to assign blame, but uncover the root cause.
• Agree on a statement of the problem; express the problem statement as a question.
• Hold a brainstorming session in which "Why?" is repeatedly asked to find the root causes. The Five Whys Master may employ a Fishbone diagram (see next section).
• Look for causes step by step. Don't jump to conclusions.
• Screen your causes by asking two questions: How likely is this cause to be the major source of the issue? V-Very Likely, S-Somewhat Likely, N-Not Likely; How easy would it be to fix or control? V-Very Easy, S-Somewhat Easy, N-Not Easy. Put the answers to the two questions together, e.g. SV.
• For each root cause, try asking "how" five times until the best solution is found ("Five hows").

Commit to make a proportional investment in corrective action at every level of the analysis. These small investments cause the team to go faster over time. Proportional means not to over-react to problems; a ground-up rewrite is not always required. Though the Five Whys Master leads the discussion, others assign responsibility for the solution to anyone in the room. The analysis should propose solutions, aka corrective actions. For each solution, ask "how will this solution prevent the error from occurring again?". Each corrective action is marked as must (mandatory), should (strongly recommended), or may (use discretion, depending on time and resources); periodically after the analysis is published, track which corrective actions have actually been implemented (this metric will be used to tune our Five Whys process). The results of the Five Whys analysis is shared with the whole company. The solution is in plain English that anyone can understand. Each Five Whys analysis is a teaching document.

Fishbone (aka Ishikawa) diagram

There may be many opinions of root cause. The fishbone visually displays many potential causes for a specific problem. Because people by nature often like to get right to determining what to do about a problem, the fishbone can help bring out a more thorough exploration of the issues behind the problem - which leads to a more robust solution. To construct a fishbone, start with stating the problem in the form of a question. Each root cause idea should answer the question.

The rest of the fishbone consists of one line drawn across the page, attached to the problem statement, and several lines, or "bones", coming out vertically from the main line. These branches are labeled with different categories; the categories are up to you to decide, for example: Four Ps (Policies, Procedures, People, Plant/Technology); Six Ms (Machines, Methods, Materials, Measurements, MotherNature, Manpower).

The team should agree on the statement of the problem and then place this question in a box at the head of the fishbone. The defect is shown as the fish's head, facing to the right, with the causes extending to the left as fishbones; the ribs branch off the backbone for major causes, with sub-branches for root-causes, to as many levels as required. One or two of these "causes" will have a greater effect than the others and will guide you to the root of the problem. This structure allows you to tackle smaller chunks which have a large impact on the problem. Looking at elements of the problem and not the whole process will likely make finding your solution less daunting and problem solving more manageable. Screen your causes as described above. Focus on causes with result VV, VS and SV.

Typical Software Problem Categories

I couldn't find much specific advice on applying Five Whys to software problems - if you know of good references, please let me know. Examples of typical software problems can be found at: Common Software Development Mistakes. As for categories of typical software problems, perhaps:

• People
• Process (e.g. Agile, Build system)
• Measurement (e.g. Estimation, Measuring rate of progress)
• Environment (e.g. Test environment)
• Machine/Materials (e.g. Slow build machines)
• Culture
• Product
• Technology

References

• Five Whys (wikipedia)
• Five Whys by Eric Ries (start with a single person as the "five whys master", email whole company with results of analysis, common question: why didn't automated test catch a bug?)
• Five Whys by Joel Spolsky
• Jeff Bezos uses Five Whys (anecdote of broken thumb in conveyor belt)
• Five Whys Process (not to assign blame but uncover the root cause. 1 invite all affected, 2. select a leader, 3. ask why five times, 4. assign responsibilities for solutions, 5. email the team the results)
• Seven Basic Tools of Quality (fishbone, checksheet, control chart, histogram, pareto chart, scatter diagram, stratification)
• Ishikawa diagram (wikipedia, fishbone diagram aka Ishikawa diagram aka Fishikawa diagram aka cause-and-effect diagram)
• Fishbone diagram (six sigma)

my $one  = 1;
my $six  =
my $nine = 9;

print $six; # prints 9

[download]

#!/usr/bin/perl -l

use strict;
use warnings;
use diagnostics;

print my $J = my $A = my $P = my $H =
'Just',' Another ','Perl',' Hacker';

[download]