Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.


by jakobi (Pilgrim)
on Sep 18, 2009 at 18:52 UTC ( [id://796180]=user: print w/replies, xml ) Need Help??

TOC bio | maintained nodes perls, dung heaps rant & pearls | FAQlets | pm-newbie: log & missing memes | mine||raw

( mbox outbox | node allowed html shortlinks | cb last hour im2 de/cloak )
( boilerplates )
    Resources to get help in Perl [Resources to get help in Perl] - selfhelp starter resources
    Debugging and Optimization see [Debugging and Optimization]
    see also my scratchpad: [jakobi's scratchpad]


Wrt computers, my main interest is most things Unix. As for perlmonks, my github archive may contain some stuff of interest, a subset of which is also on cpan and freshmeat.

I started out with perl3, rexx and g/awk, besides Assembler, C and the usual UNIX environment. But it took a royally messed up IRIX awk/nawk migration idiocy to make me leave the awkward ways of the heresy and develop an enlightened interest in Perl.

Some of my tastes are still affected by that trauma: I must confess to having a dislike for perl5 OO due to the lack of the one official default style to bind them all (see e.g. Re: use common::sense; for related issues), as well as a preference for standalone s‎crip‎ts (let me just copy this s‎crip‎t to those servers to do that...), which results in a rather high threshold for when to switch to using or writing modules. Furthermore, while I've overcome awk long ago, I still suffer from a craving for long sequences of /awkish line-noise/.


PM Nodes and Code on PM I Intend to Keep Current

{N2S: do mark&color the node pointers for these also in the FAQlets below and to pm_back}

If it's not PM-specific, you'll find most of my public stuff in the archive(s?) below With currently 90+%, that includes way too much Perl.

Bookmarks from meta-PM to pure Perl

Rants and Major Goofs

AVOID AT ALL COST: <> and perl -e

I really cannot believe my eyes:

This one turns Perl into a single major security issue that needs to be disallowed to anyone in a professional setting, users and developers alike: Don't ever use <> or perl -e: magic-diamond <> behavior -- WHAT?! and executing filenames as shell exploits even with the most trivial of filters. Which for most settings probably ought to translate into avoid Perl itself. And darn it, correctly so.

What's worse, there are actually people insisting on keeping the security hole unplugged.

Note that -i.bak renames first and seems to use the 3 argument form, so it escapes from being attacked. Note that using <> ONLY with STDIN isn't affected due to lack of filenames... . Some more bandaids are code replication, explicit use of <STDIN>, tainting with -T, or @ARGV mangling with e.g. ARGV::readonly, all of which with additional semantic changes and side-effects of their own.

And no, the issue isn't the well-known magic insecure 2-argument open (surprisingly, the 1-argument version is secure), as this is but the tiny and mostly-harmless core of the real problem. It's its unexpexted use in <> that makes an insecure mess of easy, idiomatic perl uses:

One of the shortest possible ways to loose control of your servers: perl -0pe:

Should we start to advertise awk-oneliners now, given that most Perl oneliners are too dangerous for a quick suggestion without both bandaids (larger than the original oneliner by a line or ten) and a twenty-hour lecture series on security issues? Would you trust someone of little Perl history to only ever use your shown oneliner with safe STDIN and never with unsafe filename arguments? Makes you wonder. And decide against teaching Perl to collegues.

Another goof of that order of magnitude and Perl won't have even a tatter of credibility left. Did I miss an official change of Perl's motto to Make easy things unsafe and hard things worse? Increasing problems and creating new problems wasn't what Perl was about. Now on to the damage that was done today, and on to the grep for the fallout. Just a few MB to check. (Damn right, I'm pissed. And wondering if even Perl6 is going to be capable of outliving this kind of damage).

Perfect advertisement for Perl. If you warn against using Perl for small stuff.
Otherwise, it's just compromised servers, exploits, and lost files: that's even better PR and horror stories

To be checked: are Perl6 one-liners safer? If so, this _might_ work for non-Perl-collegue-safe quick commandline hacks, provided the hack won't compile in Perl5 (or how would you avoid accidental but-I-only-changed..., just-like-you've-shown-me fatalities?).

Perl6 WIKI

Maybe I should just look at Haskell ([id://520826|tasty...) or OCAML for new stuff. Well written ML is a thing of beauty to behold.

Non-Perl Pearls

Perl FAQlets

As laziness is one of the Perl user's virtues to aspire to, I hope that I can reuse these faqlets as a resource in future answers.

{N2S: do mark&color the node pointers for owned nodes to be kept uptodate: here & add them to pm_back & the list of updated non-home nodes above}

Common Questions and Classics

Keep Scott McMahan's quote in mind when meditating on the fine points of Perl functions: Perl is the portable distillation of the UNIX philosophy. If in doubt, search the margins for the proof in Unix section 2 and 3 man pages (BSD,Linux, Solaris) as well as Posix in addition to perldoc -f.

On cleverness, there's a real nice Kernighan quote: Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
Which might be read as: So little grashopper, now carefully choose the location of the trap you want to catch yourself in.

On virtues, misconceptions and Gödel. Also meditate on the Unmentionable Virtue: Paranoia.

Keywords to read up on

Note that the links just provide an example thread for the topic, not necessarily a meditation on the topic or the authorative one and only correct answer... . Drop me a line if you want to suggest a better discussion thread or link.

  • list of filenames, find, File::Find, suitable data structures, recursion, output: Directory Tree Structure
  • when (not) to use OO: Where/When is OO useful?, esp. the issue with just-subclass vs encapsulation (->petruchio) and the idea of using a closure for the extra data. "if you can enforce constraints on the whole inheritance tree" is pretty much my pet peeve from above about the lacking of a default style to bind them all in perl5. Still hoping for perl6.
  • truth - "0 but true"
  • numeric - "string vs number" (see also: perldoc -f ref)

Debugging and Optimization

See Debugging and Optimization. I still hope for a single public node to point monks to; failing that I'll place an augmented copy of this node here, hopefully finding also the RFC threads for all of its links. A tutorial w/o RFC thread is about 70%-completed tutorial crossed with a trap and a time sink. We should do better than that, but still without drowning seekers in a vast ocean of widely dissipated partially outdated scraps of help. Especially start-off tutorials shouldn't assume seekers to be already grand masters of swimming and searching, have them train in a smaller inland lake's probably the better approach.

Don't be afraid of the foundations and the layers below Perl: For now let's just say to always first check CPAN and the module documentation, FAQs. After that use Unix basics like strace/truss or locate/find to take a peek at syscalls being used or the module source. To read up on the actual syscalls and their arguments e.g. for wait/waitpid (syscalls are more or less lightly by Perl and libc - cf. flock): grep your actual kernel's authorative definitions in /usr/include. If you can read Perl and non-trivial Perl REGEX, a mere overgrown macro-assembler like C shouldn't challenge you. And much of Perl including many of the operators _is_ the same as in C. Examples include:

Windows Survival Using Perl and Unix Commands

For generic Unix app ports to Windows or accessing Unix from Windows, check e.g. Freshmeat with suitable keywords. Be certain to include windows versions of firefox, a vnc client and putty in your hunt-list.
  • first of all, there is perl -e, even if this kind of use is crippled by quoting issues. Worse, the lack of a proper shell and standard tools is harmful for e.g. system(), open() and qx//, and you need to do the task in perl proper.
  • you can also search CPAN for ExtUtils::Command (Perl modules implement basic unix shell file commands) and Bundle::PPT (the 2002 basic Unix tool rewrite project: Perl Power Tools)
  • if you want a (non-standard) shell replacement for cmd, have a look at a Perl shell, e.g. Psh
  • when including C and a bit of disk-space, there is, which offers a Port of most GNU tools, ranging from libc over bash and sshd all the way to X11
  • see also planetscape's Re: UNIX shell commands in Windows

Notes of a new PerlMonk / Meta - Topics

While I've used Perl for ages, I only recently joined PM when being stumped by a CPAN cultural issue I'm still wondering about. This section isn't about Perl, but rather about itself. It lists both my observations as well as hard-to-find info-nuggets not listed in the main FAQs (or at least lacking the emphasis necessary to register late at night).

PM-newbie experience log

  • Step 0: find a burning question worth asking that does need a dedicated perl mailinglist or group, not being answerable with just your personal Perl skills, google, collegues, and sage or lopsa.
  • Step 1: do stand still and give up, you can no longer escape from being hooked by pm.

  • Step 2: now that I'm hooked, harvest and skim (but don't read) the meta information, adding some ear-marks all over the place. Deep breath. Condense. Jot it down on a location to find it again. Why not the profile - cannot mislay that one.

    Dear fellow victim in newbie-dom: you're welcome to a tiny peek at the first few of my private pm book marks below, which record my harvests from the FAQs and introduction nodes on perlmonks itself. But take care: There are too many of those helpful nodes, each dedicating itself to just add a nugget of detail info left and a gem over there on the top, no, more to the right, yes, that one. And they procreate like crazy: take this very section in this node... . Something like 'safer sex' seems indicated to make their fixpoint remain below the total number of pm nodes. But maybe vroom is also testing whether large numbers of memes have non-zero mass in sufficiently little volume to create a black hole.

  • Step 3: burn yourself and learn about linking, node html tags and the chatterbox. Properly burned and incensed, become a burnished monk (or something like that; hmm. that joke's en- and transcoding seems not to be lossless in English).
  • Step 4: enjoy a short temporary xp whore phase upto level 5, but remember the inertia of the xp game. Try to avoid overshooting all the way to Curate and accidentally becoming a Saint in our Book while still stumbling over community basics...
  • Step 5: introspection aka time to re-edit and extend your notes from the learning experience, both in your profile/scratchpad and in your nodes in PSoW and elsewhere (and do fix those <pre>'s). There's nothing like confusing later newbies and supporting vroom's black hole meme project.
  • Step 6: error: time-(stat(++$pm_self_step))[9] < 0

PM-newbie - the missing memes

Configuration settings and other tips

  • bookmarking nodes to revisit for a short while: As long as the list isn't growing long, the personal nodelet is nice to collect nodes for a (re)-reading-list.
  • chatterbox:
    • read but don't talk using the chatterbox nodelet on the right, as you might reload the tab and thus resend the message. Usually a few hours late, resulting in everyones' and the sender's confusion.
    • pm_chat2 is a nice chatterbox s‎crip‎t for a terminal session (apply my diff from the 3rd comment in case of ALARM not working).
      Alternatively open a small separete browser window with the simple fullscreen frame-based chat node (note that its exended cousin hates my properly secured firefox/nos‎crip‎t combination).
  • using <code> instead of the traditional <pre> is a bit painful: always download, never use the more convenient paste, as the darn line breaks have to add a + for extra damage (consider trying to copy/paste diff -u output). Until I noticed the display setting: code wrapping off in the settings. Joy! & kudos to ww.
  • more linebreaking pain: the textarea input for comments and new nodes is quite small, sometimes even just 60 chars... . Fix this by dropping some css in the settings::display_settings node: textarea { width: 90%; height: 500px; } Use the signature settings node to preseed the textarea of a new node, e.g. with boilerplates / reply templates.
  • <table border=2>... works.
    block-markup a point you make with <small>, <blockquote>, or <strike> (to "delete"-but-keep obsoleted content as part of a node),
    grey background with class=readmore (e.g. in <div>), or
    blue by default with class=settings_key (in <td> at least; though note that the commen dark theme css scrap omits providing sane settings :( )
    font colors also work (e.g. to mess up some monks' coloring theme), as do
    bgcolor attributes (in <td> at least)
  • this CSS view of perlmonks requires adding some missing links to the free nodelet, but might offer faster access.
  • free-nodelet example with providing some missing links and searches opening to new tab/window. An older version of the node looks more or less like this:


    me pub prv mb ob cb cbh rat
    <small> <form method="post" action="?" enctype="application/x-www-form-urlenco +ded" target=_blank> <tt>[Super Search|super]</tt> <input type="text" name="BIT" size=15 /> <input type="hidden" name="node_id" value="3989"> <!--search not preseed--> <input type="hidden" name="go" value="Search"> </form><br> <form method="get" action="" target=_blank> <tt>[cpan://|cpan&nbsp;]</tt> <input type=hidden name=mode value=all> <input type=text name=query size=15> </form><br> <form method="get" action="" target +=_blank> <tt>[doc://|pdocs]</tt> <input type=text name=q size=15> </form><br> <form method="get" action="" + target=_blank> <input type=hidden name=hl value=en> <input type=hidden name=num value=100> <input type="text" name="q" value=" "> </form> <br> [...|me] [...'s scratchpad|pub] <a href="?viewmode=private;node=...'s scratchpad">prv</a> [Message Inbox|mb] [Message Outbox|ob] [FullPage Chat|cb] [id://596792|cbh] [Recently Active Threads|rat] <br> <a href="">pause</a> <br> </small>
  • know enough about PM-site structure and document types to not waste pedagogue or pmdev clan time due to trivial misunderstandings and invalid assumptions. Which I still do. So these seem not to be sufficient for that worthy goal: Pmdev documentation, jdporter's site scratchpad, Editing DocLists, Restyling PerlMonks, Inner S‎crip‎torium. And then there's also that tantalizing Everything hint, some remains of which are at in CVS, and on
  • offline editing of perlmonks nodes: Consider a firefox plugin for using an external editor for the textarea fields.
    Or check this thread for a perl/tk perlmonks editor or jdporters editor wrapper. I've slightly modified it and it works like a charm with vi or whatever you specify as your $EDITOR: search for "unix strain", "console" or the alias I'm using for it myself: pm_vi.

A visit to the Monastery

On "stupid" Questions and hidden Gems

Stupid questions - assuming you stick to thread to to reply to questions and had a go at the documentation - don't really exist. Stupidly posed questions do exist: lack of use strict; use warnings;; horrible formatting of code or node; lack of input; lack of output; lack of stating what you've already read or attempted; providing code that is overly long or just does neither parse nor execute. But just about any question might result in answers that are true gems in their own right.

Thus there's no reason to NOT ask your system or Perl an interesting question (see above), nor is there any reason to NOT ask your fellow monks. If wondering whether a question is a troll, maybe wait a bit with a reply. But then, if the topic can be misdirected into more interesting waters, I'd suggest to leave some food as offering for trolls and fellow monks (do check out the link to audreyt's border-collie-style handling of hapless trolls in the 2nd PS).

Participation in Common Monk Activity is Obligatory

or is it? Anyway, you know that you spend too much time on PM when you <insert the usual>:

JAPH01> perl -MPOSIX -e 'sub _{($.,$/)=@_;eval"\$\\.=sprintf\"\%s\",\"".(grep{s/\S/do{(ord($&)<42+4*2**0.42)?chr((42\/3+1)<<3):$&}/ge,42}grep{s%(?!$)%do{$//=2;"\\\/".(($/=~s/\.\S*//)?ceil sqrt 42:int sqrt 42)}%ge,42}sprintf("%x",$.))[0]." \""};@_=(42292,14,32458834,72,58668,5,8469330,32);while(@_){_@_;@_=@_[2..$#_]};print'|tr UNIX~ REWOP

PM-related Shortcuts (private)

Bookmarks (private where-did-I-see-that-avoidance section)

Not yet sorted PM Bookmarks

should go here on being moved from the end of jakobi's scratchpad

to read: some example threads are listed with each topic
incomplete as it probably is, google seems to see the whole thread pages while supersearch sees nodes separately. - by date, but dynamics like search is mostly fail

Log In?

What's my password?
Create A New User
Domain Nodelet?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (6)
As of 2024-05-28 13:25 GMT
Find Nodes?
    Voting Booth?

    No recent polls found