It's that day again, my Monk Day. I'm up to 23 years on this site. Cool!

I had an idea recently that I wanted to scan the log files on the server I run to catch information about the life-cycle of customer tickets. I've written various scripts that run under crontab to create and to update a Freshdesk ticket.

As each of the scripts run, it uses the very handy Log::Log4perl module to log stuff, and they end up in log files which get rotated (so foo.log becomes foo.log.1) automatically. How big the files are, and how many backups you have are all configurable, of course. The logs contain lines like

2024/12/06 06:11:03 INFO : Create FD ticket 427993 for order 663363 ..
+ OK
[download]

Update ticket 413229 to add invoice 802924 tag .. OK
[download]

Of course! Named captures! (I wrote about this in Perl's hidden depths). I happily got the code working, and proudly pasted my clever solution here. And I got some feedback about how I could simplify my code. And then simplify it some more .. wow.

Clearly, I am *still* learning Perl, after 25+ years. That's how deep the language is, and that's how generous the Perl community is with sharing its knowledge.

"Nobody uses Perl anymore!!" Yeah, well, they should. it's an awesome language.

Recently I got myself a new phone. And, as one does, I installed Termux on it, to have a unix-ish environment for when I need it.

Then, of course, I went to configure Perl and install some modules I'd like there. As this was not my first time doing so, I thought about listing all the modules I want in a cpanfile and then using App::cpanminus to install these, making the process far more reproducible.

But that required some non-Perl prerequisites, like the C compiler, make and perl itself. And of course, also bootstrapping App::cpanminus.

Then I thought a bit on how to combine this setup into a single file. This is what I came up with. It's a cpanfile that doubles as shell script.

#!/bin/bash

# This will install cpanminus and then the Perl modules
# as listed at the end of this script. To run use
# the following commands:
#
# chmod ugo+x ./cpanfile && ./cpanfile
#

eval '
pkg install git perl make clang

if ! cpanm --help 2>&1 >/dev/null ; then
    curl -L https://cpanmin.us | perl - App::cpanminus
fi

DIR=$(dirname "$0")
cpanm --installdeps --notest --cpanfile "$DIR"

exit
'
if 0;

requires 'Mojolicious';
requires 'DBIx::Spreadsheet';
#requires 'App::sqldisplay'; # to be released on CPAN
requires 'DBIx::RunSQL';
[download]

Why not the other way around? Because I could not find a way to make cpanm take a file with a different name than cpanfile :)

The cpanfile is also available on Github, if you want to copy it.

Greetings from chilly, snowy Buffalo (NY, USA). I spent about 6 years doing no programming at all and now I'm using perl again. I am still hooked on it. So this week I started exploring CPAN modules that give info about and, in some cases, modify, the filesystem. I know of 3 modules that are used for that: the core module File::Spec, the module Path::Class (used it and liked it, a while back), and the module Path::Tiny. This meditation focuses on Path::Tiny, which seems to be stalking me. I see a lot of module authors using it in their code (that is, it has become a common dependency).

Path::Tiny is probably a bit of a misnomer. It has many methods. Many many. What I was most interested in this week was the method tempdir. Something about how tempdir works took me by surprise (yes, I am finally getting around to the point of this meditation :). The method returns a {normalized, right-slashed, absolute} pathname, but it also creates the temporary directory. Maybe I am odd, but I expected to be given a string and then create the directory with that pathname myself!

Below, some code (playful deliberately) that uses tempdir and will demonstrate that when tempdir is called, a directory is created.

#!/usr/bin/env perl
# Last modified: Wed Dec 04 2024 10:48:30 PM -05:00 [EST]

use strict;
use v5.18;
use utf8;
use warnings;

=head1 NAME
pathology.pl

=cut

use Term::ReadKey qw/ReadMode ReadKey/;
use File::Spec;
use Path::Tiny;

my $user;

sub versions
{
   say
       $File::Spec::VERSION, q[ ]
   ,   $Path::Tiny::VERSION ;
}

sub who
{
    no warnings 'deprecated', 'experimental';
    use Config;
    given (lc( $Config{osname} )) {
        when ('linux')    { $user = $ENV{USER}     }
        when ('cygwin')   { $user = $ENV{USER}     }
        when ('mswin32')  { $user = $ENV{USERNAME} }
        default           { say "osname is something we don't know, " 
+,
                                "we'll guess it's something Unixy, so 
+" ,
                                "let's say that user name is " ,
                               getlogin || "null";
                            $user = getlogin;
                          }
    }
    return $user;
}
# ------------------------------------------------------------- #
# For all its wealth of methods, I found no direct equivalent of
# "tmpdir" in Path::Tiny, so I use File::Spec's.
say "We use " , File::Spec->tmpdir , " for temporary files.";
# ------------------------------------------------------------- #
say "We are currently in directory " , Path::Tiny->cwd;
say "Our filesystem is rooted at " , Path::Tiny->rootdir;
say "Aha, " , who() , ", we may have a temp dir for you!";
my  $tadah = Path::Tiny->tempdir(TEMPLATE => "${user}-XXXXXXXX");
say "Maybe we have made a temp directory at " , $tadah , ", let's see:
+";

if ( -e $tadah and -d $tadah ) {
    say "'$tadah' already exists and is a directory.";
    say "Type 'y' if you wish to remove this directory:";
    ReadMode 'cbreak';
    my $reply = ReadKey(0);
    ReadMode 'normal';

    if (lc $reply eq "y") {
        print "Ok, we are going to attempt to remove it now ..." ;
        rmdir($tadah) and say "Success."
                       or say "BAH! Could not remove it, reason: $!";
    } else {
        say "Ok, leaving $tadah alone.";
    }
} else {
    mkdir($tadah => 0777) and say "created temp dir."
                           or die "We couldn't make a directory \"$tad
+ah\"", $!;
    sleep 6;
    rmdir($tadah) and say "$tadah removed."
}
[download]

         Rate   pp   xs
pp  3159521/s   -- -73%
xs 11612872/s 268%   --
      Rate    pp    xs   xs2
pp   333/s    --  -72%  -93%
xs  1192/s  258%    --  -74%
xs2 4516/s 1255%  279%    --
[download]

BEGIN {
    sub APICONTROL_CC_STD    () { 0 }
    sub APICONTROL_CC_C    () { 1 }
}
sub calltype_to_num {
    my $type = shift;

    if (!$type || $type eq "__stdcall" || $type eq "WINAPI" || $type e
+q "NTAPI"
        || $type eq "CALLBACK"  ) {
        return APICONTROL_CC_STD;
    }
    elsif ($type eq "_cdecl" || $type eq "__cdecl" || $type eq "WINAPI
+V") {
        return APICONTROL_CC_C;
    }
    else {
        warn "unknown calling convention: '$type'";
        return APICONTROL_CC_STD;
    }
}
[download]

I32
calltype_to_num_xs(type)
    SV* type
PREINIT:
    const char * p;
    I32 l;
CODE:
    SvGETMAGIC(type);
    if(!SvPOK(type)) {
        if(!SvOK(type) || (SvIOK(type) && !SvIVX(type)) || !sv_true(ty
+pe)) {
            RETVAL = APICONTROL_CC_STD;
        } else {
            unk:
            warn("unknown calling convention: '" SVf "'", type);
            RETVAL = APICONTROL_CC_STD;
        }
    } else {
        p = SvPVX(type);
        l = (U32)SvCUR(type);
        switch(l) {
            case STRLENs(""):
                if(memEQs(p,l,"")){RETVAL = APICONTROL_CC_STD;break;}
                else goto unk;
            case STRLENs("CDECL"):
                if(memEQs(p,l,"CDECL")){RETVAL = APICONTROL_CC_C;break
+;}
                else if(memEQs(p,l,"NTAPI")){RETVAL = APICONTROL_CC_ST
+D;break;}
                else if(memEQs(p,l,"cdecl")){RETVAL = APICONTROL_CC_C;
+break;}
                else goto unk;
            case STRLENs("PASCAL"):
                if(memEQs(p,l,"PASCAL")){RETVAL = APICONTROL_CC_STD;br
+eak;}
                else if(memEQs(p,l,"WINAPI")){RETVAL = APICONTROL_CC_S
+TD;break;}
                else if(memEQs(p,l,"WMIAPI")){RETVAL = APICONTROL_CC_S
+TD;break;}
                else if(memEQs(p,l,"pascal")){RETVAL = APICONTROL_CC_S
+TD;break;}
                else if(memEQs(p,l,"_cdecl")){RETVAL = APICONTROL_CC_C
+;break;}
                else goto unk;
            case STRLENs("WINAPIV"):
                if(memEQs(p,l,"WINAPIV")){RETVAL = APICONTROL_CC_C;bre
+ak;}
                else if(memEQs(p,l,"__cdecl")){RETVAL = APICONTROL_CC_
+C;break;}
                else goto unk;
            case STRLENs("APIENTRY"):
                if(memEQs(p,l,"APIENTRY")){RETVAL = APICONTROL_CC_STD;
+break;}
                else if(memEQs(p,l,"CALLBACK")){RETVAL = APICONTROL_CC
+_STD;break;}
                else if(memEQs(p,l,"IMAGEAPI")){RETVAL = APICONTROL_CC
+_STD;break;}
                else goto unk;
            case STRLENs("__CRTDECL"):
                if(memEQs(p,l,"__CRTDECL")){RETVAL = APICONTROL_CC_C;b
+reak;}
                else if(memEQs(p,l,"__stdcall")){RETVAL = APICONTROL_C
+C_STD;break;}
                else goto unk;
            case STRLENs("__fastcall"):
                if(memEQs(p,l,"__fastcall")){goto unk;RETVAL = APICONT
+ROL_CC_FC;break;}
                else if(memEQs(p,l,"__thiscall")){goto unk;RETVAL = AP
+ICONTROL_CC_TC;break;}
                else if(memEQs(p,l,"APIPRIVATE")){RETVAL = APICONTROL_
+CC_STD;break;}
                else goto unk;
            case STRLENs("__vectorcall"):
                if(memEQs(p,l,"__vectorcall")){goto unk;RETVAL = APICO
+NTROL_CC_VC;break;}
                else goto unk;
            case STRLENs("STDAPICALLTYPE"):
                if(memEQs(p,l,"STDAPICALLTYPE")){RETVAL = APICONTROL_C
+C_STD;break;}
                else goto unk;
            case STRLENs("STDAPIVCALLTYPE"):
                if(memEQs(p,l,"STDAPIVCALLTYPE")){RETVAL = APICONTROL_
+CC_C;break;}
                else goto unk;
            case STRLENs("STDMETHODCALLTYPE"):
                if(memEQs(p,l,"STDMETHODCALLTYPE")){RETVAL = APICONTRO
+L_CC_STD;break;}
                else goto unk;
            case STRLENs("STDMETHODVCALLTYPE"):
                if(memEQs(p,l,"STDMETHODVCALLTYPE")){RETVAL = APICONTR
+OL_CC_C;break;}
                else goto unk;
            default: goto unk;
        }
    }
OUTPUT:
    RETVAL

void
calltype_to_num_xs2(intype)
INPUT:
    SV* intype
PREINIT:
    SV* sv = sv_2mortal(newSVpvs("__stdcall"));
    int i;
PPCODE:
    SP = &(ST(-1));
    for(i=0;i<1000;i++) {
        PUSHMARK(SP);
        PUSHs(sv);
        PUTBACK;
        XS_Local__C_calltype_to_num_xs(aTHX_ cv);
        SPAGAIN;
        SP = &(ST(-1));
    }
    PUTBACK;
[download]

use Local::C;
use Benchmark qw(cmpthese :hireswallclock);

{   my ($pp, $xs, $xs2, $cctype) = (\&Local::C::calltype_to_num, \&Loc
+al::C::calltype_to_num_xs, 
\&Local::C::calltype_to_num_xs2);
    cmpthese( -1, {
        pp => sub{&$pp('__stdcall');}, 
        xs => sub{&$xs('__stdcall');}
    });
    cmpthese( -1, {
        pp => sub{&$pp('__stdcall') for(0..10000);}, 
    xs => sub{&$xs('__stdcall') for(0..10000);},
    xs2 => sub{&$xs2('__stdcall') for(0..10);}
    });
exit;
}
[download]

I'm semi-retired, which means I take care of a client's system of Perl scripts that mostly run without my intervention. I log everything with the excellent Log::Log4perl module, and sometimes I tail those files to keep on eye on the various scripts that run. One group of scripts creates tickets for new orders, and other scripts update these tickets based on what Sage (the accounting system) says.

Eventually, I started to think about understanding the life-cycle of these tickets -- they get created (that's logged in one file), they get updated (logged in a couple of other files), and they get closed (logged in two other files). Could I parse all of the log files and see the life-cycle just by drawing inferences? It's an academic exercise, since all I have to do is query the ticketing system's API about the history of a ticket, but like I said, I'm mostly retired, but I'm still curious.

The lines are like this:

2024/11/28 10:54:04 INFO : Update ticket 425955 to add invoice 802436 
+tag .. OK
2024/11/28 10:54:05 INFO : Update ticket 425912 to add invoice 802435 
+tag .. OK
2024/11/28 10:54:06 INFO : Add note to ticket 425912 with info about i
+nvoice 802435 .. OK
2024/11/28 10:57:02 INFO : Create FD ticket 425991 for order 662626 ..
+ OK
[download]

Eventually, some of my brain cells told me I needed to use a named capture in the regular expressions to handle this. Other brain cells complained that I'd never used that before, but the first group of brain cells said, Nonsense (or Buck Up, I forget), it's all in the Camel if you just look.

So, when you're capturing stuff in a regexp with a clause like (\d+), that first capture just gets stashed in $1. But you can also name that capture (a feature I never needed until now), like this: (?<ticket>\d+). And you get it out by looking at the magic variable %+, so the ticket value is available as $+{ ticket }. SO COOL!

I was then able to write a bunch of regular expressions, all with named captures, and collect whatever I needed from the log lines. Then, if a particular element was there, I would add it to the history hash I was building. So one of the AoH entries looked like this:

{
  filename => 'status.log',
  regexp   => qr/Update (?<ticket>\d+) status to (?<status>.+) \.\./,
  action   => 'update'
},
[download]

$history{ $+{ ticket } }{ $entry->{ action } } = {
                                     date  => $words[0],
                                    'time' => $words[1],
  ( exists ( $+{ order } ) ? (       order => $+{ order } ) : () ),
  ( exists ( $+{ invoice } ) ? (   invoice => $+{ invoice } ) : () ),
  ( exists ( $+{ shipment } ) ? ( shipment => $+{ shipment } ) : () ),
  ( exists ( $+{ scheduled_date } ) ? (
                            scheduled_date => $+{ scheduled_date } ) :
+ () ),
  ( exists ( $+{ status } ) ? (     status => $+{ status } ) : () ),
};
[download]

The code runs fine, and does what I expect. Named captures are a very cool feature, but they do exactly what I needed to do. Props to all the smart folks who came up with that idea (and then implemented it). What a cool language.

yesterday I was trying some perl code after neverending boring weeks (workwise..) and I tried to use -s to add rudimentary switch parsing as described here (emphasis added by me):

> -s enables rudimentary switch parsing for switches on the command line after the program name but before any filename arguments (or before an argument of --).

The above is true when there is a real file containing a perl program:

echo print join ' ', $0, $R > switch-test.pl

# after the program name
perl -s switch-test.pl  -R=OK
switch-test.pl OK

# before an argument of --
perl -s switch-test.pl  -R=OK --
switch-test.pl OK

# wrong usage
perl -s switch-test.pl -- -R=OK
switch-test.pl
[download]

But what if the program is just some code passed with -e (and this is a valid program also for the program name perl's internal value)?

Even with my limited English understanding, I'd suppose something like: perl -s -e "print $R" R=a or perl -s -e -n "print $R" R=a filename (before any filename arguments) but no, it works exactly in the opposite way:

# after the program name
perl -s -e "print join ' ', $0, $R"  -R=OK
Unrecognized switch: -R=OK  (-h will show valid options).

# before an argument of --
perl -s -e "print join ' ', $0, $R"  -R=OK --
Unrecognized switch: -R=OK  (-h will show valid options).

# it works in the opposite way of what perlrun states
perl -s -e "print join ' ', $0, $R" -- -R=OK
-e OK
[download]

So, given the switch is stated as rudimentary one should expect to use with small oneliners (only our dead pope used it as signature..) and not for serious programs.

In combination with other switches is even unfriendly:

perl -snwe "print join ' ', $0, $R, $/" -- -R=OK lorem.txt

-e OK
-e OK
-e OK
-e OK
[download]

..as it shuold be put before any file name but after the --

The docs also warns about warnings:

> Also, when using this option on a script with warnings enabled you may get a lot of spurious "used only once" warnings.

..but no... or.. what happens??

perl -wse  "print join ' ', $0, $R" -- -R=OK
-e OK

perl -swe  "print join ' ', $0, $R" -- -R=OK
-e OK

perl -sew  "print join ' ', $0, $R" -- -R=OK

#what the hell?! ..no output :)
[download]

I think the doc should metion the above. More: in this sight also -- is poorly documented.

Want some monk put a pr somewhere or point me, lazy peon, to the correct repository to strike?

Have a nice weekend!

L*

What's happening with the Cygwin project?

I recall recently seeing a remark (on a node I cannot find now) wherein the monk asked "is Cygwin still supported?" That's a good question. Certainly cygwinPerl seem to be alive and current on the Cygwin download servers (at the time of this writing, v5.40.0). But a mechanism for asking questions about Cygwin in general is seemingly problematic and has been for a while. The Cygwin website is completely out of date, directing users to mailing lists that do not exist anymore. Apparently to reach Cygwin developers one must use NNTP (we're talking old school here).

The Cygwin.com site says: "Please note that the gmane website and its newsgroup search interface is down since August 2016. Only the aforementioned NNTP gateway is still up."

I see a fair amount of traffic in Cygwin questions on StackOverflow and its related sites, and if someone were to ask me where to get general help with Cygwin today, that's where I would direct them.

The Wordpress and Mullenweg story popped up recently, and then just this morning, I read about Perforce and Puppet, where it sounds like Perforce took over the Slack community of Puppet.

The final comment in the Mastodon post was on point:

.. has something happened behind the scenes in tech that I don't know about? Because everybody seems to be acting proper weird lately.

I would like to show how I am creating printer-ready documents using LaTeX templates from within Perl with LaTeX::Easy::Templates which I have recently published. I will illustrate this introduction with preparing pdf (envelope) labels since a question (Printing Labels) about this was asked earlier.

Latex offers to the People easy access to professional typography producing aesthetically superb documents. And to the hacker the means to do the same programmatically and automatically, with zero mouse-clicking and menu selecting. Remember, once Latex was the best (OK IMO) alternative to the evil, closed-standard, M$'s pitiful attempts to writing software. Gosh that ugly font!

Perl users have often inlined latex in their scripts, embeded with data and then shelling out (or using LaTeX::Driver) in order to call latex to produce the final typeset. The same way HTML was produced back in the cavas [sic]! :)

Latex templates, similar to HTML templates, achieve separation of Model, View & Controller. Additionally, the particular package LaTeX::Easy::Templates handles all the typesetting (via LaTeX::Driver) and makes it easy to typeset your data for publishing. With in-memory latex templates it means you can readily typeset data from a string template with a standalone perl script.

For the problem at hand: printing (envelope) labels, posted by Bod, I will use the "labels" latex package (documentation) and two simple latex templates label.tex.tx and labels.tex.tx. The latter is the entry point and includes the former for each label in the input. Of course, a single template could be used instead, but the spirit is to be modular.

Currently the only template engine supported by LaTeX::Easy::Templates is Text::Xslate which performs really fast and has some nifty features which allow for powerful expressions. Including recursion and access to any Perl module. As a consolation to TT fans, it offers a TT-like syntax, called TTerse.

Here is the entry point latex template: ./templates/labels/labels.tex.tx:

% I am ./templates/labels/labels.tex.tx
\documentclass[12pt]{letter}
\usepackage{graphicx}
\usepackage{labels}
\begin{document}
: for $data -> $label {
:   include 'label.tex.tx' { label => $label };
: }
\end{document}
[download]

The template responsible for rendering a single label is the following. Notice how it is included in the above in a loop over each label:

% I am ./templates/labels/label.tex.tx
\genericlabel{
  \begin{tabular}{|c|}
    \hline
: if $label.sender.logo {
    \includegraphics[width=1cm,angle=0]{<: $label.sender.logo :>}\\
: }
    \hline
    <: $label.recipient.fullname :>\\
    \hline
: for $label.recipient.addresslines -> $addressline {
    <: $addressline :>
: }
    \\
    <: $label.recipient.postcode :>\\
    \hline
  \end{tabular}
}
[download]

Save these two files under a directory called ./templates/labels, optionally create a logo image here ./templates/images/logo.png. You can create as complex a directory hierarchy as it suits you but you should adjust the file paths in the script.

And here is the script to harness the beast:

use LaTeX::Easy::Templates;
use FindBin;

my $curdir = $FindBin::Bin;

# the templates can be placed anywhere as long these
# paths are adjusted. As it is now, they
# must both be placed in ./templates/labels
# the main entry is ./templates/labels/labels.tex.tx
# which calls/includes ./templates/labels/label.tex.tx
my $template_filename = File::Spec->catfile($curdir, 'templates', 'lab
+els', 'labels.tex.tx');
# optionally specify a logo image
my $logo_filename = File::Spec->catfile($curdir, 'templates', 'images'
+, 'logo.png');
if( ! -e $logo_filename ){ $logo_filename = undef }

my $output_filename = 'labels.pdf';

# see LaTeX::Driver's doc for other formats, e.g. pdf(xelatex)
my $latex_driver_and_format = 'pdf(pdflatex)';

# debug settings:
my $verbosity = 1;
# keep intermediate latex file for inspection 
my $cleanup = 1;
my $sender = {
  fullname => 'Gigi Comp',
  addresslines => [
    'Apt 5',
    '25, Jen Way',
    'Balac'
  ],
  postcode => '1An34',
  # this assumes that ./templates/images/logo.png exists, else comment
+ it out:  
  logo => $logo_filename,
};
my @labels_data = map {
  {
    recipient => {
      fullname => "Teli Bingo ($_)",
      addresslines => [
        'Apt 5',
        '25, Jen Way',
        'Balac'
      ],
      postcode => '1An34',
    },
    sender => $sender,
  }
} (1..42); # create many labels yummy

my $latter = LaTeX::Easy::Templates->new({
  'debug' => {
    'verbosity' => $verbosity,
    'cleanup' => $cleanup
  },
  'processors' => {
    'custom-labels' => {
    'template' => {
      'filepath' => $template_filename,
    },
    'latex' => {
      'filepath' => 'xyz.tex',
      'latex-driver-parameters' => {
        'format' => $latex_driver_and_format,
      }
    }
    },
  }
});
die "failed to instantiate 'LaTeX::Easy::Templates'" unless defined $l
+atter;

my $ret = $latter->format({
  'template-data' => \@labels_data,
  'output' => {
    'filepath' => $output_filename,
  },
  'processor' => 'custom-labels',
});
die "failed to format the document, most likely latex command has fail
+ed." unless defined $ret;
print "$0 : done, output in '$output_filename'.\n";
[download]

edit: check the output here: https://metacpan.org/pod/LaTeX%3A%3AEasy%3A%3ATemplates#EXAMPLE:-PRINTING-STICKY-LABELS

edit: example of other labels. Cute? In general https://www.overleaf.com allows you to browse typeset documents and show you the source for using them as a basis.

edit: I have set the "untemplated" latex file to be saved to 'xyz.tex' for inspection. Ideally that file can be rendered with latex using: pdflatex xyz.tex . Its only dependency is the logo image (if any) and the labels, graphicx latex packages. In production you omit specifying a filename and all is written in temp files to be erased at the end.

edit: make sure you have latest version of LaTeX::Easy::Templates.

OnT but OtT: LaTeX has for many years been powering the scientific publishing industry (and what an industry that is!). As free and open-source software so that they can charge $$ for the People to access free literature produced by scientists paid by their taxes.

bw, bliako

I spent many hours trying to solve a problem, found a solution, and would like to share it in case others might encounter it.

A checkbox that appears on a form on an HTML page has a "name" attribute. Typically it has a "value" attribute. (If there is no "value" attribute, the default is for the web browser to send the string "on", I think, if the user has checked the checkbox before submitting the form.)

Anyway, if it unchecked, naturally there is no value sent to the script that's receiving the submitted form data. Let's say the HTML in the form states that the name of the checkbox is "member." If the user is a member of a certain civic organization, as explained elsewhere on the web page, the web page asks the user to so advise the website owner by checking the checkbox. If not a member, then the user just leaves the checkbox unchecked.

The interesting part is that the script receiving the form data has no idea whether there was a a checkbox named "member" unless the checkbox had been checked by the user. If it was left unchecked, the script doesn't get a "&member=''" (empty string) or even a "&member" in the submitted query string.

Let's say the script uses the submitted data to add the user to a database. Probably a record in a table in a relational database. The script can be written to assume that the person is not to be recorded as a "member" of the civic organization if there is no value coming in for the "member" parameter. If so, the script inserts a record in the table, puts the empty string (or NULL) into the record's "member" field (here, the field in the table happens to be the same as the name of the form parameter), and life is good.

But let's say the new user is indeed a member of the civic organization, and said so by checking the checkbox. So the script stores a "1" in the member field of the new record.

OK. Next step: provide for the editing of the record. We want to have the script create an editing form that looks like the new-user form. The script pulls the user's record from the database, sees there is a value in the member field, and hence the form is created to include "checked=CHECKED" as an attribute, e.g., <input type="checkbox" name="member" checked=CHECKED> is somewhere in the form.

But it's been six months since the user registered as a new user, and during that time she decided to let her membership in the civic organization lapse. So she wants to use the editing screen to update her record -- to literally uncheck the "member" checkbox. She does so, and submits the form.

The script, because it doesn't receive any information about the member checkbox from the user's web browser (see above), has to ASSUME that a parameter named member was in the form, and that the failure to find any submitted value associated with that parameter, or find the parameter in the first place, means the user is wanting to have her record in the database updated so as to replace "1" with "0" (or the empty string, etc.). OK, no big deal. The code is fairly simple.

But what if there were a different form presented to the user. One that is designed to let her change only her username, for example, and doesn't include the "member" checkbox. The HTML in the form is written to submit the data to the same script. But because the script has been written to assume that no incoming data for "member" means the person is no longer a member, then there's trouble when the script updates her record to change the "1" to "0" in the member field. The user only wanted to change her username.

OK, so we revise the script to make no such assumption. Now, though, how will the script know when to change the "member" field from "1" to "0"? It can't assume that every form has given the user a "member" checkbox. This is the assumption that's been made in every code example that I've seen, though.

The solution seems to be to have the form include a hidden input field that lists the names of all of the checkboxes in the form, so the script can safely assume that they're to be treated as having been submitted and they're to be treated as if the user intended them to be unchecked when the form was submitted. This can be done by printing a <input type=hidden value="member"> line somewhere inside the <form ...> and </form> tags.

I used the CGI.pm module to create a form, used its checkbox() function to print a checkbox inside it, and found that, lo and behold, the HTML includes a line that says something like <input type="hidden" name=".cgiparams" value="member"> just before the </form> tag. I had successfully reinvented the wheel.

CGI.pm is then able to redisplay the form (if, for example, the user supplied invalid data in the form such that the script is written to redisplay the form along with an error message), and the stickiness aspect of its created checkboxes means that a checkbox that initially was displayed as being checked will be correctly redisplayed as being unchecked if the user had in fact unchecked the checkbox before submitting the form. CGI.pm apparently knows that the checkbox it creates upon redisplay of the form is not to be checked this time, because it finds the member parameter in the .cgiparams list that was part of the submitted form and it determines that no value came in for the member parameter (or that the parameter didn't come in at all).

My strategy in generating web forms has been to take advantage of CGI.pm's sticky checkboxes and to use HTML::Template for templates. (I don't want to use CGI.pm to create and display the entire form using the 1995-ish technique of $q->start_html; $q->start_form [etc.]; $q->end_form; $q->end_html; in the script, even if that seems to have been a good and novel idea back in the day.)

So my script creates a sticky checkbox by calling $q->checkbox( -name => "member" ). CGI.pm returns a string of HTML that can be sent over to the template, where there is a <TMPL_VAR MEMBER_CHECKBOX> embedded in its text, waiting to receive the HTML for the checkbox, which might be checked due to CGI.pm's stickiness, or might not be.

Alas, the sticky feature still wasn't working. A "view source" revealed that the magic <input type="hidden" name=".cgiparams" value="member"> wasn't part of the form in the generated web page. But, of course, why should it be? I hadn't sent it over to the template. But where is CGI.pm generating that string? How do I get it?

It turns out that the string containing the hidden list of fields is prepended to </form> when CGI.pm's $q->end_form() is called. But beware (ask me how I know), this is the case only if $q->start_form() has been called before $q->checkbox() was called. it's not enough that the CGI module was instantiated. My template had its own <form action="https://yaddayadda.com/script.cgi"> and its own </form> tag, so I hadn't thought of using needing to call CGI->start_form() or CGI->end_form(). I only needed the creation of the sticky checkboxes.

What wasn't intuitive to me, and might be useful to others, is that I needed to have my script actually call something like my $throwaway_string = $q->start_form(); somewhere before $template->param( MEMBER_CHECKBOX => $q->checkbox(-name => "member") ); and then needed to send over the list of hidden fields by doing something like my $hidden_list_of_fields_and_closing_form_tag = $q->end_form(); -- after the last checkbox or any other form field had been created by using a CGI.pm function -- followed by something like $template->param( HIDDEN_LIST_OF_FIELDS_AND_CLOSING_FORM_TAG => $hidden_list_of_fields_and_closing_form_tag );

I don't need to send over the results of start_form(); I just need to have called it. The sticky checkboxes work even with the <form action=...> that's hard-coded in my template. But apparently I do need to call start_form() in order for CGI.pm to start paying attention and keeping a list of the names of the checkboxes (or other HTML form elements) that it's thereafter creating, so that $q->end_form() knows what to return in the way of a hidden list of fields.

I know that a web application platform might handle this particular edge case behind the scenes, but I'm not sure. I've done some development with Dancer2, but even that fine platform seems to assume that the developer has picked and implemented some Perl module (ideally, one that's more recent and more sophisticated than CGI.pm) to handle the creation of form elements, and to somehow handle this checkbox edge case.

Edited for clarity, probably unsuccessfully due to author's inability to communicate precisely :-)

Here is my understanding of what the theory behind what I am trying to do is, correct me where I am wrong, also please check my questions at the end:

The intersection of N hyperplanes (Ndim) is also known as the solution to "the system of N linear equations(=planes) with N variables". However, I am interested in the case when I have less equations. The result is not a single point in Ndim but a lower-dimensions plane. For example in 3D. The intersection of 3 planes (M=3, N=3), assuming any pair is not parallel, is a single 3D point and the intersection of 2 planes (M=2, N=3) is a line. In Ndim, the solution would be an Mdim hyperplane "living" in Ndim. Below, I call this a "hyperline".

An example of the above problem is in this SO question. The result is the equation of the intersection line in parametric form. Arbitrary values of the parameter t yields a point on this line. Ideally this is what I want to achieve: finding points on the intersection of the planes.

Solving the system of N linear equations with N variables (Ndim) can be done with, for example, Math::Matrix. Given N planes in the form a1x+b1y+c1z+...n1=0 etc., a matrix A is formed with each row being [a1, b1, c1, ..., n1] etc. then: Math::Matrix->new([[a1,b1,c1,...,n1],...])->solve->print;

I have experimented with solve() for the case of M linear equations (Ndim) where M = N-1:

use Math::Matrix;
Math::Matrix->new(
  [1, 2, 1, -1], # ax+by+cz+n=0
  [2, 3, -2, 2]
)->solve->print;
# says
# -7.00000    7.00000 
#  4.00000   -4.00000
[download]

I interpret the result as: the 1st row refers to the 1st dim (aka x) and we have

x -7*t + 7 = 0 => x = 7*t - 7
[download]

y + 4*t -4 = 0 => y = -4*t + 4
[download]

I have also experimented with transforming the planes matrix to the Reduced Row Echelon Form. Which makes it convenient to work out the "hyperline" equation. This functionality is offered by Math::MatrixLUP :

use Math::MatrixLUP;
my @planes3D = (
  [1, 2, 1, -1], # ax+by+cz+n=0
  [2, 3, -2, 2]
);
print Math::MatrixLUP->new(\@planes3D)->rref
# says
# [1, 0, -7, 7],
# [0, 1, 4, -4]
[download]

Which I interpret as: 1st row has 1 on 1st col, that's the 1st dim (aka x) :

x -7 * t + 7 = 0 => x = 7*t-7
[download]

So that works well.

Here is a demo I whipped up:

I have some questions:

• Two planes are parallel when their normal vectors (the coefficients of the dimensions: a, b, c, ... (but not the constant n) are multiples. E.g. nv1 = k * nv2. And this translates to checking if all the ratios of the coefficients : a1/a2 = b1/b2 = c1/c2 = ... are the same. My question is: what happens if any coefficient is zero (or actually both coefficients (e.g. a1 and a2) are zero?
• How can I calculate the intersection when M < (N-1)? I.e. above I am always checking the intersection of M planes in Ndim where M = N-1. But what if there are even less planes? e.g.

  my @planes5D = (
    [1, 2, 1, -2, 3, -1], # ax+by+cz+n=0
    [2, 3, -2, 4, -4, 2],
    [3, -1, -2, -5, 6, 2],
    # only 3 planes (it was successful with 4)
  )
  [download]

  In short, does the parametric equation of the intersecting "hyperline" contain 2 parameters now?
• When I test test_in_beast_space which produces random planes in 666 dimensions, it fails to detect that a point on the intersection "hyperline" lies on all the planes. To do that I substitute the point in each plane equation and expect to have result as zero. However, it's not an exact zero. That's why I am doing a range test to check if it's close to zero. Well the closeness $SMALLNUMBER for 5 dimensions can be as small as 1E-09. But for these many dimensions it can be as low as 1E-02. Is the accuracy lost in summing 666 multiplications that much really?
• I guess Math::Matrix::solve() is safe to be used for MxN matrices (e.g. M equations=planes in N unknowns (dimensions) where M<N)? From solve() of Math::Matrix :

  Solves a equation system given by the matrix. The number of colums mus
  +t be greater than the number of rows. If variables are dependent from
  + each other, the second and all further of the dependent coefficients
  + are 0. This means the method can handle such systems. The method ret
  +urns a matrix containing the solutions in its columns or undef in cas
  +e of error.
  [download]

• The edge cases where the 1st dimension is zero for all planes in 5D and when the 1st+2nd are zero, fail. How can I find the intersection in this case?

Edits: updated the demo to correct the sub are_planes_parallel() as per the comments of hippo and LanX about what happens when plane-equation coefficients are zero.

have fun in beast space! bw bliako

In 1998, a friend gave me a computer. It was in pieces. I knew nothing about nothing. Within a year, I was communicating over dual telephone lines. Within two years of that, I was a sysadmin at an ISP. This was the year I found Perlmonks. I read a book, 'Perl in 21 days' or some such, and found Perl was what I wanted... a way to automate processes.

Within months, I learned a dangerous amount about MySQL, CGI and Perl to allow any invader to break everything. Thankfully, during that time, everyone was out for themselves, and exploitation hadn't yet become a thing.

By 2009, I'd grown a lot in many areas. No where near perfect in the security arena, but I was becoming proficient on how to interact with the open source world, and how to interact with the CPAN. It was this year that I joined Perlmonks as a member, and became a vocal person, not just a listener.

Now, as some of the old timers will attest to, I always claimed "I'm not a programmer". With that said, I have done much work in fields so closely related to programming, that I have to bend and say that yeah, maybe I can classify as a hacker.

Anyone else around who have claimed "I'm not a programmer", or who has been around since the very early days of Perlmonks who would just like to say "I'm still here!!!"?.

-stevieb

I needed to generate random strings from a regular expression and I have found (*) a C++ library (regxstring C++ library by daidodo) which does just that and works fine for my use case. It claims that it supports most Perl 5 regexp syntax. It is quite fast too. So I have ported it into a Perl module, here : String::Random::Regexp::regxstring.

perl -MString::Random::Regexp::regxstring -e 'print @{generate_random_
+strings(q{^\d{3}[-,.][A-V][a-z]{3}\d{2}})};'
[download]

use String::Random::Regexp::regxstring;
my $strings = generate_random_strings('^[a-c]{2}[-,.]\d{3}-[A-Z]{2}$',
+ 10);
print "@$strings\n"
[download]

The XS and C++ code bridging Perl to C++ library are very simple and can serve as an example for doing that for other libraries. Go forth and multiply.

(*) Via this old thread I found Regexp::Genex and there is also String::Random. Neither worked for my use case.

bw, bliako

I was thinking what my options are for controlling a remote farm remotely.

I needed to do basic stuff like run the motor to push the feed to the animals, open a valve to let mains water fill their drinking buckets, check feed and water levels, check temperature. Even count the number of eggs.

The farm has no electricity is off-grid and has no alternative power installed, so it must rely on solar energy. I want to minimise the use of batteries. There is no WIFI, but it is covered by the national 4,5G telephone network.

A smartphone has all the communication modules available and ready: sms, voice, data, bluetooth, even rfid. It also has a good power supply and provided is not loaded with apps, it can last for 24 hours at least for the next sunshine for the solar charger to work. And it allows high-level programming.

Unfortunately a smartphone lacks any connectivity to the world by means of IO ports. And so I am looking for an IO board to the smartphone via its USB or perhaps the bluetooth. The cheaper the better really and simpler too.

I found IOIO-OTG which interfaces android-based smartphone or PC via usb to a lot of IO ports which can read sensors or run motors and actuators.

I really like an IO board+Smartphone solution because I can sms to it commands like "feed animals" or "count eggs". And it can reply back via sms. I can also command it via internet ("data") and perhaps get the odd picture back. There is solar charging available and cheap. And with just a single app running perhaps it can manage 24hrs recharge cycles. From all of the above I love the control-by-sms idea.

My first point of call was the raspberry but i would prefer that there was a plugin module to the pi to do that rather than buying cells+batteries and doing calculations. E.g. https://community.element14.com/products/raspberry-pi/f/forum/53345/power-raspberry-pi-using-solar-panel

I would also like to communicate with the device. The most reliable way I see is via SMS. And the most practical is via 5G internet (data). The PI offers sending SMS but again, it's hands-on and a lot can go wrong.

So, I am looking for recommendations on other IO boards for smartphones (android is just fine), and/or similar solutions for the PI just to be fair to the PI, if people feel that's a better environment.

I have asked hacker news about this here

thanks, bw, bliako

edit:the IOIO-OTG board needs own power supply when plugged into smartphone. Not when plugged into PC

Edit 27/06/2024: I have read that when connecting the board to smartphone (not PC) it must provide its own power supply. It can not be powered from Android. Android then asks you if you want to charge the phone with what it found on its USB port or transfer photos etc. This fits well with the design that the board has its own solar power+battery which then charges the phone as well, and also drives any external motors (TODO: noise from motors into the board). If external power runs out (no sun for days) then, firstly, the board stops and then the phone runs out of its own battery (sending an sms to me when in the critical zone) after a while. When solar power recharges on sun appearing, the board will be able to charge the phone too. Problem I see, how to turn the phone on when power comes back and how to tell it that what is on the USB port (our IO board) should be run on the specified mode (transfer files, charge, whatever) WITHOUT user intervention. Just by its own.