Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

The Monastery Gates

( #131=superdoc: print w/replies, xml ) Need Help??

If you're new here please read PerlMonks FAQ
and Create a new user.

Quests
poll ideas quest 2022
Starts at: Jan 01, 2022 at 00:00
Ends at: Dec 31, 2022 at 23:59
Current Status: Active
3 replies by pollsters
    First, read How do I create a Poll?. Then suggest your poll here. Complete ideas are more likely to be used.

    Note that links may be used in choices but not in the title.

Perl News
PDL 2.079 released
on May 03, 2022 at 16:44
0 replies by etj
    PDL 2.079 has just been released. Notable changes since 2.078: Future plans, in something like intended order:
    • Restructure the TriD stuff so there is a consistent API between OpenGL and X3D/VRML - see 11143037 for more
    • fix more open GitHub issues
    • add event-handling hooks for ndarrays - see PDL::Dataflow for more
    • finish the independent C interface for making PDL usable from e.g. Python - see https://github.com/PDLPorters/pdl/issues/358 for more
    • “loop fusion” techniques to maximise locality of computation, minimising data’s trips through the “straw” between CPU and main RAM
    • use OpenCL or other means to also utilise GPUs if available - see https://github.com/PDLPorters/pdl/issues/349 for more on this and the above

    Please give the new release a try and report problems.

pdl.perl.org website updated
on Apr 15, 2022 at 12:16
0 replies by etj
    I've just (hopefully) finished tweaking the website at https://pdl.perl.org/, using Joel Berger's JavaScript-based update, enabling HTTPS by hosting on GitHub Pages. The search now uses MetaCPAN which has a neat autocomplete feature.

    I've probably missed some stuff so please have a nose around and open issues on https://github.com/PDLPorters/pdlporters.github.com/issues or use the “Website issues” link in the sidebar (which links to that) - or reply on here!

    As usual, please give the new website a try and report problems.

Supplications
LWP and Mechanize
3 direct replies — Read more / Contribute
by perlmike
on May 21, 2022 at 15:34

    I used to use LWP and Mechanize to download files from the internet, but they stopped working today. Does anyone know what's going on? Thanks!

    use strict; use warnings; use LWP::UserAgent(); my $ua = LWP::UserAgent->new(timeout => 10); $ua->env_proxy; my $response = $ua->get('https://www.sec.gov/Archives/edgar/full-index +/2019/QTR1/master.idx'); open(OUT, ">" . "master") or die "Cannot open master"; if ($response->is_success) {print OUT $response->decoded_content; } else {die $response->status_line;} close OUT;
Capture a non-printable char and test what it is
2 direct replies — Read more / Contribute
by almsdealer
on May 21, 2022 at 10:03

    Good morning,

    Can anyone tell me how I would capture a non-printable character from STDIN and test what the character is?

    Specifically, I want to test if the escape key has been pressed.

    Trying to build a console based text editor.

The wonders of Test::More
2 direct replies — Read more / Contribute
by syphilis
on May 21, 2022 at 09:29
    Hi,

    Have a laugh at this piece of brilliance:
    C:\>perl -MMath::MPFR -MTest::More -e "cmp_ok(Math::MPFR->new(2), '!=' +, Math::MPFR->new(2), 'test for blow up'); done_testing();" not ok 1 - test for blow up # Failed test 'test for blow up' # at -e line 1. Operation "eq": no method found, left argument in overloaded package Math::MPFR, right argument in overloaded package Math::MPFR at C:/perl-5.3 +4.0/site/lib/Test/Builder.pm line 1006. A context appears to have been destroyed without first calling release +(). Based on $@ it does not look like an exception was thrown (this is not + always a reliable test) This is a problem because the global error variables ($!, $@, and $?) +will not be restored. In addition some release callbacks will not work prop +erly from inside a DESTROY method. Here are the context creation details, just in case a tool forgot to c +all release(): File: -e Line: 1 Tool: Test::More::cmp_ok Here is a trace to the code that caused the context to be destroyed, t +his could be an exit(), a goto, or simply the end of a scope: Context destroyed at C:/perl-5.34.0/site/lib/Test/Builder.pm line 1006 +. eval {...} called at C:/perl-5.34.0/site/lib/Test/Builder.pm l +ine 1006 Test::Builder::cmp_ok(Test::Builder=HASH(0x24eaa60), Math::MPF +R=SCALAR(0x86b998), "!=", Math::MPFR=SCALAR(0x33cf88), "test for blow + up") called at C:/perl-5.34.0/site/lib/Test/More.pm line 511 Test::More::cmp_ok(Math::MPFR=SCALAR(0x86b998), "!=", Math::MP +FR=SCALAR(0x33cf88), "test for blow up") called at -e line 1 Cleaning up the CONTEXT stack... # Tests were run but no plan was declared and done_testing() was not s +een. # Looks like your test exited with 255 just after 1.
    Note that the problem arises at C:/perl-5.34.0/site/lib/Test/Builder.pm line 1006.
    It's not the test failure that's an issue - pretty obviously, that test should fail.
    It's the ensuing fatal over-the-top blowup that's my main concern.
    Here's the actual section of the Test::Builder code that blows up:
    ... unless($ok) { $self->$unoverload( \$got, \$expect ); if( $type =~ /^(eq|==)$/ ) { $self->_is_diag( $got, $type, $expect ); } elsif( $type =~ /^(ne|!=)$/ ) { no warnings; my $eq = ($got eq $expect || $got == $expect) ## LINE 1006 ...
    The evaluation of $got eq $expect is bound to be interesting if $got is an object and the 'eq' operator has not been overloaded

    Anyway ... my question is "what is the bug ?"
    Is it that 'eq' overloading has not been provided ? (Providing such overloading successfully works around the problem.)
    Is it that Test::More (or Test::Builder) should not be assuming that 'eq' has been overloaded ?
    Or has Test::Builder tried (and failed) to account for this possibility. I'm looking at $self->$unoverload( \$got, \$expect ) in the above snippet from Builder.pm, and wondering what that might be intended to do.

    With math objects, I've never felt the need to be able to compare $obj eq $str. If I need to do that I just make use of the overloaded stringification and compare "$obj" eq $str.

    I'm interested to hear thoughts on what is the "correct" way to deal with this issue.

    Cheers,
    Rob
getsockopt truncating values to 256 bytes ?
2 direct replies — Read more / Contribute
by Yaribz
on May 20, 2022 at 09:42

    Hello, I'm trying to retrieve the full TCP_INFO socket structure on a macOS system using Perl.

    I have no problem retrieving this structure on other systems (Linux, FreeBSD...) using the getsockopt function, but on these systems the structure is less than 256 bytes long.
    On macOS, this structure is supposed to be 292 bytes long (296 with padding), as one can see here. When I try to retrieve this structure from Perl using getsockopt on macOS 12.0.1 (Darwin 21.1.0), I only get 256 bytes of data: the last values are missing from the structure.

    Is it due to this code from Perl core, which seems to limit getsockopt results to 256 bytes ? And in this case, what can I do to retrieve the full TCP_INFO structure ?

    Here is a code snippet to reproduce the problem on macOS:

    use Socket qw'PF_INET SOCK_STREAM IPPROTO_TCP inet_aton sockaddr_in'; my $TCP_INFO = ($^O eq 'darwin' ? 0x200 : eval { Socket::TCP_INFO() }) or die "This system doesn't support the TCP_INFO structure.\n"; my ($testHost,$testPort)=('perl.org',443); socket(my $sock, PF_INET, SOCK_STREAM, IPPROTO_TCP) or die "Could not create socket - $!\n"; my $iaddr=inet_aton($testHost); my $paddr=sockaddr_in($testPort,$iaddr); connect($sock,$paddr) or die "Failed to connect to $testHost:$testPort - $!\n"; my $tcpInfoData=getsockopt($sock,IPPROTO_TCP,$TCP_INFO) or die "Error while calling getsockopt - $!\n"; my $tcpInfoLength=length($tcpInfoData); if($tcpInfoLength < 256) { print "This system doesn't have a TCP_INFO structure large enough to + reproduce the problem.\n"; }elsif($tcpInfoLength == 256) { print "The TCP_INFO structure seems to be truncated to 256 bytes on +this system.\n"; }else{ print "This system doesn't seem to be affected by the problem.\n"; }

    Thanks !
Converting entities in JSON context
2 direct replies — Read more / Contribute
by Anonymous Monk
on May 19, 2022 at 07:45

    I'm struggling with something that I thought would be very simple. I have a legacy system which sends data in JSON. The underlying data, which I can't change, uses HTML entities. I need to convert this to UTF8, because a receiving system can't handle the entities. I wrote a one-line test for this, which is failing, and I don't know why.

    When I do the conversion on the text itself, it looks fine. When I do the conversion on the JSON, it also looks fine, but when I decode the JSON for the test, it seems to re-convert the UTF8 JSON elements into something wrong. A simple test case:

    #!/usr/bin/env perl use strict; use warnings; use HTML::Entities; use Encode; use JSON::MaybeXS; my $original_string = "E&#x00F6;tv&#x00F6;s Lor&#x00E1;nd University"; my $converted_string = encode_utf8( decode_entities($original_string) +); print "Original string: [$original_string]\n"; # shows the entities print "Converted string: [$converted_string]\n"; # shows the special c +haracters my $entities_json = '{"school":"E&#x00F6;tv&#x00F6;s Lor&#x00E1;nd Uni +versity"}'; my $converted_json = encode_utf8(decode_entities($entities_json)); print "Original JSON: [$entities_json]\n"; # shows the entities print "Converted JSON: [$converted_json]\n"; # looks right: shows the +special characters my $decoded_json = decode_json($converted_json); print "School: " . $decoded_json->{'school'} . "\n"; # should be "Eötv +ös Loránd University" but is actually "&#65533;tv&#65533;s Lor&#65533 +;nd University", with the special characters messed up (N.B. Perlmonk +s is showing this incorrectly as well)
    What is going on here? And, how am I supposed to convert my JSON-with-entities to something, well, correct?
scan, match and extract
6 direct replies — Read more / Contribute
by seismofish
on May 19, 2022 at 07:13
    Greetings, Monks

    I frequently have to scan files for lines matching various patterns and extract data from those lines. I use code like

    my $re_dn = qr{^(\w+) (\d+) (\d{2}:\d{2}:\d{2}) .*: eth(\d): link down +}; my $re_dn = qr{^(\w+) (\d+) (\d{2}:\d{2}:\d{2}) .*: eth(\d): link up, +(\d+)Mbps, ([^,]+), lpa (\w+)}; while(<>) { if ( /$qr_dn/ ) { my( $mon, $day, $time, $interface ) = ($1,$2,$3,$4); # process next; } if ( /$qr_up/ ) { my( $mon, $day, $time, $interface, $rate, $duplex, $lpa ) += ($1,$2,$3,$4,$5,$6,$7); # process next; } }
    but I hate constructs like ($1,$2,$3,$4,$5,$6,$7)

    I'd love to know a more elegant idiom. Will someone enlighten me?

NTLM Authentication w/ Internal Site
1 direct reply — Read more / Contribute
by DanEllison
on May 18, 2022 at 16:09

    Whittled down to the minimum, I have:

    use strict; use warnings; use LWP::UserAgent; use HTTP::Request::Common; use Authen::NTLM; use LWP::ConsoleLogger::Everywhere; ntlmv2(1); my $ua = LWP::UserAgent->new(keep_alive => 1, ssl_opts => { verify_hos +tname => 0 }); $ua->credentials(internal.com:9004', '', 'user', 'pass'); my $req = GET “https://internal.com:9004/api”;

    I assume because it is an internal site, if I don't turn off verify_hostname, I get:
    500 Can't connect to internal.com:9004 (Bad File descriptor)

    However, if I turn off verify_hostname, the ConsoleLogger shows that I am attempting to do my NTLM authentication, however I can't seem to get by:
    401 Unauthorized

    I do notice in the result header a warning:
    Client-SSL-Warning: Peer certificate not verified

    I'm wondering if my authentication problems are due to not verifying the host. I'd prefer that, but I've tried downloading certificates, extracting a fingerprint, but can't seem to get past the 500 error without turning off verify_hostname. How can I verify my internal site? And/or should I be looking elsewhere for my NTLM Authentication issue?

    I'm attempting this from Strawberry on Windows, but I am able to authenticate against my internal site using curl from one of my linux hosts, so I know the url and credentials are correct.

running command line from Perl
2 direct replies — Read more / Contribute
by Anonymous Monk
on May 18, 2022 at 07:22

    Hello. How do I run the following from a Perl script on Windows?

    get-wmiobject Win32_ComputerSystemProduct | Select-Object -ExpandPrope +rty UUID # this runs fine from command line

    The followings do not work

    my ($UUID) = system ("get-wmiobject Win32_ComputerSystemProduct | Sele +ct-Object -ExpandProperty UUID"); # 'get-wmiobject' is not recognized + as an internal or external command my ($UUID) = system ('get-wmiobject Win32_ComputerSystemProduct | Sele +ct-Object -ExpandProperty UUID'); my ($UUID) = `get-wmiobject Win32_ComputerSystemProduct | Select-Objec +t -ExpandProperty UUID`; # same error as above, `` works perfectly fi +ne on macOS
processing a module result
1 direct reply — Read more / Contribute
by averlon
on May 18, 2022 at 01:42

    Hi - I am not a good programmer in perl.

    I use the module Masscan::Scanner. But I have not found a way to process the modules result and would like to have some help on this

    # Returns the scan results my $res = $mas->scan_results if ($scan);

    Any help how I can access e.g. the IP-Address in the result data structure or even the array (seems to be an array) of ports would be welcome

    Regards Karl-Heinz
New CPAN distributions not indexing
1 direct reply — Read more / Contribute
by stevieb
on May 17, 2022 at 10:41

    Hello there my fellow Monks,

    I recently published a new distribution to the CPAN (Net::MyIP), but even after a week, it hasn't become available by searching, nor with cpan/cpanm tools. I can see that it is there and even being tested if I go to the link directly (here), but for some reason it's not being indexed.

    I sent an email to modules@perl.org, but the only response I got was someone wondering whether it was an issue with the distribution itself. Seems pretty unlikely as this is my 59th distribution I've authored and uploaded over the course of a decade-plus. There were no errors, the PAUSE page claimed it uploaded properly, and even the PAUSE logs indicated that.

    This distribution is a prerequisite for another one I completed at the same time, but can't upload due to what would be a broken dependency.

    Is there anyone here who can help sort this out?

    Thanks,

    -stevieb

Meditations
How to get programming help
3 direct replies — Read more / Contribute
by jdporter
on May 23, 2022 at 11:21
    "Every time I have a programming question and I rly need help, I post it on PerlMonks and then log into another account and reply to it with an obscenely incorrect answer. Ppl don’t care about helping others but they LOVE correcting others. Works 100% of the time" -- @soychotic
Libraries and security
1 direct reply — Read more / Contribute
by davies
on May 23, 2022 at 10:25

    An article in today's Register, https://www.theregister.com/2022/05/23/npm_dependencies_vulnerable/ has got me thinking. One quote from it is "These 2,415 people with these email addresses are currently more trusted than your own software engineers as they can effectively run any code they want on your production infrastructure without review from anyone". When the Log4J vulnerability appeared, I found out (I hardly ever use Java) that the offending module could be in any .jar without it being obvious to the programmer. At least, when I install a CPAN module, I can see a list of the modules it brings with it, even if I don't usually pay enough attention. But sometimes I need to be very security conscious.

    Looking for vulnerabilities in CPAN took me to https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=cpan. There's not a lot there, and many of them are closed in the latest version. But looking at one of them, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-4184, I see that the module was updated <update>last month two years ago, seven years after the vulnerability was reported</update>, but that there is nothing in the changes file to indicate that the vulnerability has been addressed even though it's nine years old. Nor is there anything in the docs even acknowledging the existence of the vulnerability.

    The changes file does, however, reveal another worry.

    1.148 Thu Nov 16 10:21 2006
    - Debian has chosen to distribute their own Data::UUID, which has a different interface and breaks other modules.

    What, then, do people do in the real world? Are checks for CVEs enough? Do people do code reviews of every module they use in every language? It sounds beyond the bounds of reasonability to me. Are there certifications for modules (and if so, who certifies the certifiers - that old question)? I know people who prefer to install Perl packages on Debian via apt because it makes them easier to delete, but the quote above makes me fear that the CPAN and apt packages need not be the same.

    Regards,

    John Davies

    Update: I read the date carelessly & got it wrong. Thanks, Hippo.

Cool Uses for Perl
Dynamic DNS for your GoDaddy domains
No replies — Read more | Post response
by stevieb
on May 17, 2022 at 14:26

    With its API, GoDaddy makes it easy to do dynamic DNS updates for your domain's hostnames. I made it easy to do with Perl with Net::DynDNS::GoDaddy (which uses my new Addr::MyIP to get your current external IP address). I'll give an example, then an example use of the distribution's packaged binary script.

    use Addr::MyIP; use Net::DynDNS::GoDaddy; my $hostname = 'home'; my $domain = 'example.com'; my $current_host_ip = host_ip_get($hostname, $domain); my $my_ip = myip(); if ($current_host_ip ne $my_ip) { host_ip_set($host, $domain, $my_ip); }

    Simple. The library requires a godaddy_api.json file in your home directory (MacOS, Unix or Windows, the software has 100% test coverage on all systems) that looks like this:

    { "api_key" : "api_key", "api_secret" : "api_secret" }

    Using the binary we'll install when you install the library, it will prompt you for this information on its initial run:

    > update-ip home example.com Please enter your GoDaddy API key and hit ENTER: ...api_key... Please enter your GoDaddy API secret and hit ENTER: ..api_secret... Updated record for 'home.example.com' from x.x.x.x to x.x.x.x

    ...after the initial run, it won't prompt anymore:

    > update-ip home example.com Not updating the 'home.example.com' record, IPs are the same

    You can specify the IP if you don't want to use your current public-facing one we automatically get for you:

    > update-ip home example.com 10.7.10.2

    The most useful use for me is to have multiple hostnames ('home', 'office', 'roaming' etc) and just run the program through cron:

    # Home storage server */15 * * * * update-ip home example.com >> /tmp/update-home_cron.log 2 +>&1

    My laptop:

    */15 * * * * update-ip roaming example.com >> /tmp/update-roaming_cron +.log 2>&1

    Usage:

    Usage: update-ip host domain.name [ip.addr]

    Have fun!

    -stevieb

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (2)
As of 2022-05-24 03:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (82 votes). Check out past polls.

    Notices?