Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

The Monastery Gates

( #131=superdoc: print w/replies, xml ) Need Help??

Donations gladly accepted

If you're new here please read PerlMonks FAQ
and Create a new user.

New Questions
JSON character encoding
2 direct replies — Read more / Contribute
by Bman70
on May 28, 2017 at 23:35
    Hello! I'm using an API request to get some video descriptions from Youtube. Something like:
    #!/usr/bin/perl use LWP::Simple; use lib 'JSON253/lib/'; use JSON qw( decode_json ); use CGI::Carp qw(fatalsToBrowser set_message warningsToBrowser); use strict; use warnings; my $url = " +ippet&key=$API_KEY"; my $result = get($url); die "Could not get $url!" unless defined $result;
    Then I decode the JSON response and load what I want into variables:
    $jsonstuff = decode_json($result); $description = $jsonstuff->{items}->{snippet}->{description};
    This all works, except that some video descriptions have weird characters in them. Some are emoticons. Here's an example from a description with an emoticon smiley at the end:
    "Happy with everything at the moment \x{1f60a}."
    Another quirk is that some apostrophes get mangled. Here's another printed description:
    "It’s the fans who make it worthwhile."
    I know it's a character encoding issue.. but is Youtube using a mix of encodings? I once got the apostrophe to print correctly by changing Windows-1252 to UTF-8 in one step. But the emoticons were still mangled. I should add that not all apostrophes are mangled. On some videos they come through fine. I wondered if it had to do with the uploader's text processing method. I've been searching most the day and trying various fixes, even from a number of threads. But I haven't been able to make it work yet. Thanks
DBD::CSV - how to I coax it to read BOM prefixed files?
2 direct replies — Read more / Contribute
on May 28, 2017 at 17:06

    I have a bom prefixed file that I'd like DBD::CVS (v. 0.49) to read and query, but when I tried I kept getting the error message Execution ERROR: Missing first row due to EIF - Loose unescaped quote . But there are no unquoted fields and the only thing I can think is happening is that it sees the BOM and thinks the first field of the first row begins without a quote. (For those not familiar with the DBD::CSV documentation "Loose unescaped quotes" are quotes embedded in a field that begins without a quote).

    I'm fairly certain the problem is the BOM because if I remove the BOM, the file can be read and loaded without error

    What's puzzling is that the underlying Text::CSV_XS module is BOM friendly. I've tried a variety of things to no avail, among them:

    • setting f_encoding => 'utf8'
    • setting csv_detect_bom => 1 and detect_bom => 1 in hopes that it would be passed onto the underlying Text::CSV_XS module
    • using an explicit column list (via col_names) and skipping the first line via csv_skip_first_row
    • setting csv_allow_loose_quotes => 1. That suppresses the EIF error message but replaces it with complaints that the column name stored in the first field in the first line doesn't exist - adding to my suspicions that it thinks the BOM is part of the field name

    I had considered just removing the BOM and ignoring the problem, but this isn't a one-off. A data source I don't control is producing these BOM prefixed files. I'd rather not be manually munging files (even with a script), every few months. Furthermore the source, or one of them, is the OECD. They produce a lot of data and I'd like that data to be accessible to other Perl programmers in CSV format, either through the solutions we discuss on this thread or because DBD::CSV has been enhanced to be BOM friendly.

    Many thanks in advance

Is there a "standard" way to link to other modules in POD for a CPAN module?
4 direct replies — Read more / Contribute
by atcroft
on May 27, 2017 at 14:33

    Earlier this week, I was looking for information on a module on While reading through the module's documentation, there were several related modules mentioned, but the links referred to the module on

    Curious, I viewed the module's POD, and found they had used L<text|url> POD linking syntax. When I later viewed the POD of a few other modules that referred (linked) to other modules in a way I expected, I found those modules using the L<module> / L<module/section> / L<text|module> / L<text|module/section> syntax.

    To me, this violated the Principle of Least Astonishment (Surprise), but I can see a scenario where the author may not have noticed the issue, such as if (for instance) they use as their preferred interface for finding modules on CPAN. I considered contacting the module's author regarding the linking, but before I potentially made (more of) a fool of myself (than normal) by doing so (and because I am still (at best) a novice with respect to POD), I wanted a "second opinion" (as it were). So, my question is:

    Is there a "standard" manner for linking to other modules within CPAN I should recommend, or should I just ignore it as being my issue, not an actual problem worthy of disturbing the author?

    My thanks in advance for your shared wisdom and consideration.

Finding what modules use and what uses the modules
4 direct replies — Read more / Contribute
by Lady_Aleena
on May 26, 2017 at 21:16

    As some of you know, I'm doing an audit of my code. Part of that audit is finding what modules my modules use and what modules I have written are used by other modules I have written. If I ever get things nice enough in some of my modules, I might one day put a few up on CPAN. However, I need to know what modules my modules use so I can include them in the install if the user does not already have. Like, if I use Lingua::EN::Inflect in a module (or group of modules), it would need to be installed if the user does not have it already. At least I think so.

    I came up with the following code to find the information but there might be a better way.

    #!/usr/bin/perl use strict; use warnings; use File::Find; my @files; sub wanted { my $file = $_ =~/.pm$/ ? $File::Find::name : undef; push @files, $file if $file; } my @directories = ('/home/me/Documents/'); find(\&wanted, @directories); my $module_directory = $directories[0].'www/files/lib/'; my $modules; for my $file (@files) { open my $fh, '<', $file or die "$file: $!"; my $file_convert = $file; $file_convert =~ s/$module_directory(.+)\.pm/$1/; $file_convert =~ s/\//::/g; while (<$fh>) { chomp; if ($_ =~ s/^use ((:|\w)+)(.+)/$1/) { push @{$modules->{$file_convert}{'uses'}}, $1; # what modules + the module uses push @{$modules->{$1}{'used by'}}, $file_convert; # what modules + use this module } } } my $in = shift; use Data::Dumper; print Dumper($modules->{$in});

    Here is example output.

    me@office:~$ perl Documents/scripts/ 'RolePlaying::R +andom' $VAR1 = { 'used by' => [ 'RolePlaying::CharacterMutation', 'RolePlaying::Random::Range', 'RolePlaying::Random::Time', 'RolePlaying::Random::SpecialAttack', 'RolePlaying::Random::Title', 'RolePlaying::Random::WildPsionics', 'RolePlaying::Random::Descriptor', 'RolePlaying::Random::Spell', 'RolePlaying::Random::SavingThrow', 'RolePlaying::Random::Color', 'RolePlaying::Random::Monster', 'RolePlaying::Random::GemMetalJewelry', 'RolePlaying::Random::Thing', 'RolePlaying::Random::Weapon', 'RolePlaying::Random::Size', 'RolePlaying::Random::Food', 'RolePlaying::Random::Misc', 'RolePlaying::Random::Event', 'RolePlaying::Random::Water', 'RolePlaying::Random::Class', 'RolePlaying::Random::MagicItem', 'RolePlaying::Random::Color::VisiBone', 'RolePlaying::Random::Body::Function', 'RolePlaying::Random::Body::Modification' ], 'uses' => [ 'strict', 'warnings', 'Exporter', 'List::Util' ] };

    If you know of a better way that will give me deeper or more sophisticated results, I am interested.

    No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!
    Lady Aleena
Calling a subroutine when part of call is a variable Contant
3 direct replies — Read more / Contribute
by Anonymous Monk
on May 26, 2017 at 18:13

    Dear Monks, Hoping you can help. I can call the subroutine first_name() like this:


    However I want to call it when 'first_name' is a constant. The following code doesn't work

    use constant { STUFF => { 'name1' => 'first_name', 'name2' => 'last_name', }, }; foreach my $val (keys ${\( STUFF() ) } ){ my $value = eval("$self->xml()->${\( STUFF() ) }->{ $val }()"); }

    I can't use a code ref like below because of how it needs to be called ( perhaps there is a way? ) Note: I cannot use Readyonly either. I've also tried various combinations of \&{$variable} but can't seem to get it to work.

    use constant { STUFF => { 'name1' => $self->xml()->first_name(), 'name2' => $self->xml()->last_name(), }, };

    Please help if you can. Thanks

HTML::Parser / Regex
5 direct replies — Read more / Contribute
by MissPerl
on May 26, 2017 at 16:30
    Hi fellow Perl Monks,

    I am trying to get text/number in a html file then store them into a variable. I know that HTML::TableExtract or some other module might have easier way to do this. But for now, I want to learn and apply HTML::Parser and regex first.

    This is part of my failed attempt perl script, it got errors like bareword found (might be runaway multi-line) and can't use global $1 in my. At the beginning of the script, it prompt user for input then store them into a variable. For now, I am writing the part for the script to be able to reads the $ca html file and find match. Then next part of the script will continue for the other states' html file.

    use HMTL::Parser; my $ca = "california.html"; open (my $f1, "<" , $ca) || die ("Can't open file : california.html"); while (<$f1>){ if (my $text =~ /Employee\sA</th><th>.\d</){ my $one = $1; }elsif (my $text =~ /Employee\sB</th><th>.\d</){ my $two = $1; }elsif (my $text =~ /Employee\sC</th><th>.\d</){ my $three = $1; } } close ($f1);
    Below are a few lines from two different html files.
TCP Server: Beyond echoing request
2 direct replies — Read more / Contribute
by jeremywakeman
on May 25, 2017 at 10:03

    I'm building a TCP server to learn more about networking. What I have so far is based on various online examples: a server that responds to requests using a series of if, elsif, elsif attempts to match the request to an appropriate response.

    It seems like I should be looking to implement an application layer protocol to transfer data (text) between server and client, but I don't want to invent my own protocol or use one that's more complex than I need. Google searches have not resulted in a definitive answer. Is there an existing protocol I should look into?

    At the moment my client is just nc to the appropriate server port. I'm assuming I'll have to build a client if I implement an application level protocol

Use CGI to run a Perl script via web server
5 direct replies — Read more / Contribute
by suvajit123
on May 25, 2017 at 07:07

    Hello Monks

    I have created a Perl script which runs fine manually with a value parsed during runtime, i.e. like 'perl value'

    I need to execute this over a Web Server (IBM HTTP Server), I have configured it correctly and it is able to execute the program to display Hello World with a name given as input in the URL. like

    But my actual code which calls for a different Perl script from this, does not work.

    #!/usr/bin/perl use strict; use CGI; use IPC::System::Simple qw(system capture); #create CGI query object to get the SSO from URL my $sso = new CGI; my $sso = $query->param( "sso" ); #*********Script within a script both files in same path********** system( "perl $sso" ); print "Content-type:text/html\r\n\r\n"; print "<html>"; print "<head>"; print "<title>Some Header</title>"; print "</head>"; print "<body>"; print "<h2>You have entered $sso</h2>"; print "</body>"; print "</html>";

    This is not working as it is not detecting IPC::System::Simple qw(system capture) from this script. How could I make the CGI parse the $sso to ?

    Thanks in advance....

Tcl::Tk exit
2 direct replies — Read more / Contribute
by Sandy
on May 24, 2017 at 15:03
    Really silly question.

    I am used to Perl/Tk, with sample code that looks like

    use Tk; my $mw = MainWindow->new(-title=>"hello world"); $mw->Label(-text=>"hello")->pack(); $mw->Button(-text=>"Exit",-command=>sub{exit})->pack(); MainLoop;

    Now I am updating (?) to Tcl::Tk (because Perl/Tk requires Quartz on my Mac)

    use Tcl::Tk; my $int = new Tcl::Tk; my $mw = $int->mainwindow; $mw->Label( -text => "Hello World!" )->pack(); $mw->Button( -text => "Exit", -command => sub { exit } )->pack(); $int->MainLoop;

    But when I exit, I get the following error message (and a segmentation fault)

    perl5.18 quit unexpectedly while using the Tk plug-in.

    I have tried combinations of $mw->exit(), but that also doesn't work.


XML-LibXML-2.0129 installation issues
2 direct replies — Read more / Contribute
by Brat
on May 24, 2017 at 10:44
    I installed the latest libxml perl library XML-LibXML-2.0129 after downloading it from cpan. my installation did not seem to have gone fine.
    perl Makefile.PL FORCE=1 PREFIX=/app/localapps/perl/lib LIB=/app/localapps/perl/lib enable native perl UTF8 running xml2-config...failed The installed version of libxml2 FORCED 2.9.4 is not compatible with XML::LibXML (and probably buggy)! You may continue at your own risk using 'perl Makefile.PL FORCE=1', bu +t: - don't expect XML::LibXML to build or work correctly! - don't report errors! - don't send patches! Check the README file for more information on versionsthat are tested +with XML::LibXML Checking for ability to link against xml2...yes Generating a Unix-style Makefile Writing Makefile for XML::LibXML Writing MYMETA.yml and MYMETA.json

    The issue with the installation is that I cannot find Not sure How I can fix this. Please help. what am I missing?

Parsing .txt into arrays
6 direct replies — Read more / Contribute
by Fshah
on May 24, 2017 at 05:32

    Hi there, I have a large log file in .txt format which has several pages (each page has different format,with different colums pertaining to different fields) I want to separate each column into an array so that it can be used to process later ,how do I do this I have a picture of how log file looks let me know if u need it, Thanks in advance

    place and year data: 67 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |no.| name | age | place | year | |_ _|_ _ _ _|_ _ _ | _ _ _ | _ _ | |1 | sue |33 | NY | 2015 | |2 | mark |28 | cal | 2106 | work and language :65 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |no.| name | languages | proficiency | time taken| |_ _| _ _ _| _ _ _ _ _ |_ _ _ _ _ _ _| _ _ _ _ _ | |1 | eliz | English | good | 24 hrs | |2 | susan| Spanish | good | 13 hrs | |3 | danny| Italian | decent | 21 hrs | Position log | | |Pos |value | |bulk|lot| prev| newest| |# |Locker|(dfg) |(no) |nul|val |Id | val |val | ----------------------------------------------------------- | 0| 1| 302832| -11.88| 1| 0|Pri| 16| 0| | 1| 9| 302836| 11.88| 9| 0|Pri| 10| 0| | 2| 1| 302832| -11.88| 5| 3|Pri| 14| 4| | 3| 3| 302833| 11.88| 1| 0|sec| 12| 0| | 4| 6| 302837| -11.88| 1| 0|Pri| 16| 3|

    I want these columns into an array with name as given in the table containing values as in the table. thank you.

Wildcards for a wide array of strings?
7 direct replies — Read more / Contribute
by zarath
on May 23, 2017 at 04:01

    Perl-virgin here, be gentle, pretty please with sugar on top. + English is not my native tongue, but i pride myself in my capabilities (in written English anyways, I sound a bit like a mix of a Frenchman and a German trying to speak American).

    I have a question about the very first script I'm trying to write.

    I'm sure this will be resolved in a jiffy by the first person who bothers to read this whole thing. Will you be that person?

New Meditations
How easy it is to make mistakes
4 direct replies — Read more / Contribute
by tobyink
on May 23, 2017 at 06:54

    So, I don't use CGI a lot much these days. But I recently wrote a little joke module and decided to make a little HTML form interface for it. Plack would be a sensible way of doing the interface, but this was just a silly little thing and I was feeling lazy, so I coded it up using CGI instead. I'd used Moo rather than Moose so speed shouldn't have been an issue.

    I uploaded it to the server and visited the page and it just showed me my Perl source code. Hmmm…

    Create .htacccess and add the following lines:

    Options +ExecCGI AddHandler cgi-script .cgi .pl

    Visit the page again… still showing the source code. Hmmm…

    Scratch head for five minutes. Open up httpd.conf, add:

    <Directory /path/to/my/vhost/> AllowOverride all </Directory>

    Visit the page again… still showing the source code. Hmmm…

    SSH into the server and run ./myscript.cgi from the command line.

    "uniq" is not exported by the List::Util module

    Okay, I must have an old version of it. Upgrade.

    Visit the page again… still showing the source code. Hmmm…

    Stop using List::Util entirely.

    sub uniq { my %seen; grep !$seen{$_}++, @_; }

    Visit the page again… still showing the source code. Hmmm…

    Scratch head for ten minutes. Rename to ./

    Visit the page again… still showing the source code. Hmmm…

    Create the following file and mark it as executable:

    #!/bin/sh echo "Content-Type: text/plain"; echo ""; echo "Hello world";

    Visit my new page… also showing the source code. Hmmm…

    Scratch head some more.

    Consider buying anti-headlice shampoo.

    Finally… FINALLY… after about TWO HOURS I figure out the problem! It was…

Log In?

What's my password?
Create A New User
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (9)
As of 2017-05-29 15:26 GMT
Find Nodes?
    Voting Booth?