Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Seekers of Perl Wisdom

( #479=superdoc: print w/replies, xml ) Need Help??

If you have a question on how to do something in Perl, or you need a Perl solution to an actual real-life problem, or you're unsure why something you've tried just isn't working... then this section is the place to ask. Post a new question!

However, you might consider asking in the chatterbox first (if you're a registered user). The response time tends to be quicker, and if it turns out that the problem/solutions are too much for the cb to handle, the kind monks will be sure to direct you here.

User Questions
Capture::Tiny alternative
1 direct reply — Read more / Contribute
by melezhik
on Mar 24, 2017 at 06:23

    Hi! I have been using Capture::Tiny to read from processes I launch in my scripts. It works pretty good. Good module. The only thing I need more is reading processes STDOUT "in real time" which impossible with this module as it captures all the data till the process exits and then return it. It makes user waits till long running command finishes and don't let him to see it's STDOUT in real time.

    Another solution with well known construction:

    open(my $fh, '-|', $system_command) or die $!;
    while (my $line = <$fh>) {
        # Do stuff with each $line.
    Does not work for me, as it for mysterious reasons _sometimes_ ( some rare cases of $system_command ) it waits forever even when $system_command finishes and return all the data as STDOUT.

    Any suggestions?

    PS both workaround for open(my $fh, '-|', $system_command) or suggestion new IPC related module would be fine.

Joining multiple lines together while parsing
3 direct replies — Read more / Contribute
by Arengin
on Mar 24, 2017 at 06:08

    I have the following code:
    #!/usr/bin/env perl use strict; use warnings; use Data::Dumper; #read input data: my @rows; #set record separator to 3 line feeds. local $/ = "\n\n\n"; while ( <> ) { next unless m/Dumpdata example/; #map key-values out of this 'chunk'. my %row = m/\s*(\w+)\S*\s+(\S.*)/g; push @rows, \%row; } #print whole data structure for debugging: print Dumper \@rows; #define columns and ordering for output: my @output_cols = qw /Info Detail Warning Spec/; #iterate rows foreach my $row ( @rows ) { #print fields selected from output_cols. #use a 'hash slice' - look it up in perl docs. print join ";", @{$row}{@output_cols},"\n"; }

    It works just fine except for the problem, that it ends at the line end.
    If for example Info is on 2 lines I only get the first part in the output.

    Dumpdata example ----------------- Warning bad news here Detail: Some really nice infos these are Info: This is a problem but there is a solution Spec: 2nd of 4

    The expected output for this should be:
    "bad news here"; "Some really nice infos these are"; "This is a proble +m but there is a solution"; "2nd of 4"

    Thanks you Corion for reminding me to post that too.
    My above code would return This is a problem but it should return This is a problem but there is a solution

    Any ideas on how to get this done?

    Thank you so much

Alternative to
2 direct replies — Read more / Contribute
by sectokia
on Mar 24, 2017 at 04:59

    Hi Monks, what is built in to perl dist these days that is meant to take the place of Especially for the HTTP essentials like params and header?

    Also whats with the apparent 'crusade' against Removing it from dist is fine (great even), but it seems that someone is especially out to make it hard to even get installed. I had some older code to run and I noticed that:

    • Bigger dists (think Ubuntu) got on exclusion lists, even thought they package a pre-compiled version of pretty much everything on cpan. In particular its banned as a security threat?!
    • Someone seems to have 'minified' its build scripts on cpan. Previously I believe it had a mechanism that told you gcc/make wasn't installed. Now it just throws pages of errors that would be impossible to interpret for those who don't know about gcc/make.
Using PerlPod Creatively
3 direct replies — Read more / Contribute
by samijoseph
on Mar 24, 2017 at 03:21


    I am interested to understand what this guy did, but i am unable to, can someone please break it down to a newbie

    I've spent the last several days at work, trying to "take over" some work left behind by a departing colleague. I realized we didn't have some of his bash scripting in ansible or in a repo, so I decided this would be a good opportunity to fix all of those problems. After a little while it became clear his script was a set of functions, run in a loop-within-a-loop to iterate through a bunch of things. In the middle, between these two loops, is a pile of inline *PERL* that runs as a bash function and passes data back and forth in all directions. This Perl generates some dynamic SQL commands each loop.

    I hate SQL.

    Okay... read the Perl. Now, it's been a long time and a long way since my last string of PERLs, so i didn't really grok 100% what I was reading, but I got the gist of it. Finally figured out the SQL wasn't the problem.

    Another day goes by, and I finally figure out his code is self documenting! That was what all the little bits were in the perl I didnt get. PerlPod. So now I can figure this out easy . Run it, read the code, make a change, run it... *boom* what?


    It took me several more hours and a few beer, and it finally clicked. He was using PerlPod to document out the code he didn't want to run, and commented out the documenting code to run the code he wanted. What looked commented out, wasn't, and what looked like a pile of variables being set, was just a bunch of commentary. He was using a documentation module for *flow control*.

    What a Hacker. Holy shit. Blew. My. Mind.

    I just got schooled very seriously. It's nice to know I can still improve *that much*, even from where I am.

    When PerlPod is used to document something, anything between control codes is not interpreted but treated as commented text. He would comment out the control codes (thus rendering the text interpretable) on the parts meant to run, which would differ between machines.

Get current time using Joda time format
2 direct replies — Read more / Contribute
by ravi45722
on Mar 24, 2017 at 02:38

    I need to get current time using Joda time format ( Is there any way to get that???

     Ex : "EEE MMM dd HH:mm:ss.SSS yyyy"

    Needed to convert into

     "Fri Mar 24 11:54:55.234 2017"  
parsing a terrible /etc/hosts
3 direct replies — Read more / Contribute
by f77coder
on Mar 24, 2017 at 01:15
    Hello, I'm attempting to parse this monster of a hosts file that is a most un-formatted file with ip4 and ip6 address with comments scattered everywhere. sometimes there are two columns separated by space, sometimes 3 columns, sometimes 4. #SpySweeper.Spy.Cookie

    # 1-800-hostingAS3321069.41.160.0 - 2a02:598:2::1095

    so i want to clean the old file by removing comments, and duplicates, so far i have
    my @array = (); $#array = -1; my @tmp_array = (); $#tmp_array = -1; my @uniq = (); $#uniq = -1; my $i = 0; open(HOST_ORIG,'<', $file_read ) or die "Can't open $file_read: $!"; chomp(@array = <HOST_ORIG>); foreach $i (4...scalar(@array)-1) { (my $local_127, $tmp_array[$i] )=split(" ",$array[$i]); }; close(HOST_ORIG); my %seen; my @uniq = grep {! $seen{$_}++} sort(@tmp_array); open(TEMP, '>', $file_write)|| die "\n error opening file $file_write +\n"; print TEMP "#Hosts file\n"; print TEMP "#Last Modified -> ". localtime() . "\n"; print TEMP "# \n"; print TEMP "# localhost: Needs to stay like this to work\n"; print TEMP "\t localhost\n"; print TEMP "# \n"; foreach $i (1...scalar(@uniq)-1) { print TEMP "\t $uniq[$i]\n"; } close(TEMP);

    it works except when there are 3 or more columns, the 3rd and 4rth columns get wrapped around to a new line like this


    how do i throw away the rest of the line if it exists?


Learning Perl by Doing
2 direct replies — Read more / Contribute
by raywood
on Mar 23, 2017 at 23:04

    I would like to learn Perl by working through specific cases where I need it. This is the first such case. I have a situation much like the one described in an earlier discussion (Extracting blocks of text). Specifically, I have a number of old WordStar files in plain text. Each such file contains multiple .pa-delimited documents (consisting of various numbers of lines and paragraphs of text) that should be broken out into separate files. For example, one of these WordStar files might contain something like this:

    Text text text .pa Other text text text .pa
    In that example, resulting file no. 1 would contain "Text text text," and resulting file no. 2 would contain "Other text text text."

    I assume, but am not certain, that every .pa appears at the left margin, and is followed by no other characters on the same line.

    The earlier discussion suggested this solution, where the delimiter was the word "term" rather than ".pa":

    #! perl -slw use strict; my @array = split 'term', do{ local $/; <DATA> }; shift @array; ## Discard leading null print '---', "\n", $_, "\n" for @array; __DATA__ term { yada yada 12345 () ... } term only occurs here { could be 30 lines here but never that word again until another block starts yadada } term, etc.
    My questions, from that example:

    1. That old discussion mentioned RAM concerns when slurping. My system has 16GB RAM. The files I am working on are small. But I may adapt the solution to other, larger files. When does RAM become an issue?

    2. How would I adapt this solution to refer to a separate input file? In the suggested solution, the Perl code seems to be added to the start of the text file. I would rather have a separate Perl script and specify the target file at runtime.

    3. What would be the best reference source, for purposes of interpreting the few Perl codes suggested in that solution?

    4. Which version of Perl should I install, to run this code?

    Many thanks.
Help converting to a Windows service
1 direct reply — Read more / Contribute
by bajangerry
on Mar 23, 2017 at 14:42

    Hello Monks

    EDIT: Ok so I may have been misleading everyone. I have been able to install the script as a service but for some reason I get an error when running it as a service. What runs perfectly from a command line fails as a service with the below error:

    Error 1053: The service did not respond to the start or control request in a timely fashion

    I have been trying to wrap my head around using Win32::Daemon to have a perl script and/or pp converted executable of the same script run as a service and I can't seem to figure out how this is done. I have tried using the Windows "sc create..." command with the .exe file but that does not work so I am obviously missing the Win32::Daemon portion in the script

    The script simply monitors a network stream and based on the output it will write to a file, see below

    #!/usr/bin/perl use strict; use IO::Socket; use POSIX; use warnings; use Config::Simple; use Win32::Daemon; sub logMonitor{ #subrouting to connect to the PBX log and monitor for +the Emergency calls my ($HOST, $PORT)= @_; OUTER: if (my $sock = new IO::Socket::INET(PeerAddr => $HOST, Peer +Port => $PORT,Proto => "tcp",)) { while (<$sock>) { s/^\0+//; # Remove leading null characters chomp ($_); my $data = substr($_, 1,17); my $event = substr ($data, 2,1); my $hr = substr ($data, 3,2); my $min = substr ($data,5,2); my $year = substr($data, 9,4); my $mon = substr($data, 13,2); my $day = substr ($data,15,2); #print "$hr:$min on $year-$mon-$day \n"; if ($event eq "A") { my $agent = substr($_, 17,4); # print "\n Agent $agent logged in \n" ; my $output = "Agent $agent logged in at $hr:$min on ex +t $year"; filePrint($output); } } else { print "Failed to connect to $HOST on $PORT. Will retry in a mi +nute.\n"; sleep 60; goto OUTER; } } #End of monitor subroutine sub filePrint{ #write data to file with date and time stamp my ($DATA)= @_; print "$DATA\n"; my $dateStamp = strftime '%Y-%m-%d', localtime; my $file = "$dateStamp.log"; my $timeStamp = strftime '%H:%M:%S', localtime; if (-f $file){ open (my $fh,'>>', $file); print $fh "$timeStamp | $DATA\n"; close $file; } else { open (my $fh,'>', $file); print $fh "$timeStamp | $DATA\n"; close $file; } }# End of filePrint routine my $cfg = new Config::Simple(); $cfg->read('config.ini'); my $HOST = $cfg->param("pbx"); my $PORT = $cfg->param("port"); logMonitor($HOST, $PORT); #Open the log monitoring subroutine

    Can anyone explain to me how to use Win32::Daemon with this script to create a service from it? Right now it will run as a command line tool

    many thanks!

DateTime::Format::Flexible; for Log Parse with multiple formatted lines
2 direct replies — Read more / Contribute
by TCLion
on Mar 23, 2017 at 13:54

    Basic Info:Log is in text lines. I am combining all lines and breaking it up to separate tags/lines. The Log files are copies and not being updated currently for development.

    Trying to figure out the best way to get my time from lines in log files. Parsing the line for date time error and error message to output to csv. This process is working on some of my log files but one in particular has 2 different time/date formats. For this reason the code is not picking up the lines not formatted the specific way.

    This line is ok format:
    2017-02-20T09:30:53.177000 20[] 0000000000000000 Error Description

    Second (problem)line
    Mon Feb 20 09:31:25 2017 INFO AGENTEXEC Error Description

    Since I now have 2 formats in 1 log I now have installed DateTime::Format::Flexible. Hoping this will fix my issue and convert one format to match the other.

    #Original Working code prior to 2nd date format #!/usr/bin/perl use IO::File; my $file = 'file.location.log'; open(MYLOG, "<$file") or die "Can't open $file: " . $!; my @mylog = <MYLOG>; close(MYLOG); my $myfixedlog = join("ENDOFLINE", @mylog); # Date 1 ,T 2, Tim +e 3 , 4, 5 , 6 , 7 , + 8 , 9 while ($myfixedlog =~ /([0-9]{4}-[0-9][0-9]-[0-9][0-9])(T)([0-9][0-9]: +[0-9][0-9]:[0-9][0-9])(.\d+\s\d+.\d+.\s)(.{16}\s)(\[\w+\])(\w+\S\s+)( +.+?)(ENDOFLINE)/smg) { $date = $1; $severity = $7; $timestamp = $3; $errormsg = $6; chomp($errormsg); print "New Error Found...\n"; print "0 $0\n"; print "1 $1\n"; print "2 $2\n"; print "3 $3\n"; print "4 $4\n"; print "5 $5\n"; print "6 $6\n"; print "7 $7\n"; print "8 $8\n"; print "9 $9\n"; }

    That was to get the positions for the output.
    Then to get the desired output using this code.

    use 5.18.0; use warnings; use IO::File; use Time::Piece; my $servername = "Server1"; # This is the Location of the Original Log File my $OLF = '\\\server\logs\server.log'; # This is the location of the file parsed my $file = 'server.log'; open(MYLOG, "<$file") or die "Can't open $file: " . $!; my @mylog = <MYLOG>; close(MYLOG); # This is the Location\Name of the file Being Created my $CSVLOG = 'output.csv'; open(OUTLOG, ">>$CSVLOG") ; my @outlog = <OUTLOG>; my $runtimestamp = localtime(time); my $myfixedlog = join("ENDOFLINE", @mylog); print OUTLOG "$OLF\n"; print OUTLOG "Server, Date, TimeStamp, Severity, ErrorMsg, \n"; # Date 1 ,T 2, Tim +e 3 , 4, 5 , 6 , 7 , 8 + , 9 while ($myfixedlog =~ /([0-9]{4}-[0-9][0-9]-[0-9][0-9])(T)([0-9][0-9]: +[0-9][0-9]:[0-9][0-9])(.\d+\s\d+.\d+.\s)(.{16}\s)(\[\w+\])(\w+\S)(.+? +)(ENDOFLINE)/smg) { my $date = $1; my $timestamp = $3; my $severity = $7; my $errormsg = join "",$6, $7, $8; chomp($errormsg); if ($errormsg =~ /DfException/ || $severity eq "error +:" || $errormsg =~ /started in/){ foreach ($errormsg =~ /DfException/ || $seve +rity eq "error:" || $errormsg =~ /started in/) {print OUTLOG "$servername, $date, $timestam +p, $severity, $errormsg \n"}}; # print OUTLOG "$servername, $da +te, $timestamp, $severity, $errormsg, \n" } print OUTLOG @outlog; close(OUTLOG); print "CSV - File Created Log File: $CSVLOG";

    So now trying to add and use new code but cant figure out how to set the desired format or how to have it collect and identify properly as before from original code. This is the added code I have found so far to the code to find locations for $myfixedlog

    DateTime::Format::Flexible; my $dt = DateTime::Format::Flexible->parse_datetime( $date, lang => [' +en'], );

    Ok so I guess the questions are:
    How to declare $date if its already being used (in code i am not yet familiar with)?
    If this code pulls out the date would it have a position or just be placed by $dt per line?
    If the date is pulled and formatted correctly would I still break up my line the same way?

    Looking at the preview it looks like i might have grabbed code for 2 different log files but the basics are the same.
    Any Assistance would be appreciated. And no I did not write the original code but I have modified these listed to work. I am Still new to Perl but it is getting easier the more I do.
    Hopefully this makes sense to someone.

Any reason NV is not marked as POK when accessed as string?
2 direct replies — Read more / Contribute
by vr
on Mar 23, 2017 at 07:22

    Just curious, not a 'problem' which requires solution. Maybe developers thought real numbers stringification is usually approximate, that's why? Or simply nobody cares?

    C:\>perl -MDevel::Peek -e "$x = 0.5; qq/$x/; Dump $x" SV = PVNV(0x38bb4) at 0xd217cc REFCNT = 1 FLAGS = (NOK,pNOK) IV = 0 NV = 0.5 PV = 0xd16014 "0.5"\0 CUR = 3 LEN = 28 C:\>perl -MDevel::Peek -e "$x = 0; qq/$x/; Dump $x" SV = PVIV(0xd200c4) at 0xd2685c REFCNT = 1 FLAGS = (IOK,POK,pIOK,pPOK) IV = 0 PV = 0xd16274 "0"\0 CUR = 1 LEN = 10

    Hence, 'double' is converted to string any time it is required. E.g.:

    use Benchmark qw/ cmpthese /; $x = 42.0; %h = (); cmpthese( -1, { F => sub { $h{ pack 'F', $x } = 1 }, s => sub { $h{ $x } = 1 }, });
    Rate s F s 344025/s -- -86% F 2490475/s 624% --

    I found that one my little application, which maintains kind of 'seen' hash, gets nice boost if hash keys are packed as above, instead of 'just used as they are'. Actually, keys are pixels coordinates, they became real numbers because library I use returns them so. Pixels are usually aplenty and hash is accessed a lot, therefore speed gain was significant. But of course it would be even faster if PV was added on first access and POK was set, i.e. without packing.

Add your question
Your question:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others surveying the Monastery: (10)
    As of 2017-03-24 10:42 GMT
    Find Nodes?
      Voting Booth?
      Should Pluto Get Its Planethood Back?

      Results (300 votes). Check out past polls.