Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Seekers of Perl Wisdom

( #479=superdoc: print w/ replies, xml ) Need Help??

If you have a question on how to do something in Perl, or you need a Perl solution to an actual real-life problem, or you're unsure why something you've tried just isn't working... then this section is the place to ask. Post a new question!

However, you might consider asking in the chatterbox first (if you're a registered user). The response time tends to be quicker, and if it turns out that the problem/solutions are too much for the cb to handle, the kind monks will be sure to direct you here.

User Questions
DBIx::Class Looping thru resultset
1 direct reply — Read more / Contribute
by phildeman
on Apr 29, 2016 at 16:04

    Hi All,

    Can variables be used in place of column names when looping through DBIx::Class array of objects.
    For example:

    my @db_colnames = ('lastname','firstname','middlename'); my @rows = $schema->resultset( 'myTable' )->search({}, {order_by => [qw +/ lastname firstname middlename /] } ); foreach my $row (@rows) { foreach my $col (@db_colnames) { print $row->$col ."\t"; } print "\n"; }

    No value printed. However, when I used the actual column name to retrieve the data, it worked fine. For example:

    . . print $row->lastname . "\t" . $row->firstname . "\t" . $row->middlenam +e . "\t"; print "\n"; . .

    Thanks for your help.

Using Positions from one array to find the value in the same position of another.
3 direct replies — Read more / Contribute
by RuZombieSlayer
on Apr 29, 2016 at 15:41

    Hello, I want to use the values from one array to search for positions in another and return what was found at those positions, the code below may give you and idea of what i am trying to achieve. Basically input a sentance and have each word turned into a 6 digit number to show that words position in an array then find the value of something in another array at the same position. I am new to pearl have been learning for a few weeks. I am teaching my self and am hoping to learn through this project i have set myself. Sorry if the code is a bit sloppy or not as sleek as it could be.

    use stict; use warnings; my @words = ("a", "abbey", .. "zymophore", "zymophoric"); #in reality, this is a list of all words in the english #language apro +ximatly 400,000 words long my $sentance = <>; my @sentanceSplit = split(' ', $lcsentance); #all words need to be lower case #for simplicity, say we type "a zymophoric". my @indexes; foreach my $place (@sentanceSplit) { push @indexes, sprintf '%06d', first_index { $_ eq $place } @words +; }; #because of our input, @indexes will print 000000 399999 #because 'a' is in the 0th position of @words therfore #000000 #and 'zymophoric' is in the 400 thousandth position #retuning 399999 my @numbers2 = ("250690", "004058", ..); #400,000 numbers but in a random order instead of #(0..399999) #my goal now is to take the values from @indexes and #compare them to +@numbers2 and say (values of @indexes are #0 399999) notice we remove + the leading zeros. Then look #for what is foreach in the oth positio +n of @numbers2 ie. #"250690" and the 399999th position returning what +ever it #finds lets say 000001 #then have my @new = whatever the result is, so ("250690", #"000001");

    I hope i have given you an idea what i am trying to achieve, I am not sure how to achieve this, perhaps hashes? eitherway an example would be very helpful thanks!!

Parse multiline logfile
1 direct reply — Read more / Contribute
by chris2013
on Apr 29, 2016 at 15:04
    Dear All,

    I'd like to parse a Charon (StrongSwan) logfile.

    Somewhere it logs a failed login. If this is found, I'd like to lookup the username and IP address, that are mentioned some lines above.

    What's the best approach? Is there any framework that can do that? Where I only have to fill-in the regex?

    Should I read the whole logfile to an array and then go back? Or is there something like grep? I've tried to use a shell script and let perl one-liners do the regex stuff. Unfortunately I couldn't escape the single quotes in front and after the e-mail address. They have to be quoted because of the shell and the regex. Tried to write them to an evironment variable but that didn't work. Probably, there are completely different and better solutions.

    export MYP="identity '([A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,})'" EMAIL=`grep -B15 "$DATE" $LOGFILE | grep "$CONID" | grep identity | pe +rl -nle 'print "$1" if /$ENV{MYP}/'`


    Apr 29 10:01:10 charon: 07[NET] <con1|33> received pac +ket: from[55842] to[4500] (80 bytes) Apr 29 10:01:10 charon: 14[IKE] <con1|33> received EAP + identity '' Apr 29 10:01:11 charon: 14[IKE] <con1|33> EAP-MS-CHAPv +2 verification failed, retry (1)

    - Chris

Creating pointers in a HoH declaration
2 direct replies — Read more / Contribute
by cmv
on Apr 29, 2016 at 14:38
    Hi Monks-

    I'm a little embarrassed to ask this question, as I feel I should know the answer - however, I haven't been able to come up with a good answer (by searching or thinking) for 2 days! Please help - my mind is going...

    I want to declare a hash-of-hashes, where some of the elements point to each other. Is it possible to do it in a declaration? Here is some code that makes it happen outside the declaration.

    Any help is much appreciated!



    use strict; use warnings; use Data::Dumper; my %fields = ( version1 => { 1 => 'Field 1', 2 => 'Field 2', 3 => 'Field 3', }, version2 => { 1 => 'Field 1', 2 => 'Field 2', 3 => 'Field 3', 4 => 'Field 4', }, version3 => { 1 => 'Field 1', 2 => 'Field 2', 3 => 'Field 3', 4 => 'Field 4', 5 => 'Field 5', }, version4 => { # What if version4 is exactly the same as version3. # How do I just make version4 point to verson3? }, ); print STDERR "fields Dump1:\n", Dumper(\%fields), "\n"; $fields{version4} = $fields{version3}; print STDERR "fields Dump2:\n", Dumper(\%fields), "\n";
Filtering array of strings and numbers
3 direct replies — Read more / Contribute
by nysus
on Apr 29, 2016 at 11:28

    I'm stuck on what I think should be a simple problem. I want to filter out elements from an array. I don't know if they array contains strings or numbers or both. And I don't know if the value I want to filter out is a string or number:

    my $string_or_number; my @filtered = grep { $string_or_number ne $_ } @strings_or_numbers;

    What's the best way to accomplishing this without throwing an error if the wrong equality operator is used?

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

Detached forking in a CGI script
3 direct replies — Read more / Contribute
by xylose
on Apr 29, 2016 at 11:06

    I'm having a problem satisfying a number of constraints in a CGI script I'm writing.

    I need the script to fork a child process and to get hold of the pid of this child, but to then not make the parent (the CGI process) wait for the child, and allow it to exit immediately.

    One extra complication is that in the child I exec a new program (an R script), but capture STDOUT and STDERR into log files, so I can't close those filehandles before doing the exec.

    I've tried a number of different approaches, but anything which gives me the pid and starts the child process correctly always makes the parent hang until the child is done.

    My current (not working) best guess is:

    $SIG{CHLD} = 'IGNORE'; my $pid = fork(); if ($pid) { # We're the child and we need to start the analysis # First we'll write out pid into a file in the run # folder so that the results tracker can tell if we've # died. open (PID,'>','pid.txt') or die "Failed to write to pid file : $!" +; print PID $pid; close PID or die "Failed to write to pid file: $!"; exec("Rscript analysis.r > log.txt 2>errors.txt"); exit(0); } print $q->redirect("script.cgi?job_id=$job_id");

    In this version the child runs, but the parent doesn't exit until the child has completed.

    Any ideas how I can have my cake and eat it?


Another concurrent HTTP requests script - code review request
No replies — Read more | Post response
by alanek
on Apr 29, 2016 at 06:02
    Hi PerlMonks! After years of using Parallel::ForkManager to speed up making thousands of HTTP request, I was adviced by Ikegami at to give Net::Curl::Multi a try. And I have to say its awesome! Now, becouse its my first time using it, and becouse final version of the code will have to deal with external, 3rd party API, I would really want to be sure its working fine and doesnt have any bugs. Below You will find my test script based on Ikegami's advice. It works, I dont have any problems with it, but maybe some more experienced Perl users would notice something that I didnt, or tell me if I shouldn't do something like I did, etc. My main concern is the LINE loop and way that I add URL to N::C::M queue, fill the queue with more URLs, wait for one of them to finish, and go to the next URL and add it. Some notes:
    • Im using external txt file with URLs becouse there might be A LOT of them (hundret of thousands even) so I dont want to load everything to array.
    • As I don't fully understand how Net::Curl::Multi's concurrency works, please tell me whether I should expect any problems with putting MySQL UPDATE command (via DBI) inside RESPONSE loop (besides higher server load obviously - I expect final script to run with at least 50 concurrent N::C::M workers)
    #!/usr/bin/perl use Net::Curl::Easy qw( :constants ); use Net::Curl::Multi qw( ); sub make_request { my ( $url ) = @_; my $easy = Net::Curl::Easy->new(); $easy->{url} = $url; $easy->setopt( CURLOPT_URL, $url ); $easy->setopt( CURLOPT_HEADERDATA, \$easy->{head} ); $easy->setopt( CURLOPT_FILE, \$easy->{body} ); return $easy; } my $maxWorkers = 10; my $multi = Net::Curl::Multi->new(); my $workers = 0; my $i = 1; open my $fh, "<", "urls.txt"; LINE: while ( my $url = <$fh> ) { chomp( $url ); $url .= "?$i"; print "($i) $url\n"; my $easy = make_request( $url ); $multi->add_handle( $easy ); $workers++; my $running = 0; do { my ($r, $w, $e) = $multi->fdset(); my $timeout = $multi->timeout(); select $r, $w, $e, $timeout / 1000 if $timeout > 0; $running = $multi->perform(); RESPONSE: while ( my ( $msg, $easy, $result ) = $multi->info_r +ead() ) { $multi->remove_handle( $easy ); $workers--; printf( "%s getting %s\n", $easy->getinfo( CURLINFO_RESPON +SE_CODE ), $easy->{url} ); } # dont max CPU while waiting select( undef, undef, undef, 0.01 ); } while ( $workers == $maxWorkers || ( eof && $running ) ); $i++; } close $fh;
Execute a command line and save in memory its output
2 direct replies — Read more / Contribute
by Anonymous Monk
on Apr 29, 2016 at 04:28


    I'm reading a netCDF file to extract a timeseries using a command line named ncks.

    I dump its output in a temporary file and then I read the temporary file to extract the values and push them into an array.

    Is there a way to store in memory the extracted records (@tmplines in the code below) without writing them on a temporary file?

    Since I have to call the subroutine thousands of times I imagine that it would speed up things.

    Thanks for any suggestion and best regards.

    use strict; use IPC::Run qw( run ); use Tie::File; use Fcntl 'O_RDONLY'; # ... cut ... my @ts = get_time_series(File=>$ncfile,Variable=>$ncvar,Id=>$nc_id); # ... cut ... sub get_time_series { my %args = @_; my $file = $args{File}; my $variable = $args{Variable}; my $id = $args{Id}; my $tmpfile = "$id.tmp"; # Example: # ncks -v vmro3 -d station,1,1 | grep "station\[" # my $command = qq(ncks -v $variable -d station,$id,$id $file | grep + \"station\\[\" > $tmpfile); run $command; # This will write in file the following records: #station[3385]=3386 #time[0]=3287.02083333 station[3385]=3386 vmro3[3385]=1.53524e-08 mole + mole-1 #time[1]=3287.0625 station[3385]=3386 vmro3[7513]=1.55109e-08 mole mol +e-1 #time[2]=3287.10416667 station[3385]=3386 vmro3[11641]=1.56481e-08 mol +e mole-1 my @ts; tie my @tmplines, 'Tie::File', $tmpfile, mode => O_RDONLY; foreach my $ii (1..$#tmplines) { my $line = $tmplines[$ii]; my @pa = anytrim(split(/\s/,$line)); my ($dummy,$value) = split(/=|\s/,$pa[2]); push @ts, $value; } unlink $tmpfile or die "$tmpfile: $!"; return @ts; }
Any module for saving data as perl data?
3 direct replies — Read more / Contribute
by PerlBroker
on Apr 29, 2016 at 03:25
    Is there any module or way, to simply save data as Perl, for example if I have $a = "hello", that I can save it in exact that form in a file: $a = "hello"; including the same for hashes and arrays?
count the occurrence of second line of a paragraph in a file
2 direct replies — Read more / Contribute
by umaykulsum
on Apr 29, 2016 at 01:02
    I have multiple files like data.txt:
    I want to print the whole paragraph (four lines) and the count of second line in the file. The output for above file should be:
    The code which I have written is:
    use strict; use warnings; my @files=('data.txt'); for my $input_file (@files) { my $output_file = $input_file.".out"; process_file($input_file, $output_file); } sub process_file { my($input_file, $output_file) = @_; my %count; my $file = shift or die "Usage: $0 FILE\n"; open my $fh, '<', $file or die "Could not open '$file' $!"; open my $fa, '>', $output_file; $/=""; while (my $line = <$fh>) { foreach my $str ($line) { chomp $line; $count{$str}++; } } foreach my $str (sort keys %count) { printf $fa "%-s %s", $str."\t", $count{$str}; print $fa ":".$input_file."\n"; } }
    This gives the count of whole paragraph instead of the second line. It matches the whole paragraph but I want to print the whole paragraph and count of only the second line.

Add your question
Your question:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others meditating upon the Monastery: (7)
    As of 2016-04-29 21:17 GMT
    Find Nodes?
      Voting Booth?
      :nehw tseb si esrever ni gnitirW

      Results (441 votes). Check out past polls.