Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

The Monastery Gates

( #131=superdoc: print w/ replies, xml ) Need Help??

Donations gladly accepted

If you're new here please read PerlMonks FAQ
and Create a new user.

New Questions
perl module help
3 direct replies — Read more / Contribute
by janasec
on Apr 26, 2015 at 13:40

    hi I am learning to write automation for running some tests,to begin with I have written the following code.I need to know how I can create a perl module so I can make calls to check if a host is alive

    #!/usr/bin/perl use strict; use warnings; use lib '/home/suse/junk/automation1/emulex'; use Config::Simple; use Net::Ping::External qw(ping); use 5.010; #myconf.cfg is a file with all hosts my $cfg = new Config::Simple('/home/suse/junk/automation1/emulex/mycon +f.cfg'); #accessing values my $host = $cfg->param("host1"); print "checking $host is reachable or not\n"; my $alive = ping(hostname => "$host", count => 5, size => 1024, timeou +t => 3); print "$host is alive!\n" if $alive or die"Could not ping host '$host' + ";
A better way of lookup?
8 direct replies — Read more / Contribute
by BrowserUk
on Apr 26, 2015 at 07:50

    This has been a recurring dilemma down the years.

    Given a contiguous input and a set of break points, find the highest breakpoint lower than the input value and return the associated value.

    sub lookup { my( $v ) = shift; if( $v < 25000 ) return 2500; if( $v < 50000 ) return 5000; if( $v < 150000 ) return 12500; if( $v < 225000 ) return 25000; if( $v < 300000 ) return 37500; if( $v < 600000 ) return 60000; if( $v < 1200000 ) return 120000; if( $v < 3600000 ) return 300000; if( $v < 5400000 ) return 600000; if( $v < 10800000 ) return 900000; if( $v < 21600000 ) return 1800000; if( $v < 43200000 ) return 3600000; if( $v < 64800000 ) return 7200000; if( $v < 129600000 ) return 10800000; if( $v < 216000000 ) return 21600000; if( $v < 432000000 ) return 43200000; if( $v < 864000000 ) return 86400000; if( $v < 1728000000 ) return 172800000; if( $v < 3024000000 ) return 345600000; if( $v < 6048000000 ) return 604800000; if( $v < 12096000000 ) return 1209600000; if( $v < 31557600000 ) return 2629800000; if( $v < 63115200000 ) return 5259600000; if( $v < 78894000000 ) return 7889400000; if( $v < 157788000000 ) return 15778800000; return 31557600000; }

    Simple. Efficient. Not very pretty. Is there a better way?

    • I could stick the values in hash, iterate the keys and return the value:

      but that requires either sorting the keys each time or keeping a sorted array of the keys and duplicating memory.

    • I could put the break points and values in parallel arrays;

      but ... parallel arrays?

    • I could use an array of pairs (AoA):

      But looping over the double indirection isn't particularly efficient.

    Then there's the search method. Most time the set isn't big enough to warrant coding a binary search in place of a linear one. Most times efficiency isn't a particular concern, but in this case, the routine is called as part of a redraw function when rotating stuff on screen, so it can be called many times a second.

    Basically, there are several ways of doing it, but none of them are particularly satisfying, and I'm wondering if anyone has discovered a nice way that I haven't covered?

    (The final twist is that this is destined for JavaScript; so if any JS guys know of a good method that language supports; I'd be happy to hear of it. Perhaps off-line.)

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
    In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Net::SSH2 channel returns no output
1 direct reply — Read more / Contribute
by rama133101
on Apr 25, 2015 at 16:08

    I need to send some commands to remote linux box and fetch responses. I am using Net:SSH2 for creating a session channel and send commands using shell.

    The problem I face is that the response I receive is empty. If I add a sleep statement, the output is very well captured. I do not want to add sleep statements as I need to send multiple commands and that would reduce the performance. Please advice what I am missing.

    Here is the code.

    $session = Net::SSH2->new(); $rc = $session->connect($target_ip, $target_port, Timeout=>4000) ; print "\n rc: $rc"; $session->auth_password($username, $passwd) ; $chan = $session->channel(); $chan->shell() ; print $chan $cmd . " \n" ; my @poll = ({handle=>$chan, events=>['in', 'ext', 'channel_closed']}); $session->blocking(0) ; $session->poll(1, \@poll) ; if ($poll[0]->{revents}->{in}) { while (<$chan>) { $resp .= $_ ; } } print "\nresponse : $resp";
Wait for individual sub processes [SOLVED]
6 direct replies — Read more / Contribute
by crackerjack.tej
on Apr 25, 2015 at 03:01

    Dear monks,

    I am essentially writing a Perl script that divides a large input file for a text processing tool, so that I can process the files faster. I am working on a CentOS 6 based cluster, where each CPU has 16 cores. My idea is to split the input file into 16 parts, and run 16 instances of the text processing tool, and once all of them are done, I parse the output and merge it into a single file. In addition, the script will continue to process the next input file in a similar way. I have achieved that using fork(), wait() and exec() as follows (Omitting code that is not relevant):

    use strict; use warnings; use POSIX ":sys_wait_h"; #Split input files into parts and store the filenames into array @ +parts ... my %children; foreach my $part (@parts) { my $pid = fork(); die "Cannot fork for $part\n" unless defined $pid; if ($pid == 0) { exec("sh text_tool $part > $part.out") or die "Cannot exec + $part\n"; } print STDERR "Started processing $part with $pid at ".localtim +e."\n"; $children{$pid} = $part; } while(%children) { my $pid = wait(); die "$!\n" if $pid < 1; my $part = delete($children{$pid}); print STDERR "Finished processing $part at ".localtime."\n"; }

    While I got what I wanted, there is a small problem. Due to the nature of the text processing tool, some parts get completed much before others, in no specific order. The difference is in hours, which means that many cores of the CPU are idle for a long time, just waiting for few parts to finish.

    This is where I need help. I want to keep checking which part (or corresponding process) has exited successfully, so that I can start the processing of the same part of the next input file. I need your wisdom on how I can achieve this. I tried searching a lot on various forums, but did not understand correctly how this can be done.



    Using a hash, I can now find out which process is exiting when. But I fail to understand how to use this code in an if block, so that I can start the next process. Can someone help me with that? I have updated the code accordingly.

    ----------------UPDATE 2--------------

    I guess it's working now. Using Parallel::ForkManager, and a hash of arrays that stores the pids of each input file, I am able to track the sub processes of each file separately. By maintaining a count of number of subprocesses exited, I can call the sub for output parsing as soon as the count reaches 16 for an input file. I will come back if I run into any other problem.

    Thanks a lot for all the help :)

    P.S. Is there any flag that I have to set that this thread is answered/solved?

Out of Memory Error : V-Lookup on Large Sized TEXT File
7 direct replies — Read more / Contribute
by TheFarsicle
on Apr 24, 2015 at 09:14
    Hello perlmonks,

    I am newbie to Perl & working on the Perl script to perform an action similar to V-Lookup.


    As an input I have some large sized text files around 200 MB. These text files are to be searched for all the records present in the another file, say Reference.txt (This file is normally not more than one MB)

    I have written script to find all the lines present in these large sized files based on text (string values) in Reference.txt file. All the found records are then written into a new file per each large file iteration.

    The script works fine for normal size like 30-40 MB but when it goes more than 100 MB or so. It throws out of memory error.

    I have designed these operations as subroutine and calling them.

    The code goes something like this...

    open (FILE, $ReferenceFilePath) or die "Can't open file"; chomp (@REFFILELIST = (<FILE>)); open OUTFILE, ">$OUTPUTFILE" or die $!; foreach my $line (@REFFILELIST) { open (LARGEFILE, $LARGESIZEDFILE) or die "Can't open File"; while (<LARGEFILE>) { my $Result = index($_, $line); if ($Result > 0) { open(my $FDH, ">>$OUTPUTFILE"); print $FDH $_; } } close(LARGEFILE); } close(OUTFILE); close(FILE);

    Can you please guide me on where I am going wrong and what would be the best way to address this issue?

    Thanks in advance.


DESTROY and AUTOLOAD in 5.20.1
4 direct replies — Read more / Contribute
by szabgab
on Apr 24, 2015 at 05:36
    Given this script:
    use strict; use warnings; use 5.010; use Greeting; say 'Hi'; { my $g = Greeting->new; } say 'Bye';
    and this module:
    package Greeting; use strict; use warnings; use 5.010; use Data::Dumper; sub new { my ($class) = @_; return bless {}, $class; } sub AUTOLOAD { our $AUTOLOAD; say $AUTOLOAD; } DESTROY { say 'destroy'; } 1;
    I can see the word "destroy" printed as I would expect. However, if I remove the DESTROY from the module I don't see AUTOLOAD being called instead of the missing DESTROY. I only checked it with 5.20.1 but I wonder what am I missing here?


    Reported with perlbug as RT #124387
Getting an unknown error
5 direct replies — Read more / Contribute
by andybshaker
on Apr 23, 2015 at 10:02

    Basically, different arrays have different pieces of information in them and I have to go from one to another to another from that. In this case, I have to go from each element in @Genes and extract its corresponding element from a long file which I read in as @lines. I keep getting a strange error that reads, syntax error at line 38, near "$N (" Does anyone know what this is? Here is the code. </p?

    my @Genes = qw(A B C D) my @ptt = ("19384..003059 0 - - A","203581..39502 0 + - B) my @contig = (); my @Coordinates; my @Number; my $R; foreach my $G (@Genes){ for my $x (0..$#ptt){ if($ptt[$x] =~ /$G/){ push(@Coordinates,"$ptt[$x]"); print "$ptt[$x]\n";} } } foreach my $C (@Coordinates){ push (@Number, split(" ", $C));} my %hash = (); my $file = "scaffold_contig.txt"; open(IN, "<$file") or die "Cannot open file $file\n"; my @lines = <IN>; foreach $1 (@lines){ chomp($1); my %columns = split(">", $1);} close(IN); print "$lines[1];\n" foreach my $N (@Number){ for $R (0..$#lines){ if($lines[$R] =~ /$N/){ print "lines[$R]\n" } } }

    Here is line 38: foreach my $N (@Number){

Retrieving content from couchdb using CouchDB::Client
3 direct replies — Read more / Contribute
by shivam99aa
on Apr 23, 2015 at 09:36

    I am able to create new documents using CouchDB::Client as well as able to verify if any doc is present. What i am not able to do is to retrieve the contents of any doc. The reason is i am not able to get the correct syntax which is to be used for it. I am not a perl genius so taking a look at the source code did not help me either.

    use warnings; use CouchDB::Client; my $c = CouchDB::Client->new(uri => ''); my $db = $c->newDB('test'); my $doc = $db->newDoc('12345', undef, {'foo'=>'bar'})->create; if ($db->docExists('12345')){ print "hello\n"; } #my $doc=CouchDB::Client::Doc->new($db); print $doc->retrieve('12345');

    I am able to create document but then i need to comment that line on next run as this will give storage error. But after commenting i have no way to retrieve the doc as i have no object remaining. But this should not be the constraint as there should be a way to retrieve doc using the db object by giving id to it.

Regex for files
5 direct replies — Read more / Contribute
by bmcquill
on Apr 22, 2015 at 22:01
    I'm trying to get better at regex and I'm starting with Perl. I want to be able to go through a directory and find all the files that begin with messages and MAY have a "." and a digit behind it, but it should not match something that has say .txt, .pl, etc. Any assistance is greatly appreciated. I want to find all the files that are messages, messages., messages.1, etc. but NOT messages.txt or Does that help?
Best practice for sending results to a user via email
4 direct replies — Read more / Contribute
by Anonymous Monk
on Apr 22, 2015 at 16:30
    Dear Monks,
    I hereby ask your wisdom on the following problem:
    I have set up a simple web-server in PHP, with a submission form (textarea). When the user submits the form, the contents are put into a file and a perl script is being executed on the file. The output of the script is written in a text file at the end, also an image is being produced.
    My question has two parts:
    1) Because this script takes some time to execute, I think it is not a good practice to just let it run on the web-server, since there is great possibility it will hang and then it will not output anything. So I thought the best is to just get the input from the user and then just send an email to him saying "your work is completed" with a link that will be an HTML page with the results.
    Does this sound reasonable practice to you?
    2) Can you give me some hints as to how such a thing is accomplished? I mean, what steps should I follow and maybe point me to some examples of sending emails to user that can direct the user to an HTML page with the final results?
    Thank you in advance!
Reading Text File into Hash to Parse and Sort
8 direct replies — Read more / Contribute
by Perl_Derek
on Apr 21, 2015 at 14:49
    Hello, I am just getting started with Perl and I have not used any programming languages in the past. I hope you can help me with this.

    I am not able to effectively use hashes to parse, sort or replace data from a text file. I have read through a couple of books (Learning Perl, Perl by Example) and browsed Perl websites, but I am still having difficulty grasping the concepts. For example, I am trying to take a text file and sort the results in any way I choose. I can't seem to find a real-world example where I read a file into a hash and parse/sort and/or replace data. Here is my code for parsing data using arrays which works:


    use strict;

    use warnings;

    open (FILE, '<', 'FB_LPWAP.txt') or die ("ERROR: Could not read file");

    print <FILE>; #Prints entire file

    while (<FILE>){

    my @array = split /\t/, $_;

    print "$array[0]\t"; #Date

    print "$array1\t"; #Closing Price

    print "$array2\n"; #Weighted Average Price


    close (FILE);

    I still have a lot to learn, but I am hoping someone could help point me in the right direction as I have not be able to use hashes on a text file that has multiple columns of data without receiving a number of syntax/compilation errors. The text file I have has multiple columns and I would like to parse certain columns using hashes. I also have would like to sort the data. I don't know how to reference my columns. Here is the sample data in my test file:


    ABC | ABC Co.| 15.5| 5000| Industrials

    AB | Alpha Beta| 12| 2500| Materials

    DOZ | ZZZZZ| 5.05| 2800| Telecom

    DX | DX Co.| 77.2| 12000| Industrials

    DXX | DXX Co.| 50.25| 9000| Utilities

    Thank you.
Bizarre and provocatively irritating cpantester reports
2 direct replies — Read more / Contribute
by syphilis
on Apr 21, 2015 at 10:20

    I'm looking at this fail report for a module I've written and I'm seeing (in both Ubuntu's Firefox browser and Microsoft's IE8 browser):
    GMPz.xs: In function âRmpz_cdiv_q_2expâ: GMPz.xs:489: error: âmp_bitcnt_tâ undeclared (first use in this functi +on)
    The first thing that strikes me is that, in GMPz.xs, there is no such function as "âRmpz_cdiv_q_2expâ".
    The second thing that strikes me is that, in GMPz.xs, there is no declaration of "âmp_bitcnt_tâ".

    Sure, there's a function called "Rmpz_cdiv_q_2exp", and there's an "mp_bitcnt_t" data type declared - and, if the complaint was about the declaration of the "mp_bitcnt_t" data type then I would simply assume that Bingos had just set up yet another smoker that contained some antiquated build of gmp whose headers did not define the "mp_bitcnt_t" data type.
    And I'd probably then go ahead and fix that shortcoming in Math::GMPz.

    But ... no, the complaint is clearly and definitively about a non-existent "âmp_bitcnt_tâ" data type.

    I'm not the first person to have been subjected to such fuckbrained garbage, and I'm sure that lots of monks will have seen similar before.
    But why does this happen ?

New Meditations
Refactoring Perl5 with XS++
3 direct replies — Read more / Contribute
by rje
on Apr 25, 2015 at 01:06

    Last time I mused aloud about "refactoring" Perl, I referenced Chromatic's statement/challenge:

    "If I were to implement a language now, I'd write a very minimal core suitable for bootstrapping. ... Think of a handful of ops. Think very low level. (Think something a little higher than the universal Turing machine and the lambda calculus and maybe a little bit more VMmy than a good Forth implementation, and you have it.) If you've come up with something that can replace XS, stop. You're there. Do not continue. That's what you need." (Chromatic, January 2013)

    I know next to nothing about XS, so I started reading perldoc.

    It seems to me that XS is a low-level language that provides the structures that Perl uses, but without the syntactic sugar. Yes, it primarily maps types between C and Perl. But it does more than that.

    Now this low-level language may be very nearly self-extending. I'll have to read more about XS.

Want to make an Everything site of your own?
1 direct reply — Read more / Contribute
by thomas895
on Apr 20, 2015 at 03:48

    Ever wanted to experiment with the engine PerlMonks is built on? I did, but it's rather difficult to install, so I thought I'd write this for anyone who wanted to give it a go themselves.

    My Perl is v5.16.2. YMMV for others.


    • A MySQL/MariaDB server

    • GNU patch

    • C compiler

    • Some knowledge of how Apache works

    Estimated time: a quiet evening or so

    1. Download Apache 1.3.9 and mod_perl 1.14 from your nearest mirror, then unpack them. You may use other versions, but this guide won't apply to them.

    2. I wanted to install it all to my home directory. I ran mod_perl's Makefile.PL like so:

      perl Makefile.PL APACHE_SRC=../apache_1.39/src APACHE_PREFIX=$HOME/opt/apache1.3.9 DO_HTTPD=1 USE_APACI=1 EVERYTHING=1 PREFIX=/home/thomas/perl5
      Adjust as needed.
    3. If you have a relatively new version of gcc and a Perl v5.14 or newer, you will need to make some changes to the source. Apply this patch file to the mod_perl directory, and this one to the apache directory. It does the following (you can skip these details if you want):

      • In v5.16, $<; and friends are no longer cached. I just tried removing the problematic section that used these variables, and that seemed to work. You might not be able to run your server as root (which requires being able to change its own permissions), but I haven't checked.

      • For some reason, it was a trend to supply your own version of getline (maybe the libc one was broken, haven't looked it up) in those days. In any case, gcc complains about it, so I updated all of the code to use the Apache one. (it only affects the password utility, which is not really needed in our case, but it does cause make to fail)

      • In v5.14, you can't use the result of GvCV and friends as lvalues anymore, so I replaced the places where something was assigned to the result of that function with the appropriate _set macro, as the delta advises.

    4. Run make and make install, and go make some coffee. You can make test, too, but then also grab a sandwich.

    5. Try to start Apache as make install's instructions suggest, to make sure it works. You may need to choose a different port number, do so with the Listen and Port options in httpd.conf

      • If you installed Apache locally, you will need to modify apachectl and/or your shell startup script: make sure that the PERL5LIB environment variable is set to where mod_perl's Perl libraries are installed.

      Now for Everything else...

    6. Download this, unpack it, and follow QUICKINSTALL up to (but not yet including) the install_esite

      • When running Makefile.PL, if you want to install locally, don't forget to set PREFIX accordingly.

      • It is not necessary to let it append things to your httpd.conf, in a later step I'll show you why and what to do instead.

    7. If you have a modern mysql/mariadb, some of the SQL scripts won't work. Here is another patch to fix them.

      • It mostly has to do with the default values of the integer columns: by getting rid of the default value of a quoted zero, mysql accepts it.

      • There is also a timestamp column that has a size in the script, but mysql doesn't like that, so by getting rid of it, it works again.

    8. Now run install_esite, as QUICKINSTALL says.

    9. For some reason, only showed up as text, perhaps due to the other mod_perl settings I'd been playing with, or perhaps it was something else. I added this to httpd.conf, and then it worked:

      PerlModule Apache::Registry PerlModule Apache::DBI PerlModule CGI <Files *.pl> SetHandler perl-script PerlHandler Apache::Registry Options +ExecCGI PerlSendHeader On PerlSetupEnv On </Files>
    10. (Re)start Apache, visit /, and have lots of fun!

    If something doesn't work for you, post it below.

    Happy hacking!

    Edit: forgot the third patch

    "Excuse me for butting in, but I'm interrupt-driven..."
    Did you know this software was released when I was only 3 years old? Still works, too -- I find that amazing.
Log In?

What's my password?
Create A New User
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (13)
As of 2015-04-26 21:22 GMT
Find Nodes?
    Voting Booth?

    Who makes your decisions?

    Results (488 votes), past polls