Beefy Boxes and Bandwidth Generously Provided by pair Networks RobOMonk
P is for Practical
 
PerlMonks  

The Monastery Gates

( #131=superdoc: print w/ replies, xml ) Need Help??

Donations gladly accepted

If you're new here please read PerlMonks FAQ
and Create a new user.

Want Mega XP? Prepare to have your hopes dashed, join in on the: poll ideas quest 2013  (Don't worry; you've got plenty of time.)

New Questions
IO::Uncompress::Gunzip to scalar takes hours (on windows)
No replies — Read more | Post response
by cmv
on May 21, 2013 at 11:40
    Hi Monks-

    I'm seeing a strange issue when using IO::Uncompress:Gunzip on windows that I can't explain. The same code runs fine under Unix. I'd like to understand why.

    The code below will take about 20-seconds to unzip a 65M zipped file on my windows xp box.

    If I comment out the first gunzip line, and uncomment the second gunzip line, it will take hours on the windows box (6 hours last time I bothered to try). Any thoughts?

    Thanks

    -Craig

    #!/opt/exp/bin/perl5.8 use strict; use warnings; use IO::Uncompress::Gunzip qw(gunzip $GunzipError) ; my $ifile = shift || die "Missing input file name\n"; open(IFILE, "<$ifile") || die "Can't open $ifile"; binmode(IFILE); my $inputstr = join('', <IFILE>); print STDERR scalar(localtime), " - STARTING UNZIP!!!\n"; # This is fast on all platforms gunzip \$inputstr => "clearfile" or die "gzip failed: $GunzipError\n"; my $cleartxt; # This takes forever on windows, no problem on Unix #gunzip \$inputstr => \$cleartxt or die "gzip failed: $GunzipError\n"; print STDERR scalar(localtime), " - UNZIPPED COMPLETE!!!\n";
How to improve this data structure?
4 direct replies — Read more / Contribute
by fiddler42
on May 21, 2013 at 10:31
    Hi,

    I recently made the rookie mistake of authoring a perl script with a sample data input file that was not realistic in size. So now I am running the script with much larger data input files and it is painfully slow. Here is the root cause of the problem:-

    1) A data input file is parsed, and each line of the file is added to an array like so:-

    push (@StatsArray, {RegionNum=>$RegionNum, AR=>$AR[$RegionNum], BCR=>$BCR[$RegionNum]});
    I like the data structure above because it is easy to understand and I can add new elements to the array as the script becomes more sophisticated. I included only 3 of 15 variables, and I expect the number of variables to increase a little more.

    2) Once the array is fully populated (millions of entries) and I need to pluck data from it, I need to do two things: a) *always* numerically sort by RegionNum first and then b) numerically sort some other key. Here is an example:-
    foreach $RegionCoords (@AllRegionCoords) { $RegionNum++; foreach (sort {$$a{RegionNum} <=> $$b{RegionNum} or $$a{AR} <=> $ +$b{AR}} @StatsArray) { $RegionKey = $$_{RegionNum}; $ARKey = $$_{AR}; if ($RegionKey == $RegionNum) { # do stuff with sorted AR data in region n } } }
    Again, easy to follow. The problem is there are many different region numbers and I am not breaking down the data into smaller chunks. For example, although there may be 1 million entries in @StatsArray, all of the entries are made up of, say, 10 region numbers with 100,000 entries in each one. So instead of sorting only 100,000 entries at a time, I am always sorting 1 million entries at a time. A lot of separate sorts are issued, hence the runtime problem.

    I have two questions:-

    1) How can I reformat the StatsArray to be dependent upon region numbers and make sorting faster? 10 StatsArrays instead of 1 would be fine.

    2) If a new data structure is proposed, how would I numerically sort something like the AR key? This is really what I am struggling with the most: ways to sort only 1 of n keys in an array when its data structure gets more complicated.

    I do not use arrays like this very often, so any suggestions would be much appreciated.

    Thanks!

    -fiddler42
Configure global default variables in package at install time
2 direct replies — Read more / Contribute
by zachhh
on May 21, 2013 at 09:30

    I don't know if this is wise, but I'd like to be able to set global default variables for a package at build time. The perl module is a back end designed to communicate with a JavaScript front end (see https://github.com/zachhh/Packform). The global variables are for things like the path (as in the href attribute of a script tag, not strictly the path on disk) for including the front end JS, the desired version of jQuery and the jQuery UI, etc.

    This is entirely out of laziness -- I don't want to have to provide this information in every script using the module. The defaults are hardcoded to be suitable for the server on which it was built. I want to be able to specify these when I install it.

    This would necessarily have to change the variable initializations in the .pm file (which seems dangerous -- but hey, make test will tell them something's wrong if they mess it up). Of course, this could also come from a configuration file or the like. But, that seems dirty too.

    Is this approach unwise?

    If this is a reasonable approach, what are the recommendations? Should this be done in Makefile.PL somehow? Should there be a conf file that is provided and ask the user to modify that if they want changes? How should that be incorporated?

SIGHUP delivered on Windows
1 direct reply — Read more / Contribute
by rovf
on May 21, 2013 at 07:47
    Problem: My Perl application has signal handlers installed, which logs (if possible) the signal received, and reacts accordingly. Recently I found one case in our logs showing that a SIGHUP was delivered. The application is running using ActiveState Perl 5.14 on (64bit-)Windows 7. I wonder which event might possibly cause the SIGHUP handler being called (aside from the obvious possibility that my application is doing suicide using this signal).

    I am aware that on Windows, signal handling is simulated by turning Windows messages are turned into signals. Looking at the source code of Perl, I found indeed 1 case where SIGHUP seems to be generated. I don't understand much about the Windows API, but the code says:

    BOOL WINAPI win32_ctrlhandler(DWORD dwCtrlType) ... switch(dwCtrlType) { case CTRL_CLOSE_EVENT: /* A signal that the system sends to all processes attached to a + console when the user closes the console (either by choosing the Close com +mand from the console window's System menu, or by choosing the End Task com +mand from the Task List */ if (do_raise(aTHX_ 1)) /* SIGHUP */ sig_terminate(aTHX_ 1); return TRUE;
    I roughly (and maybe uncorrectly) understand the comment in that closing the console corresponds to SIGHUP. Trying to verify this, I wrote the following small program:
    use strict; use warnings; use Config; BEGIN { open(LOG,">","what_happens.log") or die "$!"; my $ofh = select LOG; $|=1; select $ofh; print LOG "Unbuffered Log\n"; } sub pout { print {$_} @_,"\n" for (*LOG, *STDERR); } END { pout("end handler called"); close LOG; } pout "Started"; my $signum = 0; foreach my $signame (split(' ', $Config{sig_name})) { if ($signum) { pout("Establishing $signame $signum"); $SIG{$signame} = $signame =~ /^KILL|STOP$/ ? 'IGNORE' # According to perldoc perlipc, they + can be ignored, but not trapped : sub { $SIG{$signame} = 'DEFAULT'; + pout("Murdered by signal $signame, default han +dler established"); if ($signame eq 'INT') { pout("This seems to be Keyboard interrupt" +); exit; } else { pout("Suicide"); kill($signum, $$); # proceed with norma +l handling of signal } }; } ++$signum; } pout("Interrupt handlers registered"); sleep(6); pout("Normal exit");
    For example, if I start the program in a console, and type Control-C while the application is sleeping, I see message This seems to be Keyboard interrupt in my logfile. However, if I close instead the console window, the last message in the log is Interrupt handlers registered. I conclude that closing the window, is invoking either SIGKILL or SIGSTOP - different from what I expected.

    I guess that I misunderstood the meaning of SIGHUP on Windows. Could someone help me here?
    -- 
    Ronald Fischer <ynnor@mm.st>
Copying and Running PERL SCRIPTS in Parallel
3 direct replies — Read more / Contribute
by rahulruns
on May 21, 2013 at 03:08

    I am trying to copy PERL SCRIPTS to hosts and then I need to run them in parallel. I am using Net::OpenSSH::Parallel to copy and run the commands. But I am not able to even copy the commands. It is a password less ssh. I am able to generate and store the host list in an array hosts.

    my $pssh = Net::OpenSSH::Parallel->new(); $pssh->add_host($_) for @hosts; $pssh->push('*', scp_put => '/root/cpu.pl', '/root/'); $pssh->push('*', scp_put => '/root/memory.pl', '/root/');
Android App Using Perl
1 direct reply — Read more / Contribute
by g4774g
on May 20, 2013 at 19:04

    Is there a perl module to help with writing an android app? Thx

Reading wrong value from excel sheet
4 direct replies — Read more / Contribute
by sandeep_car
on May 20, 2013 at 13:27

    Hi,

    We upload file in .xlsx or xls format. In this file there is one coloumn for which value is in decimal. For these value for some it shows differnet result while reading from excel file and for some it shows correct result.

    For example:

    at place of 1.14 it comes 1.139999999999999

    at place of 1.13 it comes 1.129999999999999

    where as for some it give correct result.

    Code used for reading .xlsx file is:

    $fileName = $_[0]; $fFnc = $_[1]; $logMsg = new LogMessage(); $logMsg->writeLog( $fFnc, "Reading $fileName \n" ); $FileData = ""; %FileHash = (); %source_headers = (); # For XLSX formats if( $fileName =~ /\.xlsx$/i ) { my $myConv = Text::Iconv->new ("utf-8" +, "windows-1251"); my %xlFileData = (); my $myBook = Spreadsheet::XLSX->new( " +$fileName", $myConv ); if( $myBook ) { foreach my $sheet (@{$myBook->{Work +sheet}}) { $sheet->{MaxRow} ||= $sheet +->{MinRow}; foreach my $row ($sheet -> +{MinRow} .. $sheet -> {MaxRow}) { $sheet -> {MaxCol} +||= $sheet -> {MinCol}; foreach my $col ($s +heet -> {MinCol} .. $sheet -> {MaxCol}) { my $cell = $ +sheet -> {Cells} [$row] [$col]; // At this place it gives incorrect result }

    Code used for reading .xls file is:

    if($fileName =~ /\.xls$/i ) { my $parser = Spreadsheet::ParseExcel-> +new(); my $workbook = $parser->parse( "$fileN +ame" ); if( defined $workbook ) { for my $worksheet ( $workbook->works +heets() ) { my ( $row_min, + $row_max ) = $worksheet->row_range(); my ( $col_min, + $col_max ) = $worksheet->col_range(); for my $row ( +$row_min .. $row_max ) { for my + $col ( $col_min .. $col_max ) { + my $cell = $worksheet->get_cell( $row, $col ); // At this place it gives incorrect result while reading. }

    Please help me on this.

    Thanks,

    Sandeep

Comparing spaceships (cmp and <=> as options)
5 direct replies — Read more / Contribute
by Random_Walk
on May 20, 2013 at 11:31

    Fellow Monks, I seek your Perls of wisdom.

    I have an array of arrays. the lower level arrays are records with values both numeric and alpha. I want to give the user the option to sort on any of these fields. I thought it would be nice to use a simple hash table referencing the record number to sort on and the comparator to use. I am having a problem using a variable containing comparator. Am I missing something or do I just need another approach?

    my $sort = 'hours'; # really comes from a switch my %map = ( # sorting map hours => [0, \sub {<=>}], code => [1, \sub {cmp}], name => [2, \sub {cmp}], ); # example data my @records = ( [10, 'xyz232', 'secret project'], [ 5, 'foo123', 'world domination'], [ 7, 'bar666', 'have a beer'], ); for ( sort {$a->[$map{$sort}->[0]] $map{$sort}->[1] $b->[$map{$sort}->[ +0]]} @records ) { print join ", ", @$_; }

    Update

    among many variations of syntax I also tried the following. I feel it may be getting closer, other than the fact it won't compile :)
    my %map = ( hours => sub {sort { $_[0] <=> $_[0] } };, number => sub {sort { $_[0] <=> $_[0] } };, name => sub {sort { $_[0] <=> $_[0] } };, task => sub {sort { $_[0] <=> $_[0] } };, );

    Cheers,
    R.

    Pereant, qui ante nos nostra dixerunt!
xml document preface
1 direct reply — Read more / Contribute
by fionbarr
on May 20, 2013 at 11:05
    xml_writer constructs this preface:
    <?xml version="1.0"?>
    Is there anything else that (advisedly) might be included?
Why upgrade perl?
6 direct replies — Read more / Contribute
by poulhs
on May 20, 2013 at 04:41

    We (as so many others) use perl intensivly, but we are currently stuck with the stock RHEL6 perl (that is 5.10.1).

    I am considering building our own perl (in /opt/somehwere), adding the modules that we need, and keeping that up-to-date (using jenkins to build our own RPMs for deployment). I have a feeling about the amount of work involved with this (been down that road before), and my boss asked the ultimative question:

    Is it worth it?

    So I started to prepare a what's-the-difference tutorial (from perldelta's), then 5.18.0 came along and threw given/when and ~~ into experimental::smartmatch, so now I am (almost) back to square 0. Almost all I got is some optimizations and some hash-key security...

    Compared to 5.10.1, why should I upgrade to 5.18.0?

    • I know the pain, what is the gain?
    • What do you think is the most important arguments for upgrading to 5.18.0?
    • How will 5.18.0 make us happier, our life easier and sun shine more frequently?
    • Or am I complete lost, and should I continue being stuck with the stock?
    Thank you Monks, for allowing a humble perler interrupt you.

Scalar followed by parenthetical...
3 direct replies — Read more / Contribute
by Anonymous Monk
on May 18, 2013 at 22:15
    Why doesn't the following produce an error??
    #!/usr/bin/perl use warnings; use strict; use Data::Dumper; my @h = ( { A => 1, B => undef }, { A => 2, B => 2 }, { A => 2, B => undef }, ); $_->{B} ||= $_{A} foreach @h; # Bug! print Dumper(\@h);
operation inside reference
3 direct replies — Read more / Contribute
by hegotf
on May 18, 2013 at 09:30

    Hi Monks

    Can anyone explain what does that do? @{$doc->{links} // []}

    I don't understand the // [] part.

    here is the link to entire code: https://gist.github.com/creaktive/4607326

New Meditations
What is the impact of 5.18.0?
2 direct replies — Read more / Contribute
by Tux
on May 19, 2013 at 06:53

    I use my laptop as test environment. When all (upgrades) pass on production/utility software, I upgrade my workstation. If that shows no problems after a few weeks, I start upgrading other production servers.

    So I installed 5.18.0 on my laptop as default perl, and then tried to install all CPAN modules I ever used in 5.16.x

    There are just a few things that stand out as a reason for FAILure:

    • Test files that use for $foo qw( … ) { which is now deprecated. The module autor should rewrite that to for $foo (qw( … )) {. I filed RT tickets for all I encountered. Lets hope the authors will fix their code.
    • POD failures. As the POD checker is now way stricter than the old version was, it shows more real trouble. And it should! The problem is that many authors ship the pod tests too, so the test suite will fail if Test::Pod and friends are installed, and thus cpan won't install those modules. That means manual work ignoring the pod errors (and when obvious file RT tickets). Authors: unless you keep in sync with reality and release often, please do not include pod tests in you distribution.
    • Loads of warnings when a module uses given/when as those are now marked experimental. I can ignore the warnings when testing, but will they fill up my logs when run in production code? When the module also uses Test::NoWarnings, one cannot install. A module like MooseX::App now won't install, and it causes many other modules that have this as a prerequisite to fail too.
    • XS authors that never updated ppport.h from Devel::PPPort. Easy to fix.
    • Modules that use Module::Install but ship ancient or incomplete versions of it in inc. The easiest solution for me was to just recursively remove inc/ and all problems vanish. I have no tuits to RT all of them.
    • Several modules fail because they declare optional deps that are actually hard deps, like using JSON::XS, Lexical::Sub or Data::Alias - which won't build under 5.18.0
    • Test failures, or maybe even code failures, due to hash randomization. Unless I am absolutely sure that the test is wrong, I cannot install these modules.
    • Some failures have been reported to the authors but the available fix has not (yet) been applied (like in Template::Toolkit RT#84778) or not yet released (Tk and SQL::Statement). When their repository is open, they build fine from the repo and I am confident the next release will build fine.

    The message is: perl-5.18.0 is awesome, but please please test the module you require before starting the installation on machines other than a test environment.


    Enjoy, Have FUN! H.Merijn
New Cool Uses for Perl
Batch Printing in Linux
1 direct reply — Read more / Contribute
by jmlynesjr
on May 15, 2013 at 15:26

    I had a bunch of music lyrics that I wanted to print and I didn't want to manually drive gedit. I did, however, want to retain the document formatting. I found a hint in Re^2: using Brother QL-570 printer with Perl and also found sample commands in an Ubuntu Community Documentation post for using the Open Office command line interface for batch printing or viewing. The are also posts out there for doing a similar thing with MS-Word.

    The following is what I threw together. It does the job for me, YMMV. I needed to print UTF-8 text files, but Open Office will try to print whatever file you give it using the file extension as a guide to the file format. I have also printed .odt files.

    James

    There's never enough time to do it right, but always enough time to do it over...

Log In?
Username:
Password:

What's my password?
Create A New User
Chatterbox?
[ambrus]: tea is ready
[Corion]: Interesting - I didn't know Tumblr was that big. That somewhat explains what they blew their VC money on.

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (22)
As of 2013-05-21 18:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best material for plates (tableware) is:









    Results (443 votes), past polls