Beefy Boxes and Bandwidth Generously Provided by pair Networks Joe
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

The Monastery Gates

( #131=superdoc: print w/ replies, xml ) Need Help??

Donations gladly accepted

If you're new here please read PerlMonks FAQ
and Create a new user.

Want Mega XP? Prepare to have your hopes dashed, join in on the: poll ideas quest 2013  (Don't worry; you've got plenty of time.)

New Questions
how to unpack a C struct with length array preceding data array
4 direct replies — Read more / Contribute
by johnlumby
on May 22, 2013 at 15:28

    my perl app receives a buffer containing this format

    { short lengths[2]; char data[???]; } where there are two items in data, of length lengths[0] and lengths[1] respectively.

    I have tried and tried to unpack this into 4 perl variables in a single unpack() call and failed. I had to resort to two unpack()'s, the first to extract the lengths followed by something like

    eval('($junk1 , $junk2, $data0, $data1) = unpack "ssA' . $data0_length + . "A" . $data1_length . '",$rcvd_buf;');

    which works fine, but can some kind Monk tell me how to do this in one unpack.

perl in a container (lxc or other)
No replies — Read more | Post response
by djzort
on May 22, 2013 at 07:03
    Im curious if someone would be so kind as to share their experiences setting up a minimal perl container in lxc, openvz, bsd jails, linux-vserver or even chroots.

    App::Staticperl (http://software.schmorp.de/pkg/App-Staticperl.html) looks like it might be a good starting point - if i wanted to avoid a stripped down installed of <favourite distro or bsd>

    The ideal outcome would provide a container with mcpan and a given version of perl. Possibly some fusion of perlbrew and some dark magic to cook up the perl version then drop it in the container.

    Use case is to wrap up apps and put resource limits on them. PAAS like really. I know uwsgi can use namespaces and what not, if someone has used that - please share!
eval point in hash ref
3 direct replies — Read more / Contribute
by ag4ve
on May 22, 2013 at 02:50
    I'd like to be able to do:
    ./dat_query '{seek}'
    or
    ./dat_query '{seek}{messages}'

    and have code do:
    print Dumper($nethist->{seek});
    or
    print Dumper($nethist->{seek}{messages});

    I've been trying:
    my $in = $ARGV[0]; my $val = eval '$nethist->$in'; print Dumper($val);
    and the like with no luck
reverse sort arrays in subroutines
4 direct replies — Read more / Contribute
by JockoHelios
on May 21, 2013 at 17:34

    I can sort an array passed into a subroutine, but I can't reverse sort or use a custom sort on that array.

    I can't get subroutines to do much of anything with passed arrays. This is one of sevearl array-subroutine tests that I've tried over the last two days. I have tried variations with passing array references, but to no avail.

    # test subroutines with an array $var = 8; push( @TaR, $var ); $var = 2; push( @TaR, $var ); $var = 5; push( @TaR, $var ); $var = 4; push( @TaR, $var ); $var = 5; push( @TaR, $var ); $var = 6; push( @TaR, $var ); $var = 1; push( @TaR, $var ); $var = 8; push( @TaR, $var ); print STDOUT "\n\n"; print STDOUT "\n\nAs-is "; # This segment works for $OneLine( @TaR ) { print STDOUT " $OneLine"; } print STDOUT " Done\n\n"; SortedArray( @TaR ); ReversedArray( @TaR ); print STDOUT "\n\n"; # This sort works sub SortedArray { my @PassedArray = @_; my @SortedArray = sort @PassedArray; print STDOUT "\n\nSorted "; for $SubLine( @SortedArray ) { print STDOUT " $SubLine"; } print STDOUT " Done\n\n"; return 1; } # This sort, but doesn't reverse the sort sub ReversedArray { my @PassedArray = @_; my @SortedArray = sort reverse @PassedArray; print STDOUT "\n\nRvrsd "; for $SubLine( @SortedArray ) { print STDOUT " $SubLine"; } print STDOUT " Done\n\n"; return 1; }
Mac to Windows Data Incompatibility
No replies — Read more | Post response
by goshawk
on May 21, 2013 at 15:49

    Hello, all. I'm having an issue with a large and difficult-to-produce piddle matrix that I made on a Mac OS with 64 architecture(8-byte long), which I want to use in a perl program on a Windows OS with 64 architecture (4-byte long). If at all possible, I want to avoid recreating this data.

    I want to load in a matrix, slice a row, then calculate the Pearson's correlation of that row against each other row using PDL::Stats::Basic corr. This works fine when I create a matrix in Windows and find the correlation in Windows, and it also works when I use the data I want to use to find the correlation on Mac, but not when I use data from the Mac OS in Windows. I've tried using PDL::IO::Storable and PDL::IO::Dumper. When I save/load the matrix using PDL::IO::Storable, I get this error:

    Long integer size is not compatible at C:/Perl64/lib/Storable.pm line +380, at test.pl line 6.
    When I save/load the matrix using PDL::IO::Dumper, it loads fine, but the correlation no longer works. Here's a MWE:
    # On Mac OS use PDL; use PDL::IO::Dumper; my $a = random(8,150); PDL::IO::Dumper::fdump($a, 'a.pldl'); # On Windows OS use PDL; use PDL::Stats::Basic; use PDL::IO::Dumper;` my $a = frestore('a.pldl'); my $b = $a->slice(':,3'); my $c = $a->corr($b); print $c->slice('(0)'), "\n";

    This code returns '-1.#IND', which is Windows for nan. Am I getting caught by the different data sizes again?

Soap::Lite Issue
1 direct reply — Read more / Contribute
by rbala
on May 21, 2013 at 15:04
    Hi Monks, I have a Soap::Lite running in server side , which I am using to tranfer a network capture of wireshark . I am returning this as an array to the soap client . I am getting the following error :

    Use of inherited AUTOLOAD for non-method Soap::Serializer::as_base64Binary is deprecated at /opt/Activeperl/site/lib/SOAP/Lite.pm.

    . Also , Soap server hangs therein. Can you please help me to solve this ? Thanks, Bala.
IO::Uncompress::Gunzip to scalar takes hours (on windows)
4 direct replies — Read more / Contribute
by cmv
on May 21, 2013 at 11:40
    Hi Monks-

    I'm seeing a strange issue when using IO::Uncompress:Gunzip on windows that I can't explain. The same code runs fine under Unix. I'd like to understand why.

    The code below will take about 20-seconds to unzip a 65M zipped file on my windows xp box.

    If I comment out the first gunzip line, and uncomment the second gunzip line, it will take hours on the windows box (6 hours last time I bothered to try). Any thoughts?

    Thanks

    -Craig

    #!/opt/exp/bin/perl5.8 use strict; use warnings; use IO::Uncompress::Gunzip qw(gunzip $GunzipError) ; my $ifile = shift || die "Missing input file name\n"; open(IFILE, "<$ifile") || die "Can't open $ifile"; binmode(IFILE); my $inputstr = join('', <IFILE>); print STDERR scalar(localtime), " - STARTING UNZIP!!!\n"; # This is fast on all platforms gunzip \$inputstr => "clearfile" or die "gzip failed: $GunzipError\n"; my $cleartxt; # This takes forever on windows, no problem on Unix #gunzip \$inputstr => \$cleartxt or die "gzip failed: $GunzipError\n"; print STDERR scalar(localtime), " - UNZIPPED COMPLETE!!!\n";
    UPDATE1:

    Here is some profile data I got from running both cases on a smaller gzipped input file (4.8M)...

    Profile Data for FAST Method

    Total Elapsed Time = 3.667486 Seconds User+System Time = 1.200486 Seconds Exclusive Times %Time ExclSec CumulS #Calls sec/call Csec/c Name 37.2 0.447 0.447 7281 0.0001 0.0001 Compress::Raw::Zlib::infl +ateStream ::inflate 19.8 0.238 1.296 1 0.2377 1.2963 IO::Uncompress::Base::_rd +2 14.9 0.179 0.200 7293 0.0000 0.0000 IO::Uncompress::Base::sma +rtRead 8.66 0.104 0.979 7281 0.0000 0.0001 IO::Uncompress::Base::_ra +w_read 6.25 0.075 1.059 7283 0.0000 0.0001 IO::Uncompress::Base::rea +d 5.66 0.068 0.068 7281 0.0000 0.0000 IO::Uncompress::Base::pus +hBack 4.66 0.056 0.056 14575 0.0000 0.0000 IO::Uncompress::Base::sav +eStatus 3.25 0.039 0.039 14562 0.0000 0.0000 U64::add 3.08 0.037 0.496 7281 0.0000 0.0001 IO::Uncompress::Adapter:: +Inflate:: uncompr 1.92 0.023 0.224 7281 0.0000 0.0000 IO::Uncompress::Base::rea +dBlock 1.33 0.016 0.016 1 0.0160 0.0160 IO::bootstrap 1.33 0.016 0.076 4 0.0040 0.0191 main::BEGIN 1.33 0.016 0.060 9 0.0018 0.0067 IO::Uncompress::Gunzip::B +EGIN 1.25 0.015 0.015 10 0.0015 0.0015 strict::unimport 1.25 0.015 0.030 37 0.0004 0.0008 IO::Compress::Base::Commo +n::BEGIN

    Profile Data for SLOW Method

    Total Elapsed Time = 154.4478 Seconds User+System Time = 154.2188 Seconds Exclusive Times %Time ExclSec CumulS #Calls sec/call Csec/c Name 98.6 152.1 152.14 7286 0.0209 0.0209 Compress::Raw::Zlib::infl +ateStream 0 ::inflate 0.34 0.519 0.573 7298 0.0001 0.0001 IO::Uncompress::Base::sma +rtRead 0.28 0.438 154.04 7286 0.0001 0.0211 IO::Uncompress::Base::_ra +w_read 0.18 0.283 152.49 7286 0.0000 0.0209 IO::Uncompress::Adapter:: +Inflate:: 9 uncompr 0.18 0.275 154.31 7288 0.0000 0.0212 IO::Uncompress::Base::rea +d 0.10 0.158 0.158 14585 0.0000 0.0000 IO::Uncompress::Base::sav +eStatus 0.10 0.151 0.151 7286 0.0000 0.0000 IO::Uncompress::Base::pus +hBack 0.09 0.134 0.134 7288 0.0000 0.0000 IO::Uncompress::Base::sma +rtEof 0.08 0.128 0.128 14572 0.0000 0.0000 U64::add 0.04 0.060 0.060 21864 0.0000 0.0000 Compress::Raw::Zlib::__AN +ON__ 0.02 0.025 154.34 1 0.0248 154.34 IO::Uncompress::Base::_rd +2 0.01 0.016 0.016 4 0.0040 0.0040 Compress::Raw::Zlib::cons +tant 0.01 0.016 0.016 8 0.0020 0.0020 Exporter::export 0.01 0.016 0.016 7 0.0023 0.0023 IO::File::BEGIN 0.01 0.016 0.016 26 0.0006 0.0006 Compress::Raw::Zlib::BEGI +N

    UPDATE2:

    I believe Corion and BrowserUk are correct about this issue.

    Watching Pagefaults with Process Explorer, here is what I see:

    ~26,000,000 Pagefaults - Slow Method
    ~6,000 Pagefaults - Fast Method

    UPDATE3:

    I believe this talks about the root cause:
    https://groups.google.com/forum/?fromgroups#!topic/perl.perl5.porters/44PTHwefYUk

How to improve this data structure?
7 direct replies — Read more / Contribute
by fiddler42
on May 21, 2013 at 10:31
    Hi,

    I recently made the rookie mistake of authoring a perl script with a sample data input file that was not realistic in size. So now I am running the script with much larger data input files and it is painfully slow. Here is the root cause of the problem:-

    1) A data input file is parsed, and each line of the file is added to an array like so:-

    push (@StatsArray, {RegionNum=>$RegionNum, AR=>$AR[$RegionNum], BCR=>$BCR[$RegionNum]});
    I like the data structure above because it is easy to understand and I can add new elements to the array as the script becomes more sophisticated. I included only 3 of 15 variables, and I expect the number of variables to increase a little more.

    2) Once the array is fully populated (millions of entries) and I need to pluck data from it, I need to do two things: a) *always* numerically sort by RegionNum first and then b) numerically sort some other key. Here is an example:-
    foreach $RegionCoords (@AllRegionCoords) { $RegionNum++; foreach (sort {$$a{RegionNum} <=> $$b{RegionNum} or $$a{AR} <=> $ +$b{AR}} @StatsArray) { $RegionKey = $$_{RegionNum}; $ARKey = $$_{AR}; if ($RegionKey == $RegionNum) { # do stuff with sorted AR data in region n } } }
    Again, easy to follow. The problem is there are many different region numbers and I am not breaking down the data into smaller chunks. For example, although there may be 1 million entries in @StatsArray, all of the entries are made up of, say, 10 region numbers with 100,000 entries in each one. So instead of sorting only 100,000 entries at a time, I am always sorting 1 million entries at a time. A lot of separate sorts are issued, hence the runtime problem.

    I have two questions:-

    1) How can I reformat the StatsArray to be dependent upon region numbers and make sorting faster? 10 StatsArrays instead of 1 would be fine.

    2) If a new data structure is proposed, how would I numerically sort something like the AR key? This is really what I am struggling with the most: ways to sort only 1 of n keys in an array when its data structure gets more complicated.

    I do not use arrays like this very often, so any suggestions would be much appreciated.

    Thanks!

    -fiddler42
Configure global default variables in package at install time
3 direct replies — Read more / Contribute
by zachhh
on May 21, 2013 at 09:30

    I don't know if this is wise, but I'd like to be able to set global default variables for a package at build time. The perl module is a back end designed to communicate with a JavaScript front end (see https://github.com/zachhh/Packform). The global variables are for things like the path (as in the href attribute of a script tag, not strictly the path on disk) for including the front end JS, the desired version of jQuery and the jQuery UI, etc.

    This is entirely out of laziness -- I don't want to have to provide this information in every script using the module. The defaults are hardcoded to be suitable for the server on which it was built. I want to be able to specify these when I install it.

    This would necessarily have to change the variable initializations in the .pm file (which seems dangerous -- but hey, make test will tell them something's wrong if they mess it up). Of course, this could also come from a configuration file or the like. But, that seems dirty too.

    Is this approach unwise?

    If this is a reasonable approach, what are the recommendations? Should this be done in Makefile.PL somehow? Should there be a conf file that is provided and ask the user to modify that if they want changes? How should that be incorporated?

SIGHUP delivered on Windows
1 direct reply — Read more / Contribute
by rovf
on May 21, 2013 at 07:47
    Problem: My Perl application has signal handlers installed, which logs (if possible) the signal received, and reacts accordingly. Recently I found one case in our logs showing that a SIGHUP was delivered. The application is running using ActiveState Perl 5.14 on (64bit-)Windows 7. I wonder which event might possibly cause the SIGHUP handler being called (aside from the obvious possibility that my application is doing suicide using this signal).

    I am aware that on Windows, signal handling is simulated by turning Windows messages are turned into signals. Looking at the source code of Perl, I found indeed 1 case where SIGHUP seems to be generated. I don't understand much about the Windows API, but the code says:

    BOOL WINAPI win32_ctrlhandler(DWORD dwCtrlType) ... switch(dwCtrlType) { case CTRL_CLOSE_EVENT: /* A signal that the system sends to all processes attached to a + console when the user closes the console (either by choosing the Close com +mand from the console window's System menu, or by choosing the End Task com +mand from the Task List */ if (do_raise(aTHX_ 1)) /* SIGHUP */ sig_terminate(aTHX_ 1); return TRUE;
    I roughly (and maybe uncorrectly) understand the comment in that closing the console corresponds to SIGHUP. Trying to verify this, I wrote the following small program:
    use strict; use warnings; use Config; BEGIN { open(LOG,">","what_happens.log") or die "$!"; my $ofh = select LOG; $|=1; select $ofh; print LOG "Unbuffered Log\n"; } sub pout { print {$_} @_,"\n" for (*LOG, *STDERR); } END { pout("end handler called"); close LOG; } pout "Started"; my $signum = 0; foreach my $signame (split(' ', $Config{sig_name})) { if ($signum) { pout("Establishing $signame $signum"); $SIG{$signame} = $signame =~ /^KILL|STOP$/ ? 'IGNORE' # According to perldoc perlipc, they + can be ignored, but not trapped : sub { $SIG{$signame} = 'DEFAULT'; + pout("Murdered by signal $signame, default han +dler established"); if ($signame eq 'INT') { pout("This seems to be Keyboard interrupt" +); exit; } else { pout("Suicide"); kill($signum, $$); # proceed with norma +l handling of signal } }; } ++$signum; } pout("Interrupt handlers registered"); sleep(6); pout("Normal exit");
    For example, if I start the program in a console, and type Control-C while the application is sleeping, I see message This seems to be Keyboard interrupt in my logfile. However, if I close instead the console window, the last message in the log is Interrupt handlers registered. I conclude that closing the window, is invoking either SIGKILL or SIGSTOP - different from what I expected.

    I guess that I misunderstood the meaning of SIGHUP on Windows. Could someone help me here?
    -- 
    Ronald Fischer <ynnor@mm.st>
Copying and Running PERL SCRIPTS in Parallel
3 direct replies — Read more / Contribute
by rahulruns
on May 21, 2013 at 03:08

    I am trying to copy PERL SCRIPTS to hosts and then I need to run them in parallel. I am using Net::OpenSSH::Parallel to copy and run the commands. But I am not able to even copy the commands. It is a password less ssh. I am able to generate and store the host list in an array hosts.

    my $pssh = Net::OpenSSH::Parallel->new(); $pssh->add_host($_) for @hosts; $pssh->push('*', scp_put => '/root/cpu.pl', '/root/'); $pssh->push('*', scp_put => '/root/memory.pl', '/root/');
Android App Using Perl
1 direct reply — Read more / Contribute
by g4774g
on May 20, 2013 at 19:04

    Is there a perl module to help with writing an android app? Thx

New Meditations
What is the impact of 5.18.0?
2 direct replies — Read more / Contribute
by Tux
on May 19, 2013 at 06:53

    I use my laptop as test environment. When all (upgrades) pass on production/utility software, I upgrade my workstation. If that shows no problems after a few weeks, I start upgrading other production servers.

    So I installed 5.18.0 on my laptop as default perl, and then tried to install all CPAN modules I ever used in 5.16.x

    There are just a few things that stand out as a reason for FAILure:

    • Test files that use for $foo qw( … ) { which is now deprecated. The module autor should rewrite that to for $foo (qw( … )) {. I filed RT tickets for all I encountered. Lets hope the authors will fix their code.
    • POD failures. As the POD checker is now way stricter than the old version was, it shows more real trouble. And it should! The problem is that many authors ship the pod tests too, so the test suite will fail if Test::Pod and friends are installed, and thus cpan won't install those modules. That means manual work ignoring the pod errors (and when obvious file RT tickets). Authors: unless you keep in sync with reality and release often, please do not include pod tests in you distribution.
    • Loads of warnings when a module uses given/when as those are now marked experimental. I can ignore the warnings when testing, but will they fill up my logs when run in production code? When the module also uses Test::NoWarnings, one cannot install. A module like MooseX::App now won't install, and it causes many other modules that have this as a prerequisite to fail too.
    • XS authors that never updated ppport.h from Devel::PPPort. Easy to fix.
    • Modules that use Module::Install but ship ancient or incomplete versions of it in inc. The easiest solution for me was to just recursively remove inc/ and all problems vanish. I have no tuits to RT all of them.
    • Several modules fail because they declare optional deps that are actually hard deps, like using JSON::XS, Lexical::Sub or Data::Alias - which won't build under 5.18.0
    • Test failures, or maybe even code failures, due to hash randomization. Unless I am absolutely sure that the test is wrong, I cannot install these modules.
    • Some failures have been reported to the authors but the available fix has not (yet) been applied (like in Template::Toolkit RT#84778) or not yet released (Tk and SQL::Statement). When their repository is open, they build fine from the repo and I am confident the next release will build fine.

    The message is: perl-5.18.0 is awesome, but please please test the module you require before starting the installation on machines other than a test environment.


    Enjoy, Have FUN! H.Merijn
Log In?
Username:
Password:

What's my password?
Create A New User
Chatterbox?
[marto]: hmm, with some tweaks got my Raspberry Pi server down to using 39MB RAm

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2013-05-23 08:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best material for plates (tableware) is:









    Results (478 votes), past polls