Donations gladly accepted
If you're new here please read PerlMonks FAQ
and Create a new user.
Want Mega XP? Prepare to have your hopes dashed, join in on the: poll ideas quest 2013 (Don't worry; you've got plenty of time.)
|
New Questions
|
Cross-platform GUI for UNIX based scripts
3 direct replies — Read more / Contribute
|
by Anonymous Monk
on May 23, 2013 at 08:27
|
|
|
Dear Monks,
I am developing a Perl/C pipeline which is placed on an UNIX server. At the moment I am accessing it in Windows through Putty, providing simple command line arguments.
I need to however make it easy to use for non-computer savvy people who use Linux/mac/windows. I am quite good in in making the software work but have no experience in cross-platforms compatibility, networks or GUIs.
Does anyone have any idea what would be the best approach? The user need to submit a string and select multiple options. The program will then run for 2-3 days so it would be nice if there was a way to see the progress, cancel the script and see if it is finished. After it's done I need to visualize the data which atm I provide in excel spreadsheet.
I was thinking about an JavaScript webpage with all the options but don't know is people will be able to access it easily in Windows. Maybe a Java or a Perl GUI?
A multi-platform desktop app that connects to a server would be absolutely ideal but I have no idea how difficult it is to do for someone with limited knowledge of distributed systems.
Any ideas are welcome. I spend a lot of time developing the pipeline but it won't be used a lot unless I make it incredibly easy to access. Thanks in advance for your advice. I am open to learning new stuff and spending 1 or 2 months on it.
|
HTML::TreeBuilder, HTML::Element, as_XML()
1 direct reply — Read more / Contribute
|
by perlig
on May 23, 2013 at 06:35
|
|
|
Dear omniscient monks,
i got some html/tei like data and want to parse it to xml format. it is working pretty well for some files.. but not for all..
here is my code:
# pragma
use strict;
use warnings;
# modules
use XML::Simple;
use XML::Tidy;
use Data::Dumper;
use Data::Diver qw( Dive DiveRef DiveError );
use HTML::TreeBuilder;
use XML::Tidy::Tiny;
# little helper
use constant false => 0;
use constant true => 1;
...
# get instance of treebuilder
my $root = HTML::TreeBuilder->new();
# configure treebuilder
$root->ignore_unknown( false );
# dump data to the treebuilder
$root->parse( $fileData );
# get name for target file
my $target = $file;
$target =~ s/$fileExtension$/xml/;
# open output filehandle
open( $FH, '>', $target );
# configure output
binmode $FH, ":utf8";
# ERROR HERE
208: my $data = $root->guts()->as_XML();
print $FH xml_tidy( $data );
close $FH;
...
caption has an invalid attribute name 'n' at script.pl line 208
i substite all 'n' in the file.. but got still the same error. for that the 'n' is not the anchor of this error.. i dont know what going on here?!
$root->guts()
is okey.. it is all about the ->as_XML() :-((
kindly, perlig
$perlig =~ s/pec/cep/g if 'errors expected';
|
http request with LWP::Useragent
1 direct reply — Read more / Contribute
|
by Anonymous Monk
on May 23, 2013 at 05:34
|
|
|
Hi,
I am using LWP::UserAgent module to get HTTP response from YY.JSP and embed the same in div in my web page.
I am getting the response. But the problem I get is below.
In that JSP there is a frameset with mulitple frames.
Each frame loads another X.JSP. When I get the response and check in browser, I do get the frame with JSP. But it does not load contents or should I say the response is not returning the contents of that X.JSP.
I get following in response.
<HTML>...<Body>....
<frameset cols=33%,* onunload="" ><frame border=1 frameborder=yes name
+=client src=X.jsp></frameset>
But, it does not show the contents from X.jsp. The page stays blank.
Is it that LWP useragent module only process the first YY.JSP but not any X.JSP inside YY.JSP?
Thanks in advance.
- Amit
|
Net::Pcap, parsing packets offline
No replies — Read more | Post response
|
by raj_paps
on May 23, 2013 at 05:14
|
|
|
I am parsing packets offline. I am able to do so with
my $pcap = Net::Pcap::open_offline($fp, \$err);
Net::Pcap::dispatch($pcap, 0, \&process_pkt_sip, $out);
my problem is, I need to get the timestamp/timings of the packet for certain measurement. I saw perl code in some forums to retrieve timings for real time captures. any suggestions for offline..?
|
how to unpack a C struct with length array preceding data array
5 direct replies — Read more / Contribute
|
by johnlumby
on May 22, 2013 at 15:28
|
|
|
{
short lengths[2];
char data[???];
}
where there are two items in data, of length
lengths[0] and lengths[1] respectively.
I have tried and tried to unpack this into 4 perl variables in a single unpack() call and failed.
I had to resort to two unpack()'s, the first to extract the lengths followed by something like
eval('($junk1 , $junk2, $data0, $data1) = unpack "ssA' . $data0_length
+ . "A" . $data1_length . '",$rcvd_buf;');
which works fine,
but can some kind Monk tell me how to do this in one unpack.
|
perl in a container (lxc or other)
No replies — Read more | Post response
|
by djzort
on May 22, 2013 at 07:03
|
|
|
Im curious if someone would be so kind as to share their experiences setting up a minimal perl container in lxc, openvz, bsd jails, linux-vserver or even chroots.
App::Staticperl (http://software.schmorp.de/pkg/App-Staticperl.html) looks like it might be a good starting point - if i wanted to avoid a stripped down installed of <favourite distro or bsd>
The ideal outcome would provide a container with mcpan and a given version of perl. Possibly some fusion of perlbrew and some dark magic to cook up the perl version then drop it in the container.
Use case is to wrap up apps and put resource limits on them. PAAS like really. I know uwsgi can use namespaces and what not, if someone has used that - please share!
|
eval point in hash ref
3 direct replies — Read more / Contribute
|
by ag4ve
on May 22, 2013 at 02:50
|
|
|
I'd like to be able to do:
./dat_query '{seek}'
or
./dat_query '{seek}{messages}'
and have code do:
print Dumper($nethist->{seek});
or
print Dumper($nethist->{seek}{messages});
I've been trying:
my $in = $ARGV[0];
my $val = eval '$nethist->$in';
print Dumper($val);
and the like with no luck
|
reverse sort arrays in subroutines
4 direct replies — Read more / Contribute
|
by JockoHelios
on May 21, 2013 at 17:34
|
|
|
I can sort an array passed into a subroutine, but I can't reverse sort or use a custom sort on that array.
I can't get subroutines to do much of anything with passed arrays. This is one of sevearl array-subroutine tests that I've tried over the last two days. I have tried variations with passing array references, but to no avail.
# test subroutines with an array
$var = 8; push( @TaR, $var ); $var = 2; push( @TaR, $var );
$var = 5; push( @TaR, $var ); $var = 4; push( @TaR, $var );
$var = 5; push( @TaR, $var ); $var = 6; push( @TaR, $var );
$var = 1; push( @TaR, $var ); $var = 8; push( @TaR, $var );
print STDOUT "\n\n"; print STDOUT "\n\nAs-is ";
# This segment works
for $OneLine( @TaR ) { print STDOUT " $OneLine"; }
print STDOUT " Done\n\n";
SortedArray( @TaR );
ReversedArray( @TaR );
print STDOUT "\n\n";
# This sort works
sub SortedArray
{
my @PassedArray = @_;
my @SortedArray = sort @PassedArray;
print STDOUT "\n\nSorted ";
for $SubLine( @SortedArray )
{ print STDOUT " $SubLine"; }
print STDOUT " Done\n\n";
return 1;
}
# This sort, but doesn't reverse the sort
sub ReversedArray
{
my @PassedArray = @_;
my @SortedArray = sort reverse @PassedArray;
print STDOUT "\n\nRvrsd ";
for $SubLine( @SortedArray )
{ print STDOUT " $SubLine"; }
print STDOUT " Done\n\n";
return 1;
}
|
Mac to Windows Data Incompatibility
No replies — Read more | Post response
|
by goshawk
on May 21, 2013 at 15:49
|
|
|
Hello, all. I'm having an issue with a large and difficult-to-produce piddle matrix that I made on a Mac OS with 64 architecture(8-byte long), which I want to use in a perl program on a Windows OS with 64 architecture (4-byte long). If at all possible, I want to avoid recreating this data.
I want to load in a matrix, slice a row, then calculate the Pearson's correlation of that row against each other row using PDL::Stats::Basic corr. This works fine when I create a matrix in Windows and find the correlation in Windows, and it also works when I use the data I want to use to find the correlation on Mac, but not when I use data from the Mac OS in Windows. I've tried using PDL::IO::Storable and PDL::IO::Dumper. When I save/load the matrix using PDL::IO::Storable, I get this error:
Long integer size is not compatible at C:/Perl64/lib/Storable.pm line
+380, at test.pl line 6.
When I save/load the matrix using PDL::IO::Dumper, it loads fine, but the correlation no longer works. Here's a MWE:
# On Mac OS
use PDL;
use PDL::IO::Dumper;
my $a = random(8,150);
PDL::IO::Dumper::fdump($a, 'a.pldl');
# On Windows OS
use PDL;
use PDL::Stats::Basic;
use PDL::IO::Dumper;`
my $a = frestore('a.pldl');
my $b = $a->slice(':,3');
my $c = $a->corr($b);
print $c->slice('(0)'), "\n";
This code returns '-1.#IND', which is Windows for nan. Am I getting caught by the different data sizes again?
|
Soap::Lite Issue
1 direct reply — Read more / Contribute
|
by rbala
on May 21, 2013 at 15:04
|
|
|
Hi Monks,
I have a Soap::Lite running in server side , which I am using to tranfer a network capture of wireshark . I am returning this as an array to the soap client . I am getting the following error :
Use of inherited AUTOLOAD for non-method Soap::Serializer::as_base64Binary is deprecated at /opt/Activeperl/site/lib/SOAP/Lite.pm.
.
Also , Soap server hangs therein.
Can you please help me to solve this ?
Thanks,
Bala.
|
IO::Uncompress::Gunzip to scalar takes hours (on windows)
4 direct replies — Read more / Contribute
|
by cmv
on May 21, 2013 at 11:40
|
|
|
Hi Monks-
I'm seeing a strange issue when using IO::Uncompress:Gunzip on windows that I can't explain. The same code runs fine under Unix. I'd like to understand why.
The code below will take about 20-seconds to unzip a 65M zipped file on my windows xp box.
If I comment out the first gunzip line, and uncomment the second gunzip line, it will take hours on the windows box (6 hours last time I bothered to try). Any thoughts?
Thanks
-Craig
#!/opt/exp/bin/perl5.8
use strict;
use warnings;
use IO::Uncompress::Gunzip qw(gunzip $GunzipError) ;
my $ifile = shift || die "Missing input file name\n";
open(IFILE, "<$ifile") || die "Can't open $ifile";
binmode(IFILE);
my $inputstr = join('', <IFILE>);
print STDERR scalar(localtime), " - STARTING UNZIP!!!\n";
# This is fast on all platforms
gunzip \$inputstr => "clearfile" or die "gzip failed: $GunzipError\n";
my $cleartxt;
# This takes forever on windows, no problem on Unix
#gunzip \$inputstr => \$cleartxt or die "gzip failed: $GunzipError\n";
print STDERR scalar(localtime), " - UNZIPPED COMPLETE!!!\n";
UPDATE1:
Here is some profile data I got from running both cases on a smaller gzipped input file (4.8M)...
Profile Data for FAST Method
Total Elapsed Time = 3.667486 Seconds
User+System Time = 1.200486 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
37.2 0.447 0.447 7281 0.0001 0.0001 Compress::Raw::Zlib::infl
+ateStream
::inflate
19.8 0.238 1.296 1 0.2377 1.2963 IO::Uncompress::Base::_rd
+2
14.9 0.179 0.200 7293 0.0000 0.0000 IO::Uncompress::Base::sma
+rtRead
8.66 0.104 0.979 7281 0.0000 0.0001 IO::Uncompress::Base::_ra
+w_read
6.25 0.075 1.059 7283 0.0000 0.0001 IO::Uncompress::Base::rea
+d
5.66 0.068 0.068 7281 0.0000 0.0000 IO::Uncompress::Base::pus
+hBack
4.66 0.056 0.056 14575 0.0000 0.0000 IO::Uncompress::Base::sav
+eStatus
3.25 0.039 0.039 14562 0.0000 0.0000 U64::add
3.08 0.037 0.496 7281 0.0000 0.0001 IO::Uncompress::Adapter::
+Inflate::
uncompr
1.92 0.023 0.224 7281 0.0000 0.0000 IO::Uncompress::Base::rea
+dBlock
1.33 0.016 0.016 1 0.0160 0.0160 IO::bootstrap
1.33 0.016 0.076 4 0.0040 0.0191 main::BEGIN
1.33 0.016 0.060 9 0.0018 0.0067 IO::Uncompress::Gunzip::B
+EGIN
1.25 0.015 0.015 10 0.0015 0.0015 strict::unimport
1.25 0.015 0.030 37 0.0004 0.0008 IO::Compress::Base::Commo
+n::BEGIN
Profile Data for SLOW Method
Total Elapsed Time = 154.4478 Seconds
User+System Time = 154.2188 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
98.6 152.1 152.14 7286 0.0209 0.0209 Compress::Raw::Zlib::infl
+ateStream
0 ::inflate
0.34 0.519 0.573 7298 0.0001 0.0001 IO::Uncompress::Base::sma
+rtRead
0.28 0.438 154.04 7286 0.0001 0.0211 IO::Uncompress::Base::_ra
+w_read
0.18 0.283 152.49 7286 0.0000 0.0209 IO::Uncompress::Adapter::
+Inflate::
9 uncompr
0.18 0.275 154.31 7288 0.0000 0.0212 IO::Uncompress::Base::rea
+d
0.10 0.158 0.158 14585 0.0000 0.0000 IO::Uncompress::Base::sav
+eStatus
0.10 0.151 0.151 7286 0.0000 0.0000 IO::Uncompress::Base::pus
+hBack
0.09 0.134 0.134 7288 0.0000 0.0000 IO::Uncompress::Base::sma
+rtEof
0.08 0.128 0.128 14572 0.0000 0.0000 U64::add
0.04 0.060 0.060 21864 0.0000 0.0000 Compress::Raw::Zlib::__AN
+ON__
0.02 0.025 154.34 1 0.0248 154.34 IO::Uncompress::Base::_rd
+2
0.01 0.016 0.016 4 0.0040 0.0040 Compress::Raw::Zlib::cons
+tant
0.01 0.016 0.016 8 0.0020 0.0020 Exporter::export
0.01 0.016 0.016 7 0.0023 0.0023 IO::File::BEGIN
0.01 0.016 0.016 26 0.0006 0.0006 Compress::Raw::Zlib::BEGI
+N
UPDATE2:
I believe Corion and BrowserUk are correct about this issue. Watching Pagefaults with Process Explorer, here is what I see:
~26,000,000 Pagefaults - Slow Method
~6,000 Pagefaults - Fast Method
UPDATE3:
I believe this talks about the root cause:
https://groups.google.com/forum/?fromgroups#!topic/perl.perl5.porters/44PTHwefYUk
|
How to improve this data structure?
7 direct replies — Read more / Contribute
|
by fiddler42
on May 21, 2013 at 10:31
|
|
|
Hi,
I recently made the rookie mistake of authoring a perl script with a sample data input file that was not realistic in size. So now I am running the script with much larger data input files and it is painfully slow. Here is the root cause of the problem:-
1) A data input file is parsed, and each line of the file is added to an array like so:-
push (@StatsArray, {RegionNum=>$RegionNum,
AR=>$AR[$RegionNum],
BCR=>$BCR[$RegionNum]});
I like the data structure above because it is easy to understand and I can add new elements to the array as the script becomes more sophisticated. I included only 3 of 15 variables, and I expect the number of variables to increase a little more.
2) Once the array is fully populated (millions of entries) and I need to pluck data from it, I need to do two things: a) *always* numerically sort by RegionNum first and then b) numerically sort some other key. Here is an example:-
foreach $RegionCoords (@AllRegionCoords) {
$RegionNum++;
foreach (sort {$$a{RegionNum} <=> $$b{RegionNum} or $$a{AR} <=> $
+$b{AR}} @StatsArray) {
$RegionKey = $$_{RegionNum};
$ARKey = $$_{AR};
if ($RegionKey == $RegionNum) {
# do stuff with sorted AR data in region n
}
}
}
Again, easy to follow. The problem is there are many different region numbers and I am not breaking down the data into smaller chunks. For example, although there may be 1 million entries in @StatsArray, all of the entries are made up of, say, 10 region numbers with 100,000 entries in each one. So instead of sorting only 100,000 entries at a time, I am always sorting 1 million entries at a time. A lot of separate sorts are issued, hence the runtime problem.
I have two questions:-
1) How can I reformat the StatsArray to be dependent upon region numbers and make sorting faster? 10 StatsArrays instead of 1 would be fine.
2) If a new data structure is proposed, how would I numerically sort something like the AR key? This is really what I am struggling with the most: ways to sort only 1 of n keys in an array when its data structure gets more complicated.
I do not use arrays like this very often, so any suggestions would be much appreciated.
Thanks!
-fiddler42
|
|
|
New Meditations
|
Memorizing The s/// Option List For Fun and Profit
1 direct reply — Read more / Contribute
|
by QM
on May 23, 2013 at 08:34
|
|
|
Did you ever want to memorize the s/// option list? The full list (from perlop) is msixpodualgcer.
Here are some anagrams I found:
* discourage p/m/xl (petite, medium, extra-large)
* glamorised x cpu
* Proclaimed, "g sux!" (oft heard?)
* xl CPU ideograms (extra-large CPU)
* goru x misplaced ("guru is misplaced")
* ex-cpu marigolds
* dog pux miracles (poo or pukes)
Can anyone come up with something better?
-QM
--
Quantum Mechanics: The dreams stuff is made of
|
What is the impact of 5.18.0?
2 direct replies — Read more / Contribute
|
by Tux
on May 19, 2013 at 06:53
|
|
|
I use my laptop as test environment. When all (upgrades) pass on production/utility software, I upgrade my workstation. If that shows no problems after a few weeks, I start upgrading other production servers.
So I installed 5.18.0 on my laptop as default perl, and then tried to install all CPAN modules I ever used in 5.16.x
There are just a few things that stand out as a reason for FAILure:
- Test files that use for $foo qw( … ) { which is now deprecated. The module autor should rewrite that to for $foo (qw( … )) {. I filed RT tickets for all I encountered. Lets hope the authors will fix their code.
- POD failures. As the POD checker is now way stricter than the old version was, it shows more real trouble. And it should! The problem is that many authors ship the pod tests too, so the test suite will fail if Test::Pod and friends are installed, and thus cpan won't install those modules. That means manual work ignoring the pod errors (and when obvious file RT tickets). Authors: unless you keep in sync with reality and release often, please do not include pod tests in you distribution.
- Loads of warnings when a module uses given/when as those are now marked experimental. I can ignore the warnings when testing, but will they fill up my logs when run in production code? When the module also uses Test::NoWarnings, one cannot install. A module like MooseX::App now won't install, and it causes many other modules that have this as a prerequisite to fail too.
- XS authors that never updated ppport.h from Devel::PPPort. Easy to fix.
- Modules that use Module::Install but ship ancient or incomplete versions of it in inc. The easiest solution for me was to just recursively remove inc/ and all problems vanish. I have no tuits to RT all of them.
- Several modules fail because they declare optional deps that are actually hard deps, like using JSON::XS, Lexical::Sub or Data::Alias - which won't build under 5.18.0
- Test failures, or maybe even code failures, due to hash randomization. Unless I am absolutely sure that the test is wrong, I cannot install these modules.
- Some failures have been reported to the authors but the available fix has not (yet) been applied (like in Template::Toolkit RT#84778) or not yet released (Tk and SQL::Statement). When their repository is open, they build fine from the repo and I am confident the next release will build fine.
The message is: perl-5.18.0 is awesome, but please please test the module you require before starting the installation on machines other than a test environment.
Enjoy, Have FUN! H.Merijn
|
|
|
New Cool Uses for Perl
|
Check network MTU size
1 direct reply — Read more / Contribute
|
by 5mi11er
on May 22, 2013 at 10:47
|
|
|
I recently ran into a bunch of problems where our MPLS provider inadvertently modified the MTU sizes of several of our locations. Unfortunately at about the same time we had swapped out our firewall, so after spending significant time thinking we had weird issues with the new firewall, we finally discovered the actual problem was an incorrect MTU size for those sites.This wasn't the first time the MTU sizes had been monkeyed with, so I decided to throw something together that might help us identify MTU problems more quickly. I'd found Network Duplex speed test so, I used much of that for the actual ping packet creation, which referenced Net::Ping code, so I also compared the current version
of that against what was given in the duplex test node. Then I had to figure out how to turn on the don't fragment flags of the packet. Anyway, the results are below. Hopefully someone else will find this useful. -Scott
|
|
|
|
