Problems? Is your data what you think it is? PerlMonks

### The Monastery Gates

 ( #131=superdoc: print w/replies, xml ) Need Help??

and Create a new user.

New Questions
 Can't locate object method "" 2 direct replies — Read more / Contribute by chris212 on Dec 05, 2016 at 11:32 Does anyone know what may cause this error? Can't locate object method "" via package "IO::File" at G:\path\to\mys +cript.pl line 1189 [download] It happens very intermittently. I have run the script thousands of times in a loop unable to reproduce the problem, but I have seen it twice. This is line 1189: my $chunk =$th->join(); [download] That line is in an "output" thread. It is using a file handle, and will write the data contained in $chunk (2D array reference) to a file using Text::CSV_XS. The thread it is joining is one of many "worker" threads which do not use file handles. The "input" thread which starts the worker threads does use a file handle and reads a different file using Text::CSV_XS. The "input" and "output" threads both require the Text::CSV_XS module (depending on arguments passed to script), so they will both load the module rather than inherit it from the parent thread. Using Perl 5.18.2 on Windows. I have not seen it on Linux yet. Memory usage while tallying instances of lines in a .txt file 4 direct replies — Read more / Contribute by TJCooper on Dec 05, 2016 at 11:30 Given input data in the form of: c 8 336158 75 75M 74 c 12 828707 74 74M 73 w 10 528559 74 74M 0 c 15 267766 74 74M 73 c 12 828707 74 74M 73 c 14 491797 74 74M 73 [download] I am trying to tally the instances of records based on columns 1 (which has the header 'Strand' - this can be variable in position hence the use of List::Util qw(first)) as well as columns 2 and 3. The main chunk of code that accomplishes this is simply:  my @headers = split("\t",<$IN>); my $index = first{$headers[$_] eq 'Strand'} 0..$#headers; while (<$IN>) { chomp$_; my @F = split("\t", $_); if (exists$hits{$F[$index+1]}{$F[$index+2]}) { } else { $hits{$F[$index+1]}{$F[$index+2]}{'w'} = 0;$hits{$F[$index+1]}{$F[$index+2]}{'c'} = 0; } $hits{$F[$index+1]}{$F[$index+2]}{$F[$index]}++ } [download] This is then printed in a simple manner to form files like these: 1 4 1 0 1 5 1 0 1 31 1 0 1 74 1 0 1 89 1 0 1 116 1 1 1 118 1 0 1 122 1 0 1 126 0 1 1 140 0 1 1 141 0 1 1 148 2 0 1 158 0 1 1 159 1 0 [download] Column 2 and 3, along with the frequency counts of each for W and C. This approach however requires a rather a lot of memory - around 800MB for an input file of ~100Mb. Are there any clever tricks or alternative methods that I could use in order to reduce the memory requirements? I note that for any given column 2-column 3 combination, a key and a blank (zeroed) value is stored the first time it is encountered - this is done as the output file is required in the format shown above where '0' is filled in. This may be increasing memory usage further when the zeros could be added afterward (perhaps during printing), but i'm entirely sure or how I would do this. Testing Wrapped LDAP Classes 1 direct reply — Read more / Contribute by yulivee07 on Dec 05, 2016 at 10:39 Hello fellow Perl-Monks, I am expanding my knowledge in testing right now. I want to test some functions of mine which interact with an LDAP-Server. This might sound like a stupid question, but what is the best way to deal with Wrapper-Classes? Context: I write a module, which gets some Users from LDAP. To communicate with the LDAP, I use a provided module from my workplace which wraps some system-specific parameters in the LDAP-methods (Utils::Ldap::CompanyLdap) and provides some additional methods. This module again is a subclass of another LDAP-Wrapper Module (Utils::Ldap::Base) which finally uses Net::LDAP. Now, in my program I use the first Ldap-Module e.g. package MyAwesomeProgram; use Utils::Ldap::CompanyLdap; sub read_users { # code that reads users from Ldap # searchGetEntries/get_value is provided by Utils::Ldap::CompanyLda +p foreach my$entry ( $ldap->searchGetEntries() ) { my$uid = $entry->get_value( 'uid' ); #... do something more } } 1; [download] I want to test the read_users function of my program, which gets its users via Utils::Ldap::CompanyLdap. How do I mock this? As you see, I do not directly interact with a Net::LDAP-Object I have already understood that there is Test::Net::LDAP::Mock/Test::Net::LDAP::Utils qw(ldap_mockify) which can mock an Ldap Connection. What I do not understand yet: How can I use this in conjunction with my Wrapper Module? My module uses the code and the functions of our Ldap-Wrappers. Would the correct way be to write mocking code for the wrappers (e.g. Utils::Ldap::CompanyLdap::Mock), or is there a way of overwriting the Net::LDAP Object which is used by the Base-Wrapper Class Utils::Ldap::Base? I am not shure how to best approach this. Second Question: In case I get this working, is there a way of taking a Net::LDAP object and feeding it into my Mock-Object? I'd like to copy the datastructe of our LDAP to the Mock-Object. Kind regards, yulivee Capacity Planning 3 direct replies — Read more / Contribute by Aatus on Dec 05, 2016 at 10:25 I am VERY new to Perl but know its power and really need help to create a script that: a. Uses Perl::SSH b. SSH into appliances using username:password c. OPens a notepad or excel for IP address list d. Pings particular IP e. Writes to each IP the average latency d. Moves to next IP and logs into different appliance and repeats process Printing to STDOUT buffered when invoking Perl within a bash-script using tee 1 direct reply — Read more / Contribute by TJCooper on Dec 05, 2016 at 10:01 As a solution to recording terminal output into a log file, whilst still viewing output on the terminal (using code within a bash script itself rather than piping to tee on the command-line) I typically use: exec > >(tee "$DIR/Logs/System.log") Near the top of the bash script. This bash script eventually invokes a Perl script. Output printed to the terminal comes from both Bash and Perl, however, the inclusion of the above line causes the buffering of the Perl-specific portion of the output such that it's only displayed on the terminal once the script fully completes. Running the Perl script as a standalone provides the intended real-time updates, as does invoking it via Bash without the above line. Could anybody explain why this is happening and also suggest a solution? The ultimate goal here is to provide real-time feedback to the user on the terminal, whilst keeping a hard-copy of this feedback in a log file. Sorting array 4 direct replies — Read more / Contribute by negativ_m on Dec 05, 2016 at 09:52 Hello All, I have an array which looks like this: my @ar1 = (text1.txt, text1a.txt, text2.txt, text54.txt,...text1g.txt,...text54f.txt,...); I would like to have always the last version of my names inside the array. The result should look like this: my @ar2 = (text2.txt,...text1g.txt,...text54f.txt,...); I would appreciate some help. Thank you in advance. [Perl6] Slurpy array of named arguments? 2 direct replies — Read more / Contribute by ctilmes on Dec 05, 2016 at 09:16 I can ask for a slurpy hash of named arguments with '*%args', or a slurpy array of non-named arguments with '*@args', but I want an ordered list of Pairs (specified with 'name => value'). Is there a good way to get that? class testing { method newhash(*%args) { for %args.kv -> $k,$v { say "Hash Key $k => Value$v"; } } method newarray(*@args) { for @args -> $k,$v { say "Array Key $k => Value$v"; } } } my $x = testing.newhash(a => 1, b => 2); # Messes with specified ord +er my$y = testing.newarray(c => 3, d => 4); # Looks good, but doesn't w +ork my $z = testing.newarray('e', 5, 'f', 6); # Works, but ugly [download] Hash Key b => Value 2 Hash Key a => Value 1 Array Key e => Value 5 Array Key f => Value 6 [download] Pattern matching and deriving the data between the "(double quotes) in HTML tag 6 direct replies — Read more / Contribute by sp4rperl on Dec 04, 2016 at 22:29 #/usr/bin/perl my$timeLimit = ''; $timeLimit =~ m/startTime="(.*?)"/g; my$startTime = $1; chomp($timeLimit); if ($timeLimit =~ m/endTime/) {$timeLimit =~ m/endTime="(.*?)"/g; my $endTime =$1; chomp($endTime); } print "[$startTime],[$endTime]\n"; [download] Hello fellow monks!! I want to get the date that is enclosed in between the "". Please help me in understanding what is the change that is needed in the code to obtain the desired output. desired output: [2016-09-30T00:00:00],[016-12-28T23:59:59] unexpected output obtained: [2016-09-30T00:00:00],[] How do I determine if a variable contains a type glob? 4 direct replies — Read more / Contribute by dpchrist on Dec 04, 2016 at 15:52 I would like to write a subroutine is_typeglob($) that returns true when passed a type glob and returns false when not passed a type glob. (I already have a subroutine is_typeglob_ref($).) This is a scratch program I wrote to experiment with some ideas: 2016-12-04 12:35:05 dpchrist@debian ~/sandbox/perl$ cat is_typeglob.pl #!/usr/bin/perl use strict; use warnings; use Data::Dumper; $Data::Dumper::Indent = 0; use Scalar::Util qw(blessed openhandle reftype); { no warnings 'once'; open (FH, '<',$0) or die "ERROR: open(): $!"; } for ( q(*ARGV ), q(*ARGV cmp '*ARGV' ), q(*ARGV cmp "*ARGV" ), q(*ARGV cmp '*::ARGV' ), q(*ARGV cmp "*::ARGV" ), q(ref *ARGV ), q(blessed *ARGV ), q(openhandle *ARGV ), q(reftype *ARGV ), ) { my$code = $_; my$r = eval $code; print Data::Dumper->Dump([$code, $r,$@], [qw(code r @)]), "\n"; } # $Id: is_typeglob.pl,v 1.1 2016/12/04 20:35:04 dpchrist Exp$ [download] Here is a sample run: 2016-12-04 12:42:14 dpchrist@debian ~/sandbox/perl $perl is_typeglob.pl$code = '*ARGV ';$r = *::ARGV;$@ = ''; $code = '*ARGV cmp \'*ARGV\' ';$r = 1;$@ = '';$code = '*ARGV cmp "*ARGV" ';$r = 1;$@ = ''; $code = '*ARGV cmp \'*::ARGV\' ';$r = 1;$@ = '';$code = '*ARGV cmp "*::ARGV" ';$r = 1;$@ = ''; $code = 'ref *ARGV ';$r = '';$@ = '';$code = 'blessed *ARGV ';$r = undef;$@ = ''; $code = 'openhandle *ARGV ';$r = undef;$@ = '';$code = 'reftype *ARGV ';$r = undef;$@ = ''; [download] How do I determine if a variable contains a type glob? sane way to configure perlbrew and h2ph on Ubuntu 1 direct reply — Read more / Contribute by glasswalk3r on Dec 04, 2016 at 13:53 Hello fellow monks! Meanwhile I'm working to release a new module to CPAN, I find out that Ubuntu and h2ph don't play nice with each other (at least when I using perlbrew). After running h2ph as documented, I find out that the perl was expecting to find the ph files in different locations that they are stored in /usr/include. Some research led me to find that this is because change to a "multiarch", whatever that means). Unfortunately, I didn't find anything that would help fix the issue. I was getting errors like: # Error: Can't locate bits/syscall.ph in @INC (did you run h2ph?) + (@INC contains: /home/me/Projetos/Linux-NFS-BigDir/.build/MHr69O96uB +/blib/lib /home/me/Projetos/Linux-NFS-BigDir/.build/MHr69O96uB/blib/a +rch /home/me/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0/x8 +6_64-linux /home/me/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5. +24.0 /home/me/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linu +x /home/me/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0 .) at /home/me +/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0/sys/syscall.ph + line 9. [download] In order to fix such things, I searched for the corresponding ph file generated by h2ph in $Config{'installsitearch'} and created a symbolic link to the directory by using the expected alias, for instance: ln -s /home/me/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0/x +86_64-linux/x86_64-linux-gnu/bits /home/me/perl5/perlbrew/perls/perl- +5.24.0/lib/site_perl/5.24.0/bits [download] That works... but I had to repeat that three times to fix all the errors. An educated guess is that I'm going to repeat the process for every instance of Perl installed with Perlbrew (I have several). Is there a cleaner way to do that? Maybe a different options when compiling perl? Thanks in advance, Alceu Rodrigues de Freitas Junior --------------------------------- "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill portability / system / elegance 3 direct replies — Read more / Contribute by skendric on Dec 04, 2016 at 08:19 I'm writing semi-portable code (Linux / Windows). I use 'system' to execute an external binary ('tshark', part of the Wireshark collection). I handle slashes like this: use File::Which qw(which);$tshark_binary = which('tshark'); $tshark_binary =~ s/\\/\\\\/g if$tshark_binary =~ /\\/; [download] The last line functions essentially as a "If this is Windows, escape the slashes in the path name", such that 'C:/Program Files/Wireshark/tshark.EXE' turns into 'C://Program Files//Wireshark//tshark.EXE'. And then, when I actually execute tshark, I use the following: system("\"$tshark_binary\" -r$pcap -e frame.number -e frame.time_epo +ch -e ip.src -e ip.id -T text -T fields -E separator=, > $temp_file") +; [download] The key 'portability' change being the escaped quotes around$tshark_binary, which aren't necessary in Linux but are needed under Windows, to dodge the: "C:\Program" is not an executable errors. This is fine, and it works. But is there a more elegant / common way to accomplish these two tasks? --sk Are we lacking behind in YAML spec versions? 2 direct replies — Read more / Contribute by mikkoi on Dec 02, 2016 at 11:55 The new API description language RAML uses YAML 1.2 as file format. But it seems that neither YAML, YAML::Tiny, YAML::XS nor YAML::Syck support YAML 1.2 spec, only 1.1. And 1.1 came out already in 2009. Situation can't be this?
New Meditations
 (Placeholder) Imagine! 5 direct replies — Read more / Contribute by BrowserUk on Nov 29, 2016 at 22:24 This place excels in starting with a single, clearly defined, abstracted problem, and refining the set of proposed solutions down to a clear, concise, efficient solution. Many hands make light work! Now, imagine if each of the Perl(5) opcodes was proffered here -- say: one per weekend -- on a Friday night - say: US West Coast time -- in the form of a hack-a-thon task for that weekend. What might result from that exposure of a clearly defined task to the assembled (and usually quite bored on weekends), diverse populous? If those more enlightened tolorant members of p5p were enthused to cast a critical eye over the proceedings -- to catch the less obvious pitfalls in the evolution of solutions -- then the results might be usefully fed back into the p5p process and result in benefits for all. Thoughts on: Is the idea viable? Thoughts on: Are any members of p5p willing to participate? Caveats? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity. In the absence of evidence, opinion is indistinguishable from prejudice.
New Cool Uses for Perl
picwoodpecker -- a Tk program to choose and modify your photos
3 direct replies — Read more / Contribute
by Discipulus
on Nov 30, 2016 at 03:37
Hello monks,

after months of sparetime works I'm proud to present you picwoodpecker my last Tk effort (now also on github).

It is application to review and copy pictures with many features.

Suggestions are welcome.Tests with exif tags from different cameras are even more welcome.

The post is splitted in 3 parts: here you have the rendered documentation, below there is the code and finally the POD that must be pasted after the __DATA__ token of the program.

               This software is dedicated to my parents, who need to see printed pictures -- 6 July 1966

# NAME

PicWoodPecker

# SYNOPSIS

perl picwoodpecker.pl [-s -d -debug -pr -wr -p -x -y -nothumbs -e -quality -pngcompression -df]

# OPTIONS

     -s|src|source        path
the path where search for jpg files to be loaded
Can be modified in the Tk interface

-d|dest|destination  path
path used to save files
Can be modified in the Tk interface

-debug
Can be modified in the Tk interface

-pr|phratio          floating point
the ratio used to display the current photo
Can be modified in the Tk interface

-wr|winratio         floating point
the ratio to size the window where the photo is displayed
Can be modified in the Tk interface

how many photos load in memory after and before the current
one. Can increase drawing speed time

-x|gridx             integer
how many columns in the thumbnail grid

-y|gridy             integer
how many rows in the thumbnail grid

-nothumbs
does not load thumbnails at all

-e|extension             jpg|gif|png|gd|gd2
the exstension of saved files
Can be modified in the Tk interface

-quality|jpegquality     0-100
the quality of the file used by GD when saving the current
photo in jpeg format
An empty value let GD to choose a good default
Can be modified in the Tk interface

-pngcompression          0-9
the compression factor used by GD when saving the current
photo in png format

-dateformat|df         string
the format used for dates. It defaults to %Y_%m_%d_%H_%M_%S
in such way resulting pics can be ordered correctly.
See C<strftime> in L<POSIX> to more hints about formatting.

# DESCRIPTION

The program is aimed to let you to easely choose among photos and save one (or more) copy in the preferred format (jpg as default; gif png gd and gd2 are also available). The name of the copy is crafted using the timestamp when the photo was taken.

Basically the program will load all jpg files found globbing the path given with command line parameter -source or entered in the graphical interface, and foreach file will examine some exif tags to get sizes, timestamps and the thumbnail (if not present showing a black empty one).

Orientation of the image is handled automatically for thumbnails and main photo.

Advanced options are available to manipulate how many photos are copied, in which format and let you to postprocess via exiftool each created image.

The program uses GD for image manipulation and Image::ExifTool to load infos from photos and in the postprocess of them.

# THE GRAPHICAL INTERFACE

A main control window and a display one are created. Optionally a third window is created to access the advanced copy options. The display window tends to take the focus being the program focused on displaying photos.

### control window

The control window contains:

• all controls to manipulate the photo list (&#39;browse&#39; &#39;new list&#39; &#39;view list&#39; &#39;add to list&#39; and &#39;clear list&#39;) Note the that the &#39;browse&#39; does not fill the list; you need to use &#39;new list&#39; or &#39;add to list&#39; after using it.
• an entry to choose the destination folder (that will be checked for existence)
• the photo ratio and the window ratio controls and the debug switch
• an informative text about the current displayed photo or about the grid of thumbnails
• the editable name of the current photo (and an eventual suffix) used to save it
• an information text about the status of the main program (with only relevant information about copying and loading operations as eventual errors)
• the save button and the advanced options one.
• controls to navigate the photo list

### display window

The display window will starts showing a grid of thumbnails. The first one is selected. You can navigate the grid using right arrow and left arrow of the keyboard to move the current selection on or back.

up arrow and down arrow let you load previous or next grids of thumbanails.

Enter key will open the currently selected thumbanil in a bigger resolution ( determinted by photo ratio parameter) photo filling the whole window.

When just one photo is displayed right arrow and left arrow of the keyboard can be used to show next and previous photo while up arrow and down arrow bring you back to the thumbnail view.

In both control and display window space bar can be used to save the current photo and p key can be used to toggle autoplay. If autoplay is active the time between photos can be set in the control window. Please note that specifying a short amount of time (shorter than the time needed to load photos data) can produce weird showing behaviours.

This menu lets you to have a granular control about how original photo will be copied.

The allow overwrite if checked silently overwrite a photo wich has the same name of what is composed for the current one.

bypass original file elaboration (simple copy) make a copy of the original file using the new name but without processing it with GD

output file type lets you to choose among different file fomrmats (jpeg, gif, png, gd and gd2) and to set the quality (0-100) for jpeg ones. For png files the compression factor (0-9) can be specified only via the command line parameter -pngcompression

enable multiple copies is trickier. If enabled lets you to choose to copy the file many times, each one with a different resolution. In the multi copies pattern you can specify different resolutions in the format widthxheigth as in 800x600 or 1024x768 and for each format a copy will be created.

enable post processing can be used to postprocess every image with exiftool program that ships with Image::ExifTool module. You can use alternative program to postprocess your image inserting the full path in the program field. Arguments to such program can be also specified where $can be used to refer to the full path of the original image. In the exiftool command line you can also specify @ to refer to the current file. So you can postprocess every copy using the following arguments: -overwrite_original -all= -tagsFromFile$ -ImageSize -ImageWidth -ImageHeight -ThumbnailImage -Orientation -DateTimeOriginal

This exiftool command will be issued for every copy made, overwriting each exif tags in the copy, removing them all but taking some tag frome the original file and applying them to the copy. See the exiftool page for a lot of options this wonderful program lets you to use.

# LIMITATIONS

The program was tested against few different camera formats; i dont know if exif tags extracted are widely valid.

The autoplay feature does not plays well with little time intervals: infact when the interval is smaller than the overall time taken to load the the photo and to redesign the display i'll notice photos and information skipped and the timer going on. I tried fixing this using waitVisibility Tk method, with no luck.

This software and icons are copyright of Discipulus as found on www.perlmonks.org You may redistribute or modify it under the same term of Perl itself.

L*

Create A New User
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2016-12-06 00:47 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
On a regular basis, I'm most likely to spy upon:

Results (95 votes). Check out past polls.