Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Seekers of Perl Wisdom

( #479=superdoc: print w/replies, xml ) Need Help??

If you have a question on how to do something in Perl, or you need a Perl solution to an actual real-life problem, or you're unsure why something you've tried just isn't working... then this section is the place to ask. Post a new question!

However, you might consider asking in the chatterbox first (if you're a registered user). The response time tends to be quicker, and if it turns out that the problem/solutions are too much for the cb to handle, the kind monks will be sure to direct you here.

User Questions
Uncompress streaming gzip on the fly in LWP::UserAgent/WWW::Mechanize
1 direct reply — Read more / Contribute
by Your Mother
on Dec 18, 2018 at 17:15

    I include only the bare bones because I tried something like 20 different things without success and I'm embarrassed. :( Non-streaming requests are working perfectly with approximately this code. The endpoint for this code is a streaming quote and trade ticker. The URL shown just asks for the info for APPL. The compressionówell, the ongoing, memory sane handling of itóis the only problem I have.

    The # $data is where the handling, or a core part of it, is missing.

    The closest I get streaming to work is gunzip from IO::Uncompress::Gunzip. It works fine but it's a standalone conversion which is goofy/impossible for an "infinite" stream of data; append, gunzip, append, gunzipÖ It must be uncompressed in toto. So a non-starter but I could see it works fine at least. Whatever the solution, I guess it will require IO handle truncation as it goes.

    I played with opening a pipe to gunzip(1), I tried every variation of Compress::Zlib I could think but wasn't able to get any data out or apparently even write/uncompress the data properly. I tried everything I could figure out with IO::Uncompress::Gunzip too. A few PM and SO nodes looked promising but I didn't find anything that seemed to apply directly or fix it. I even tried PerlIO::gzip. I'm just stick stupid on this. Maybe one of the attempts was close but I have no idea which, and, again, I'm embarrassed to share any of the *many*.

    The doc page I tried to read the sample streaming code for other languages but there is no gzip handling surfaced in them. If they work, it's hidden in the libraries used.

    #!/usr/bin/env perl use strictures; no warnings "uninitialized"; use WWW::Mechanize; use WWW::OAuth; # You have to have a trading account to use the service- my $oauth = WWW::OAuth->new( client_id => "...", client_secret => "...", token => "...", token_secret => "..." ); my $mech = WWW::Mechanize->new( autocheck => undef ); $mech->add_handler( request_prepare => sub { $oauth->authenticate($_[0 +]) } ); $mech->default_header( Accept => "application/json" ); $mech->add_handler( response_data => sub { my ( $response, $ua, $h, $data ) = @_; # $data; # Handle the gzip data. $response->content(undef); 1; }); $mech->get(" +");

    On the plus side, the WWW::OAuth was trivial to mix in and works great for this.

    Thanks for looking!

Alternative for multiple while loops to improve performance
2 direct replies — Read more / Contribute
by MissPerl
on Dec 18, 2018 at 10:13
    I wrote this script, but it's not performance-wise, so i try to improve it,

    I am lack of knowledge to know how can I possible improve this

    So was hoping fellow perl monks can point me into the direction so I can get a start from there.

    this script is used to find the first decreased price, will get 1.67 in this case to insert to db.

    3.88 1.67 1.53 1.52 + - -
    i got multiple tables in database, fruit_load , fruit_pricing country zipcode city name mgroup subgroup fruit special_fruit fruit_price special_fruit_price To get the pricing for each fruit in each group in each city, zipcodeand country, I do multiple while loops,
    SELECT DISTINCT (country, zipcode, city) FROM fruit_load WHERE fruit_l +oad.sold_date BETWEEN NOW() - INTERVAL 60 DAY AND NOW(); SELECT DISTINCT (owner) FROM fruit_pricing b JOIN fruit_load a ON +uit_load_id = b.fruit_load_id WHERE '$data[0]' AND a.zipco +de = '$data[1]' AND = '$data[2]'; SELECT DISTINCT (mgroup) FROM fruit_pricing b JOIN fruit_load a ON a.f +ruit_load_id = b.fruit_load_id WHERE = '$data[0]' AND +code = '$data[1]' AND = '$data[2]' AND b.owner ='$data2[0]'; SELECT DISTINCT (fruit) FROM fruit_pricing b JOIN fruit_load a ON a. f +ruit_load_id = b. fruit_load_id WHERE = '$data[0]' AND a.zi +pcode = '$data[1]' AND = '$data[2]' AND b.owner = '$data2[0]' +AND b.mgroup = '$data3[0]'; SELECT * FROM fruit_load a JOIN fruit_pricing b ON a.fruit_load_id = b.fruit_load_id WHERE b.sold_date BETWEEN NOW() - INTERVAL 180 DAY AND NOW() AND = '$data[0]' AND a.zipcode = '$data[1]' AND = '$data[2]' AND b.owner = '$data2[0]' AND b.mgroup = '$data3[0]' AND b.fruit = '$data4[0]' ORDER BY sold_date DESC; put found row into array in this loop SELECT DISTINCT (subgroup) FROM special_fruit_price c JOIN fruit_load +a ON a.fruit_load_id = c.fruit_load_id WHERE = '$data[0]' A +ND a.zipcode = '$data[1]' AND = '$data[2]' AND c.owner = '$dat +a2[0]'; SELECT DISTINCT (special_fruit) FROM special_fruit_price c JOIN fruit_load a ON a.fruit_load_id = c.fruit_load_id WHERE = '$data[0]' AND a.zipcode = '$data[1]' AND = ' +$data[2]' AND c.owner = '$data2[0]' AND c.mgroup = '$data3[0]' AND c.subgroup = '$data6[0]'; SELECT * FROM fruit_load a JOIN special_fruit_price c ON a.fruit_load_id = c.fruit_load_id WHERE c.sold_date BETWEEN NOW() - INTERVAL 180 DAY AND NOW() AND = '$data[0]' AND a.zipcode = '$data[1]' AND = '$data[2]' AND c.owner = '$data2[0]' AND c.mgroup = '$data3[0]' AND c.subgroup = '$data6[0]' AND c.special_fruit = '$data7[0]' ORDER BY sold_date DESC; put found row into array in this loop
    Let meknow if I can provide more information, I can give the whole code, but I try to makeit short and sweet.

    The code currently run 15 mins to 2 hours varies, is it possible to shorten the time ?

    I believe the multiple insert will slow down the process, at current I am trying to put them into array and insert at once.
Push array into array of arrays
5 direct replies — Read more / Contribute
by Anonymous Monk
on Dec 18, 2018 at 09:12
    push (@myArrofArray, \@myArray);

    1) I tried to print @myArrofArray, but i get this "ARRAY(0x7c8f60)" which I couldn't get any relevant results on Google. :(

    I tried to use perl debugger, x Dumper \@myArrofArray

    but there is still one original array in it, without the new (@myArray)

    May I know if the line is correct?

    If it is correct, then I have trace back to check maybe the array isn't being imported into array in the first place, as it was retrieved sql results.

    2) I am trying to get last array in the myArrofArray,


    print @a, I got ARRAY(0x858bd8)

    expected output: @a = ('x', 'y', 'z', 8,9,7);

    Can anyone point out the mistake?

    I can provide more information if it is needed. Thanks!!
Reorganizing the content of a file
8 direct replies — Read more / Contribute
by ovedpo15
on Dec 18, 2018 at 04:44
    Consider the following content of a file:

    15,10,name3 10,#,name1 12,10,name2 5,12,name4 8,5,name4

    Each line is in the following format:

    Notice that the the great parent (the one without a parent) has # in the parent-id field.
    I'm trying to create a new list which each line is a path from each id to its parent-id.
    The expected output of the example I gave:

    name1 name2,name1 name3,name1 name4,name2,name1 name5,name4,name2,name1

    I was trying to solve this issue with a loop and an array.
    Problem is, I don't know how many fields should the array contain.
    How can I solve this issue in the cleanest and simplest way possible, without using any external modules?
    I feel like a loop over the file should solve it, but can't figure the right order.
Binding Perl tk table scrollbars with keyboards up and down arrow keys
1 direct reply — Read more / Contribute
by Anonymous Monk
on Dec 17, 2018 at 12:28

    Hi Monks, I have table with scrolled option used with table,here I have labels embedded on each table row, By default scrollbar visible area for vertical scrollbar is for 10 rows.
    My question is how to adjust the scrollbar visible area as we we move up and down using keyboard up and down keys.
    Let's say I have 20 rows, I want to adjust the vertical scrollbar visible area, when I move down from row 10 or move up from current visible area.

LWP::UserAgent timeout
3 direct replies — Read more / Contribute
by IB2017
on Dec 17, 2018 at 08:04


    I use the following to retrieve from server the version number of the latest available update of my software.

    my $ua = LWP::UserAgent->new; my $req = HTTP::Request->new(GET => $url); my $res = $ua->request($req); $ua->timeout(0.1);

    This works fine execpt if there is a connection problem. For example, the server has been down for the whole weekend now (!!!). In such a case, my GUI application - while running this peace of code - freezes for ca. 15 seconds. This is not acceptable. What I want is that if there is no response from the server, no matter the reasons, the application just skip this check. I tried to replace LWP::UserAgent with LWPx::ParanoidAgent, but this did not lead to any improvement. Any other idea?

Prompt to save file working with excel::writer::xlsx for desktop application
2 direct replies — Read more / Contribute
by Anonymous Monk
on Dec 17, 2018 at 01:54

    How to make desktop application Prompt to save excel file with prepopulated filename, working with excel::writer::xlsx. Thanks

Log::Any with die and carp
2 direct replies — Read more / Contribute
by learnedbyerror
on Dec 16, 2018 at 13:43

    Hello Oh Wise Monks

    The village idiot returns with another request for your guidance. I have long been a devotee of the most excellent Log::Log4perl. However, I am currently working on a module that I think may benefit others and will place it on CPAN. As such, I plan to change the log producer to Log::Any. For most of my logging calls, this is a trivial change. There are several cases where the change is not so trivial. They are replacing $logger->logdie (which I shouldn't be using in a module anyway ) and more importantly the Log4perl levels that handle Carp: $logger->log(carp|cluck|croak|confess).

    My question is, "what are your recommended means of integrating Log::Any where die and Carp functionalities are needed?"

    Some of the options are not too disruptive, replace $logger->logdie(...)with $logger->logcritical(...) && die(...)

    Whereas cluck or confess would be something more like

    $logger->critical(...); my $long_message = Carp::longmess; foreach my $line ( $long_message =~ m/^.+$/msg ) { $logger->critical( $line ); } confess(...)

    NOTE: untested code example, intended for representative use only

    While functional, it seems inelegant to have to add these boilerplate lines or add a subroutine or class method to handle this. Are their more elegant ways of handling this need with Log::Any?

    Thanks in advance for your sage advice!


How to Extract Calendar Events in a Specified Date Range from 'Internet Calendaring' (.vcs, .ics) Files
2 direct replies — Read more / Contribute
by ozboomer
on Dec 15, 2018 at 03:38

    (..."Internet Calendaring"!? *Ugh* What a mouthful...)

    I have an Android smartphone and I use MyPhoneExplorer ("MPE") on a Windows 8 desktop to manipulate the calendar(s) in the phone.

    A feature of MPE is that it can export the calendar data in a few formats, being:-

    • vCalendar (.vcs);
    • iCal (.ics); and
    • XML (.xml)

    I have no knowledge of processing these formats... and so, I'm looking for some pointers on how I might approach dealing with the requirements I have to process this data.

    I'm hoping it makes sense to do something like:-

    Magic_Processing(input_file, events_structure) for date is startdate to enddate if exists events_structure{date} event_detail = events_structure{date} further_process(date, event_detail) endif endfor

    ...where Magic_Processing() is "aware" of the different types of events so the pseudocode I've shown actually works. This means the function would understand about:-

    • a 'one shot' event with a single occurrence date; OR
    • a repeating event, specified with: (a start date + no. of repeats) OR (a start date and an end date) OR ...other forms I don't (yet) know...

    ...or something like that.

    Simplified, I have a lot of repeating events... and I want to be able to specify a date range and have all the repeating events that will occur in that date range added to a list. Looking at the exported files, entries like:

    (in an .ics file) are sort-of meaningless, straight off the bat.

    I understand RFC 5545 and possibly RFC 3283 are (probably?) the documents I should attack (at least, for the .ics files, which seem sort of the "most often seen"(!?))... but dealing with detailed RFCs like that is like trying to read microfilm through a mud-filled beer bottle to me... hence this request for some pointers on where to start.

    (Apologies if this is sacrilege to write in the Monastery) ...even if there's some Python code I could examine which would give some clues (as I'm trying to learn this language at present)...

    Thanks a heap for any thoughts or pointers.

create clone script for utf8 encoding
6 direct replies — Read more / Contribute
by Aldebaran
on Dec 15, 2018 at 02:34

    Just as soon as ikegami made me aware that us-ascii is my default encoding, I seem to be developing problems with it. These problems are in the variety of my computer not behaving the way I expect it to.

    I'm running ubuntu with bash, and when I touch a file into existence, it is us-ascii. Likewise, files that are formed from redirecting STDOUT begin their lives as us-ascii on this platform. Where is this determined on POSIX systems?

    So this is a day in the life, where I use this nifty software: translate shell

    $ trans :de -brief "over" > $ trans :de -brief "He must." >> $ cat ‹ber Er muss. $ iconv -f us-ascii -t UTF-8 -o iconv: illegal input sequence at position 0 $

    On STDOUT for me, I get ‹ as the zeroth character. Does ascii have a representation for ‹?

    Then I keep trying to get an iconv command to do something for me, but an effective syntax eludes me. Why is ‹ illegal in the iconv command?

    If I'm going to have source that has utf8 characters in it, doesn't it make sense to change the underlying encoding to utf8 or create it that way from the git-go?

    After I've touched a file into existence, I use a bash script to clone the next version of a script. All of my scripts have a taxonomy of a positive integer followed by a period, followed by a word. The cloned script is incremented, given execute privileges, and has its name written to a manifest. There isn't any language in it for determining the underlying encoding. I've gotten a lot of mileage out of this script, but I think it's time that I need to replace it with shiny new, lexical perl. I'll put it in readmore tags for being somewhat OT:

    I'd like to write a perl equivalent that would give me freedom to choose the underlying encoding. I'd show previous attempts, but they look awful.

    Finally, what makes any of these en_**.utf8 encodings different from another?

    $ locale charmap UTF-8 $ locale -a C C.UTF-8 en_AG en_AG.utf8 en_AU.utf8 en_BW.utf8 en_CA.utf8 en_DK.utf8 en_GB.utf8 en_HK.utf8 en_IE.utf8 en_IL en_IL.utf8 en_IN en_IN.utf8 en_NG en_NG.utf8 en_NZ.utf8 en_PH.utf8 en_SG.utf8 en_US.utf8 en_ZA.utf8 en_ZM en_ZM.utf8 en_ZW.utf8 POSIX ru_RU.utf8 ru_UA.utf8 $ locale -m ANSI_X3.110-1983 ANSI_X3.4-1968 ... UTF-8 VIDEOTEX-SUPPL VISCII WIN-SAMI-2 WINDOWS-31J $

    Thanks for your comment

Add your question
Your question:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others exploiting the Monastery: (2)
    As of 2018-12-19 03:20 GMT
    Find Nodes?
      Voting Booth?
      How many stories does it take before you've heard them all?

      Results (83 votes). Check out past polls.