Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^2: LWP::Simple is it get decoded or not ?

by remiah (Hermit)
on Aug 11, 2012 at 04:07 UTC ( #986854=note: print w/replies, xml ) Need Help??


in reply to Re: LWP::Simple is it get decoded or not ?
in thread LWP::Simple is it get decoded or not ?

It was version difference of LWP::Simple to get decoded character or not decoded bytes.

5.810 of LWP::Simple doesn't use decoded_content() but it uses content() method. On the other hand 5.835 uses decoded_content() method, so it returns decoded character.

I decided to use useragent and decoded_content explicitly, to work with each enviornment. Below is test script with LWP::Simple 5.810

#!/usr/bin/perl use strict; use warnings; use lib './libwww-perl-5.818/lib'; use LWP::Simple; use HTTP::Response; use LWP::UserAgent; my $url='http://www.youtube.com/user/tbsnewsi/videos?sort=dd&view=0&pa +ge=1'; sub test1 { ### with get my($html); $html=get($url); print "perl=$]\n"; print "LWPsimple v=$LWP::Simple::VERSION\n"; print "HTTP::Response v=$HTTP::Response::VERSION\n"; print "url=$url\n"; my $message=(utf8::is_utf8($html)) ? "html is utf8 flaged" : "html + is not utf8 flaged"; print "$message\n"; } sub test2{ ### with useragent and requst my ($ua, $req, $res, $html); $ua = LWP::UserAgent->new; $ua->agent("test user agent"); $req = HTTP::Request->new(GET => $url); $res = $ua->request($req); if ($res->is_success) { $html=$res->decoded_content; } else { print $res->status_line, "\n"; return undef; } print "perl=$]\n"; print "LWPsimple v=$LWP::Simple::VERSION\n"; print "HTTP::Response v=$HTTP::Response::VERSION\n"; print "url=$url\n"; my $message=(utf8::is_utf8($html)) ? "html is utf8 flaged" : "html + is not utf8 flaged"; print "$message\n"; } print join ',', @INC, "\n"; print "###with Simple,get#############\n"; test1(); print "###with ua,decoded content #############\n"; test2(); __DATA__ [tetsu]$/usr/home/tetsu/perl/videonews/Simple_Test% perl Simple_Test.p +l ./libwww-perl-5.818/lib,/usr/local/lib/perl5/5.12.2/BSDPAN,/usr/local/ +lib/perl5/site_perl/5.12.2/mach,/usr/local/lib/perl5/site_perl/5.12.2 +,/usr/local/lib/perl5/5.12.2/mach,/usr/local/lib/perl5/5.12.2,., ###with Simple,get############# perl=5.012002 LWPsimple v=5.810 HTTP::Response v=5.818 url=http://www.youtube.com/user/tbsnewsi/videos?sort=dd&view=0&page=1 html is not utf8 flaged ###with ua,decoded content ############# perl=5.012002 LWPsimple v=5.810 HTTP::Response v=5.818 url=http://www.youtube.com/user/tbsnewsi/videos?sort=dd&view=0&page=1 html is utf8 flaged

thanks a lot.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://986854]
help
Chatterbox?
[stevieb]: ...you're testing is a test system itself. 10+ documented issues fixed, another half dozen found while fixing the original ones. I'm getting there...
[stevieb]: Then throw in changes to Windows that throws everything you trusted right out the window.
[shmem]: that's why it is called "Windows"
[stevieb]: shmem thanks for the 'insight' :P
[shmem]: good thing that Sun already took "OpenWindows", otherwise I'd not stop to shudder imagining an "OpenWindows" from MS
[shmem]: more garbage in, more garbage out that would be
[stevieb]: I found that win10 broke a C# library I was using for one project while enhancing tests for a Perl dist, which breaks other Perl dists, and I'm about to throw my hands up on berrybrew. win2k12 broke one thing, win10 breaks something...
[stevieb]: ...unrelated which requires replacing a lot of code and a whole lib. I'm about to go nix only ffs
[shmem]: stevieb: what you're doing sounds afwully complex. Too much for me this evening to provide brighter insight ;-)

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (8)
As of 2017-03-28 22:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Should Pluto Get Its Planethood Back?



    Results (342 votes). Check out past polls.