Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^2: LWP::Simple is it get decoded or not ?

by remiah (Hermit)
on Aug 11, 2012 at 04:07 UTC ( #986854=note: print w/ replies, xml ) Need Help??


in reply to Re: LWP::Simple is it get decoded or not ?
in thread LWP::Simple is it get decoded or not ?

It was version difference of LWP::Simple to get decoded character or not decoded bytes.

5.810 of LWP::Simple doesn't use decoded_content() but it uses content() method. On the other hand 5.835 uses decoded_content() method, so it returns decoded character.

I decided to use useragent and decoded_content explicitly, to work with each enviornment. Below is test script with LWP::Simple 5.810

#!/usr/bin/perl use strict; use warnings; use lib './libwww-perl-5.818/lib'; use LWP::Simple; use HTTP::Response; use LWP::UserAgent; my $url='http://www.youtube.com/user/tbsnewsi/videos?sort=dd&view=0&pa +ge=1'; sub test1 { ### with get my($html); $html=get($url); print "perl=$]\n"; print "LWPsimple v=$LWP::Simple::VERSION\n"; print "HTTP::Response v=$HTTP::Response::VERSION\n"; print "url=$url\n"; my $message=(utf8::is_utf8($html)) ? "html is utf8 flaged" : "html + is not utf8 flaged"; print "$message\n"; } sub test2{ ### with useragent and requst my ($ua, $req, $res, $html); $ua = LWP::UserAgent->new; $ua->agent("test user agent"); $req = HTTP::Request->new(GET => $url); $res = $ua->request($req); if ($res->is_success) { $html=$res->decoded_content; } else { print $res->status_line, "\n"; return undef; } print "perl=$]\n"; print "LWPsimple v=$LWP::Simple::VERSION\n"; print "HTTP::Response v=$HTTP::Response::VERSION\n"; print "url=$url\n"; my $message=(utf8::is_utf8($html)) ? "html is utf8 flaged" : "html + is not utf8 flaged"; print "$message\n"; } print join ',', @INC, "\n"; print "###with Simple,get#############\n"; test1(); print "###with ua,decoded content #############\n"; test2(); __DATA__ [tetsu]$/usr/home/tetsu/perl/videonews/Simple_Test% perl Simple_Test.p +l ./libwww-perl-5.818/lib,/usr/local/lib/perl5/5.12.2/BSDPAN,/usr/local/ +lib/perl5/site_perl/5.12.2/mach,/usr/local/lib/perl5/site_perl/5.12.2 +,/usr/local/lib/perl5/5.12.2/mach,/usr/local/lib/perl5/5.12.2,., ###with Simple,get############# perl=5.012002 LWPsimple v=5.810 HTTP::Response v=5.818 url=http://www.youtube.com/user/tbsnewsi/videos?sort=dd&view=0&page=1 html is not utf8 flaged ###with ua,decoded content ############# perl=5.012002 LWPsimple v=5.810 HTTP::Response v=5.818 url=http://www.youtube.com/user/tbsnewsi/videos?sort=dd&view=0&page=1 html is utf8 flaged

thanks a lot.


Comment on Re^2: LWP::Simple is it get decoded or not ?
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://986854]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (4)
As of 2015-07-04 23:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (60 votes), past polls