Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Re^2: Parsing UTF-8 HTML w/ HTML::Parser

by Purdy (Hermit)
on Jun 24, 2010 at 18:49 UTC ( #846374=note: print w/replies, xml ) Need Help??

in reply to Re: Parsing UTF-8 HTML w/ HTML::Parser
in thread Parsing UTF-8 HTML w/ HTML::Parser

If it helps, as soon as I do the $mech->get on the production box, I get a warning message. If I run it from the debugger, I see this:

Parsing of undecoded UTF-8 will give garbage when decoding entities at /usr/local/share/perl/5.8.8/HTML/ line 93.
at /usr/local/share/perl/5.8.8/HTML/ line 93
HTML::TreeBuilder::new_from_content('HTML::TreeBuilder', '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN...') called at /tmp/ line 133
  • Comment on Re^2: Parsing UTF-8 HTML w/ HTML::Parser

Replies are listed 'Best First'.
Re^3: Parsing UTF-8 HTML w/ HTML::Parser
by ikegami (Pope) on Jun 24, 2010 at 18:57 UTC
    Sounds like you didn't decode the HTML before passing it to HTML::TreeBuilder, so decode it. If this was LWP (which WWW::Mechanize derives), I'd say use ->decoded_content instead of ->content.

      That helps, but I still don't understand what's going on. At this point, I have:

      my $root = HTML::TreeBuilder->new_from_content( $mech->response->d +ecoded_content );

      And that's working on the production box ... now I just want to back away slowly and hope it doesn't go all screwy on me again. ;)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://846374]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (4)
As of 2020-10-22 01:45 GMT
Find Nodes?
    Voting Booth?
    My favourite web site is:

    Results (225 votes). Check out past polls.