http://www.perlmonks.org?node_id=846374


in reply to Re: Parsing UTF-8 HTML w/ HTML::Parser
in thread Parsing UTF-8 HTML w/ HTML::Parser

If it helps, as soon as I do the $mech->get on the production box, I get a warning message. If I run it from the debugger, I see this:

Parsing of undecoded UTF-8 will give garbage when decoding entities at /usr/local/share/perl/5.8.8/HTML/TreeBuilder.pm line 93.
at /usr/local/share/perl/5.8.8/HTML/TreeBuilder.pm line 93
HTML::TreeBuilder::new_from_content('HTML::TreeBuilder', '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN...') called at /tmp/update_wire.pl line 133
  • Comment on Re^2: Parsing UTF-8 HTML w/ HTML::Parser

Replies are listed 'Best First'.
Re^3: Parsing UTF-8 HTML w/ HTML::Parser
by ikegami (Pope) on Jun 24, 2010 at 18:57 UTC
    Sounds like you didn't decode the HTML before passing it to HTML::TreeBuilder, so decode it. If this was LWP (which WWW::Mechanize derives), I'd say use ->decoded_content instead of ->content.

      That helps, but I still don't understand what's going on. At this point, I have:

      my $root = HTML::TreeBuilder->new_from_content( $mech->response->d +ecoded_content );

      And that's working on the production box ... now I just want to back away slowly and hope it doesn't go all screwy on me again. ;)