Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re^2: HTML::TreeBuilder::XPath not loading the complete $page

by Lord Gartlar (Initiate)
on Mar 19, 2013 at 20:11 UTC ( #1024379=note: print w/ replies, xml ) Need Help??


in reply to Re: HTML::TreeBuilder::XPath not loading the complete $page
in thread HTML::TreeBuilder::XPath not loading the complete $page

I looked line by line at the output html file containing $response->content

It seems that sometimes there's an internal error printed inside another HTML tag...

In other words, the $response->content goes as follows:

<html> <head> ... ... </head> <body> ... ... ... <DIV></DIV> # Div number 12 <HTML> <HEAD> </HEAD> <BODY> <p>You have an error blah blah blah</p> </BODY> </HTML> <DIV></DIV> # Div number 13 and so on until number 19 ... ... </body> </html>

The problem is that it's not inside any iframe or something like that, so the onl way out that comes to my mind is use a while to look for such issues and take them out, then regenerate the output and go on...

Does TreeBuilder have an option to avoid this things?? Google is not helping me


Comment on Re^2: HTML::TreeBuilder::XPath not loading the complete $page
Download Code
Re^3: HTML::TreeBuilder::XPath not loading the complete $page
by tangent (Curate) on Mar 19, 2013 at 21:00 UTC
    Are you sure there isn't something else going on? When I use your sample content it still works:
    my $content = q| <html> <head> </head> <body> <DIV>Div number 12</DIV> <HTML> <HEAD> </HEAD> <BODY> <p>You have an error blah blah blah</p> </BODY> </HTML> <DIV>Div number 13</DIV> </body> </html> |; my $tree = HTML::TreeBuilder::XPath->new; $tree->parse_content( $content ); my @divs = $tree->findnodes( '/html/body/div' ); for my $div (@divs) { print $div->as_HTML . "\n"; }
    Output:
    <div>Div number 12</div> <div>Div number 13</div>

      There are some errors displayed there inside the new HTML...

      As mentioned, I tried deleting them with a mere while and it finally worked...

Re^3: HTML::TreeBuilder::XPath not loading the complete $page
by Anonymous Monk on Mar 20, 2013 at 06:56 UTC

    Why don't you post html that proves your point? TreeBuilder has options such as ignore_unknown and implicit_tags ...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1024379]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (5)
As of 2014-12-29 05:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (184 votes), past polls