Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re^4: web::scraper using an xpath

by Anonymous Monk
on Dec 10, 2010 at 11:17 UTC ( #876438=note: print w/replies, xml ) Need Help??


in reply to Re^3: web::scraper using an xpath
in thread web::scraper using an xpath

Are you sure? In that case I would guess that he must also remove the 'tr' from his further process statements.

Yup, I'm sure, and oddly no, there is no need to remove tr.

It would have been easier to see if ag4ve had posted runnable code , but I adapted eg/dave-trailer-HD.pl

I have no idea if this is intended behavior of this module

#!/usr/bin/perl -- use strict; use warnings; use Data::Dumper; use Web::Scraper; my $html_content = <<'__HTML__'; <html> <body> <div></div> <div id="wrapper"> <div></div> <div id="outer"> <div id="inner"> <div></div> <div id="center"> <div></div> <div id="main"> <div></div> <div> <table id="wrappedcontent"> <tbody class="shnitzel" bgcolor='red'> <tr> <td> <table class="someclass" style="width:508px;" id="Any_20"> <tbody> <tr> <td><strong>key1</strong></td> <td>val1</td> </tr> <tr> <td><strong>key2</strong></td> <td>val2</td> </tr> <tr> <td><strong>key3</strong></td> <td>val3</td> </tr> <tr> <td><strong>key4</strong></td> <td>val4</td> </tr> <tr> <td><strong>key5</strong></td> <td>val5</td> </tr> <tr> <td><strong>key6</strong></td> <td>val6</td> </tr> <tr> <td><strong>key7</strong></td> <td>val7</td> </tr> <tr> <td><strong>key8</strong></td> <td>val8</td> </tr> <tr> <td><strong>key9</strong></td> <td>val9</td> </tr> <tr> <td><strong>key10</strong></td> <td>val10</td> </tr> <tr> <td><strong>key11</strong></td> <td>val11</td> </tr> </tbody> </table> </td> </tr> </tbody> </table> </div> </div> </div> </div> </div> </div> </body> </html> __HTML__ my @pdata; push @pdata, scraper { process '//*/table[@class="someclass"]', 'table[]' => scraper { process '//tr/td[1]', 'name' => 'TEXT'; process '//tr/td[2]', 'attr' => 'TEXT'; }; }; push @pdata, scraper { process '//*/table[@class="someclass"]', 'table[]' => scraper { process '//tr/td[1]', 'name[]' => 'TEXT'; process '//tr/td[2]', 'attr[]' => 'TEXT'; }; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => { process('//tr/td[1]', 'name[]' => 'TEXT'), process('//tr/td[2]', 'attr[]' => 'TEXT'), }; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => scraper { process '//tr/td[1]', 'name' => 'TEXT'; process '//tr/td[2]', 'attr' => 'TEXT'; }; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => scraper { process '//tr/td[1]', 'name' => 'TEXT'; process '//tr/td[2]', 'attr' => 'TEXT'; }; result 'table'; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => scraper { process '//tr/td[1]', 'name[]' => 'TEXT'; process '//tr/td[2]', 'attr[]' => 'TEXT'; }; }; for my $pagedata ( @pdata ){ my $res = $pagedata->scrape( \$html_content ) or die "Can't define content to parser $!"; print Dumper( $res ), "\n\n"; } __END__ $VAR1 = { 'table' => [ { 'name' => 'key1', 'attr' => 'val1' } ] }; $VAR1 = { 'table' => [ { 'name' => [ 'key1', 'key2', 'key3', 'key4', 'key5', 'key6', 'key7', 'key8', 'key9', 'key10', 'key11' ], 'attr' => [ 'val1', 'val2', 'val3', 'val4', 'val5', 'val6', 'val7', 'val8', 'val9', 'val10', 'val11' ] } ] }; $VAR1 = { 'table' => [ undef, undef, undef, undef, undef, undef, undef, undef, undef, undef, undef ], 'name' => [ 'key1val1key2val2key3val3key4val4key5val5key6val +6key7val7key8val8key9val9key10val10key11val11', 'key1', 'key2', 'key3', 'key4', 'key5', 'key6', 'key7', 'key8', 'key9', 'key10', 'key11' ], 'attr' => [ 'val1', 'val2', 'val3', 'val4', 'val5', 'val6', 'val7', 'val8', 'val9', 'val10', 'val11' ] }; $VAR1 = { 'table' => [ { 'name' => 'key1', 'attr' => 'val1' }, { 'name' => 'key2', 'attr' => 'val2' }, { 'name' => 'key3', 'attr' => 'val3' }, { 'name' => 'key4', 'attr' => 'val4' }, { 'name' => 'key5', 'attr' => 'val5' }, { 'name' => 'key6', 'attr' => 'val6' }, { 'name' => 'key7', 'attr' => 'val7' }, { 'name' => 'key8', 'attr' => 'val8' }, { 'name' => 'key9', 'attr' => 'val9' }, { 'name' => 'key10', 'attr' => 'val10' }, { 'name' => 'key11', 'attr' => 'val11' } ] }; $VAR1 = [ { 'name' => 'key1', 'attr' => 'val1' }, { 'name' => 'key2', 'attr' => 'val2' }, { 'name' => 'key3', 'attr' => 'val3' }, { 'name' => 'key4', 'attr' => 'val4' }, { 'name' => 'key5', 'attr' => 'val5' }, { 'name' => 'key6', 'attr' => 'val6' }, { 'name' => 'key7', 'attr' => 'val7' }, { 'name' => 'key8', 'attr' => 'val8' }, { 'name' => 'key9', 'attr' => 'val9' }, { 'name' => 'key10', 'attr' => 'val10' }, { 'name' => 'key11', 'attr' => 'val11' } ]; $VAR1 = { 'table' => [ { 'name' => [ 'key1' ], 'attr' => [ 'val1' ] }, { 'name' => [ 'key2' ], 'attr' => [ 'val2' ] }, { 'name' => [ 'key3' ], 'attr' => [ 'val3' ] }, { 'name' => [ 'key4' ], 'attr' => [ 'val4' ] }, { 'name' => [ 'key5' ], 'attr' => [ 'val5' ] }, { 'name' => [ 'key6' ], 'attr' => [ 'val6' ] }, { 'name' => [ 'key7' ], 'attr' => [ 'val7' ] }, { 'name' => [ 'key8' ], 'attr' => [ 'val8' ] }, { 'name' => [ 'key9' ], 'attr' => [ 'val9' ] }, { 'name' => [ 'key10' ], 'attr' => [ 'val10' ] }, { 'name' => [ 'key11' ], 'attr' => [ 'val11' ] } ] };

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://876438]
help
Chatterbox?
[Lady_Aleena]: I wish I could think of a simple script to find out what modules my modules use (if any and including my other modules) and whicch of my modules use my modules (if any).
[LanX]: inspect %INC ?
[Lady_Aleena]: LanX, for every module I've written?
[huck]: Lady_Aleena see Re: pl script in webserver and/or Re: pl script in webserver for what they use
[Lady_Aleena]: Wouldn't I have to run the modules somehow to get their %INC?
[huck]: the first one works just fine from the command line but requires editing
[huck]: but neither will handle autoloads

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (9)
As of 2017-05-27 00:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?