Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^4: web::scraper using an xpath

by Anonymous Monk
on Dec 10, 2010 at 11:17 UTC ( #876438=note: print w/replies, xml ) Need Help??


in reply to Re^3: web::scraper using an xpath
in thread web::scraper using an xpath

Are you sure? In that case I would guess that he must also remove the 'tr' from his further process statements.

Yup, I'm sure, and oddly no, there is no need to remove tr.

It would have been easier to see if ag4ve had posted runnable code , but I adapted eg/dave-trailer-HD.pl

I have no idea if this is intended behavior of this module

#!/usr/bin/perl -- use strict; use warnings; use Data::Dumper; use Web::Scraper; my $html_content = <<'__HTML__'; <html> <body> <div></div> <div id="wrapper"> <div></div> <div id="outer"> <div id="inner"> <div></div> <div id="center"> <div></div> <div id="main"> <div></div> <div> <table id="wrappedcontent"> <tbody class="shnitzel" bgcolor='red'> <tr> <td> <table class="someclass" style="width:508px;" id="Any_20"> <tbody> <tr> <td><strong>key1</strong></td> <td>val1</td> </tr> <tr> <td><strong>key2</strong></td> <td>val2</td> </tr> <tr> <td><strong>key3</strong></td> <td>val3</td> </tr> <tr> <td><strong>key4</strong></td> <td>val4</td> </tr> <tr> <td><strong>key5</strong></td> <td>val5</td> </tr> <tr> <td><strong>key6</strong></td> <td>val6</td> </tr> <tr> <td><strong>key7</strong></td> <td>val7</td> </tr> <tr> <td><strong>key8</strong></td> <td>val8</td> </tr> <tr> <td><strong>key9</strong></td> <td>val9</td> </tr> <tr> <td><strong>key10</strong></td> <td>val10</td> </tr> <tr> <td><strong>key11</strong></td> <td>val11</td> </tr> </tbody> </table> </td> </tr> </tbody> </table> </div> </div> </div> </div> </div> </div> </body> </html> __HTML__ my @pdata; push @pdata, scraper { process '//*/table[@class="someclass"]', 'table[]' => scraper { process '//tr/td[1]', 'name' => 'TEXT'; process '//tr/td[2]', 'attr' => 'TEXT'; }; }; push @pdata, scraper { process '//*/table[@class="someclass"]', 'table[]' => scraper { process '//tr/td[1]', 'name[]' => 'TEXT'; process '//tr/td[2]', 'attr[]' => 'TEXT'; }; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => { process('//tr/td[1]', 'name[]' => 'TEXT'), process('//tr/td[2]', 'attr[]' => 'TEXT'), }; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => scraper { process '//tr/td[1]', 'name' => 'TEXT'; process '//tr/td[2]', 'attr' => 'TEXT'; }; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => scraper { process '//tr/td[1]', 'name' => 'TEXT'; process '//tr/td[2]', 'attr' => 'TEXT'; }; result 'table'; }; push @pdata, scraper { process '//*/table[@class="someclass"]//tr', 'table[]' => scraper { process '//tr/td[1]', 'name[]' => 'TEXT'; process '//tr/td[2]', 'attr[]' => 'TEXT'; }; }; for my $pagedata ( @pdata ){ my $res = $pagedata->scrape( \$html_content ) or die "Can't define content to parser $!"; print Dumper( $res ), "\n\n"; } __END__ $VAR1 = { 'table' => [ { 'name' => 'key1', 'attr' => 'val1' } ] }; $VAR1 = { 'table' => [ { 'name' => [ 'key1', 'key2', 'key3', 'key4', 'key5', 'key6', 'key7', 'key8', 'key9', 'key10', 'key11' ], 'attr' => [ 'val1', 'val2', 'val3', 'val4', 'val5', 'val6', 'val7', 'val8', 'val9', 'val10', 'val11' ] } ] }; $VAR1 = { 'table' => [ undef, undef, undef, undef, undef, undef, undef, undef, undef, undef, undef ], 'name' => [ 'key1val1key2val2key3val3key4val4key5val5key6val +6key7val7key8val8key9val9key10val10key11val11', 'key1', 'key2', 'key3', 'key4', 'key5', 'key6', 'key7', 'key8', 'key9', 'key10', 'key11' ], 'attr' => [ 'val1', 'val2', 'val3', 'val4', 'val5', 'val6', 'val7', 'val8', 'val9', 'val10', 'val11' ] }; $VAR1 = { 'table' => [ { 'name' => 'key1', 'attr' => 'val1' }, { 'name' => 'key2', 'attr' => 'val2' }, { 'name' => 'key3', 'attr' => 'val3' }, { 'name' => 'key4', 'attr' => 'val4' }, { 'name' => 'key5', 'attr' => 'val5' }, { 'name' => 'key6', 'attr' => 'val6' }, { 'name' => 'key7', 'attr' => 'val7' }, { 'name' => 'key8', 'attr' => 'val8' }, { 'name' => 'key9', 'attr' => 'val9' }, { 'name' => 'key10', 'attr' => 'val10' }, { 'name' => 'key11', 'attr' => 'val11' } ] }; $VAR1 = [ { 'name' => 'key1', 'attr' => 'val1' }, { 'name' => 'key2', 'attr' => 'val2' }, { 'name' => 'key3', 'attr' => 'val3' }, { 'name' => 'key4', 'attr' => 'val4' }, { 'name' => 'key5', 'attr' => 'val5' }, { 'name' => 'key6', 'attr' => 'val6' }, { 'name' => 'key7', 'attr' => 'val7' }, { 'name' => 'key8', 'attr' => 'val8' }, { 'name' => 'key9', 'attr' => 'val9' }, { 'name' => 'key10', 'attr' => 'val10' }, { 'name' => 'key11', 'attr' => 'val11' } ]; $VAR1 = { 'table' => [ { 'name' => [ 'key1' ], 'attr' => [ 'val1' ] }, { 'name' => [ 'key2' ], 'attr' => [ 'val2' ] }, { 'name' => [ 'key3' ], 'attr' => [ 'val3' ] }, { 'name' => [ 'key4' ], 'attr' => [ 'val4' ] }, { 'name' => [ 'key5' ], 'attr' => [ 'val5' ] }, { 'name' => [ 'key6' ], 'attr' => [ 'val6' ] }, { 'name' => [ 'key7' ], 'attr' => [ 'val7' ] }, { 'name' => [ 'key8' ], 'attr' => [ 'val8' ] }, { 'name' => [ 'key9' ], 'attr' => [ 'val9' ] }, { 'name' => [ 'key10' ], 'attr' => [ 'val10' ] }, { 'name' => [ 'key11' ], 'attr' => [ 'val11' ] } ] };

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://876438]
help
Chatterbox?
[marto]: good morning all
[Corion]: Hi marto!
[Corion]: I hope all is well marto! ;) My godson had a surprise visit to the hospital yesterday because he fell and had cut his skin besides his eye, but everything was glued together again and all is well

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (8)
As of 2016-12-08 09:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:













    Results (137 votes). Check out past polls.