Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re^2: Getting Text from Website

by bitingduck (Chaplain)
on Jun 22, 2012 at 06:25 UTC ( #977783=note: print w/replies, xml ) Need Help??

in reply to Re: Getting Text from Website
in thread Getting Text from Website

Since it looks like calc HW you're getting and not computer HW, here's a hint:
#!/usr/bin/perl # find calc homework use strict; use warnings; use HTML::TreeBuilder; my $date='4/27/012'; #set the date, you can do this dynamically my $url= ' +lic_html/1112hcal/1112hcal.htm'; # get the page and make a tree structure out of it my $tree= HTML::TreeBuilder->new_from_url($url); #break the table into rows my @elements = $tree->find_by_tag_name('tr'); #loop through the rows looking for the date #and use the as_trimmed_text to get rid of all the extra htmlness foreach (@elements){ if((my $hw=$_->as_trimmed_text())=~m%$date%){ print $hw."\n"; } }

It takes approach 1 that aaron_baugher describes, but mostly ignores the details of the page structure. We know it's a table and we want the rows. Knowing that the first column is just the day and date, I'm going to assume we want to keep them anyway. The find_by_tag_name just gets all the rows and all the stuff inside them. There's a bunch of <p> and <span> tags that really aren't interesting, so I take the lazy approach and use as_trimmed_text to throw those away and just keep the contents of the two cells all together. It's also useful to know that HTML::TreeBuilder gets a bunch of methods from HTML::Element.

Update: tweaked the code formatting to keep the comments from wrapping

...And to note that some of your assignments have links in them-- you can use HTML::Element to dig those out before you apply as_trimmed_text, or dig them out an of other various possible ways.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://977783]
[marioroy]: Hello all. After several attempts, am able to move forward and will soon release a Parallel:: ForkManager compatible MCE::Hobo:: Manager, also MCE::Hobo::Simple, and refactored MCE::Hobo engine supporting multiple instances. Wait works beautifully.
[marioroy]: MCE::Shared has been refined that it allows sharing Tie::File and hash objects containing {fh} key.
[marioroy]: It's taken so long to take Hobo to a new level. Threads-like and Parallel:: ForkManager management capaiblities.
[marioroy]: This will complete the 4 years in the making and likely time to move on to something els.
[marioroy]: s/els/else.

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (7)
As of 2017-05-26 08:26 GMT
Find Nodes?
    Voting Booth?