Re^2: Getting Text from Website

[bitingduck]

in reply to Re: Getting Text from Website
in thread Getting Text from Website

Since it looks like calc HW you're getting and not computer HW, here's a hint:

#!/usr/bin/perl
# find calc homework

use strict;
use warnings;


use HTML::TreeBuilder;

my $date='4/27/012';        #set the date, you can do this dynamically

my $url= 'http://staweb.sta.cathedral.org/departments/math/mhansen/pub
+lic_html/1112hcal/1112hcal.htm';

# get the page and make a tree structure out of it
my $tree= HTML::TreeBuilder->new_from_url($url);    

#break the table into rows
my @elements = $tree->find_by_tag_name('tr');         

#loop through the rows looking for the date
#and use the as_trimmed_text to get rid of all the extra htmlness
foreach (@elements){                        
    if((my $hw=$_->as_trimmed_text())=~m%$date%){    
        print $hw."\n";
    }

}

It takes approach 1 that aaron_baugher describes, but mostly ignores the details of the page structure. We know it's a table and we want the rows. Knowing that the first column is just the day and date, I'm going to assume we want to keep them anyway. The find_by_tag_name just gets all the rows and all the stuff inside them. There's a bunch of <p> and <span> tags that really aren't interesting, so I take the lazy approach and use as_trimmed_text to throw those away and just keep the contents of the two cells all together. It's also useful to know that HTML::TreeBuilder gets a bunch of methods from HTML::Element.

Update: tweaked the code formatting to keep the comments from wrapping

...And to note that some of your assignments have links in them-- you can use HTML::Element to dig those out before you apply as_trimmed_text, or dig them out an of other various possible ways.