Re: Parsing HTML files to recover data...

[GrandFather]

For this sort of 'Parse and extract' from HTML problem I reach for HTML::TreeBuilder. A first cut solution might look like:

use strict;
use warnings;
use HTML::TreeBuilder;

my $str = do {local $/; <DATA>};
my $tree = HTML::TreeBuilder->new;

$tree->parse ($str);

my @jobs;

for my $para ($tree->find ('p')) {
    my @class = $para->look_down ('class', 'jobname');
    my @names = $para->look_down ('name', 'em');
    my @offices = $para->look_down ('name', 'offices');
    next unless @class && @names && @offices;
    
    my $job = $class[0]->as_text ();;
    
    $job .= ': ' . join '; ', map {$_->as_text ()} @names;
    $job .= ' (' . join (', ', map {$_->as_text ()} @offices) . ')';
    push @jobs, $job;
}

print join "\n", @jobs;

__DATA__
<p><b><span class="jobname">Sandbagger, Level 2</span>  
<span class="jobserial">(19000)</span><br />
Current members:<br />
<span name='em'>Fred</span><span name='em'>Wilma</span><br />
<span name='offices'>Erewon</span>
</p>

<p><b><span class="jobname">Accounting Assistant, Level 2</span>  
<span class="jobserial">(19203)</span><br />
Current members:<br />
<span name='em'>Plow, Elliot</span><span name='em'>Wang, Susan</span><
+br />
<span name='offices'>Huston</span>
</p>
<blockquote>
Job descriptions here.
This block quoted text contains a job description and it what I am rea
+lly looking to recover.
</blockquote>
<blockquote><a href="#top">Go to the top of this page</a>.</blockquote
+>
<blockquote><a href='companyHR.html'>Check for open positions now!</a>
+</blockquote>

Prints:

Sandbagger, Level 2: Fred; Wilma (Erewon)
Accounting Assistant, Level 2: Plow, Elliot; Wang, Susan (Huston)

I imagine that there might be lists of people fillin a particular role by office. This code will munge all of those entries together and list the offices, thus losing the association of people with offices. That can be fixed by using a $para->look_down ('name', qr/em|offices/) then iterating over the list spitting out employee names and office names as appropriate.

[UrbanHick]

Thank you very much for your detailed response. The example code is most instructive. I will likely use this code as a frame work to fashion several programs to each parse a specific document for the data that the Powers That Be want extracted from each old webpage.

-UH