in reply to Perl :: Mechanize - running a single while loop
combine with Re: Help With Online Table Scraper.GetSome( starting url ); sub GetSome { my $mech ... my @pages = ... while(@pages) { my $page = shift @pages; $mech->get( $page ); push @pages, GetMorePages( $mech ); SomethingImportant( $mech ); SomethingXPATH( $mech ); } }
All the interesting stuff happens in GetMorePages, SomethingImportant, and SomethingXPATH
Can you give me a hint for the beginning - the processing of the entry pages - doing this in Perl:: Mechanize
Before writing any code, make a nice flowchart, maybe with Text::Flowchart, of how you think your program should work.
Once you have an overall idea of how your program should work, once you have the skeleton (like I outlined above), then think about writing some code (like the skeleton).
Next part is to write a small program in the spirit of Re^5: WWW::Mechanize clicking checkboxes or Re^2: Help With Online Table Scraper (or Re^2: Printing just the file name for all the cases), to solve one part of your diagram, like GetMorePages.
Then incorporate GetMorePages into your main program, and then repeat these steps for some other part, like SomethingXPATH
And repeat until you're finished
|
---|
Replies are listed 'Best First'. | |
---|---|
many thanks!! Great points
by Perlbeginner1 (Scribe) on May 19, 2011 at 21:59 UTC |