You might have luck with
wget, which is a super cool program for fetching entire web pages. It does recursive gets (you said you needed this), and allows you to specify how many levels deep you are willing to go. You could call this from a script, then use perl to parse the output.
BlueLines
Disclaimer:
This post may contain inaccurate information, be habit forming, cause atomic warfare between peaceful countries, speed up male pattern baldness, interfere with your cable reception, exile you from certain third world countries, ruin your marriage, and generally spoil your day. No batteries included, no strings attached, your mileage may vary.
