<?xml version="1.0" encoding="windows-1252"?>
<node id="946548" title="Any spider framework?" created="2012-01-06 03:03:50" updated="2012-01-06 03:03:50">
<type id="115">
perlquestion</type>
<author id="961">
Anonymous Monk</author>
<data>
<field name="doctext">
HI,all&lt;BR&gt;
I want to get all urls like 'http://site/fixed_string/random_string.html' from one site. Where should I start?&lt;BR&gt;
&lt;BR&gt;
Is there any spider framework, support proxy,cache, and so on, suit my need? &lt;BR&gt;
&lt;BR&gt;
Or, If I start from scratch, using LWP, is there some guide for write a spider?
&lt;BR&gt;&lt;BR&gt;
thanks.</field>
<field name="reputation">
3</field>
</data>
</node>
