Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Cleaning up text for indexing in DB

by t'mo (Pilgrim)
on Jul 16, 2003 at 15:23 UTC ( #274864=note: print w/replies, xml ) Need Help??


in reply to Cleaning up text for indexing in DB

As other monks suggested in another thread, why not get the text via the program 'lynx', letting it get rid of any HTML-junk for you?

$text = `lynx -dump $url`;

...every application I have ever worked on is a glorified munger...

Replies are listed 'Best First'.
Re: Re: Cleaning up text for indexing in DB
by TVSET (Chaplain) on Jul 16, 2003 at 16:28 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://274864]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (6)
As of 2016-09-30 00:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Extraterrestrials haven't visited the Earth yet because:







    Results (562 votes). Check out past polls.