Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Re: Perl Search Engine

by leocharre (Priest)
on Jul 22, 2010 at 07:30 UTC ( #850790=note: print w/replies, xml ) Need Help??

in reply to Perl Search Engine

What do you mean by follow php links?

The question you posted is vague. Maybe you could refine it- Are you thinking of a cgi script that you put in cgi-bin through which you can search the content of what visitors to the website can see?

Are you familiar with unix? Do you know how you can do a find command and search content of files? For example

find ~/public_html -type f | xargs grep carl

Well, that is something of a sloppy search. It's a real time event, highly cpu needy- and would probably cripple your system if you offered multiple people to use that as a 'search engine' backend.

What (likely) you have in mind is something more advanced. As moritz pointed out- the tasks are divided. One part of the process is to have something that finds or lists *what* it will be you will be searching through- for example, what images, pdf files, html files, whatever- are in your website that you want users to be able to search for.

Another step, afterwards, is another program/system/part of the 'search engine'- that actually looks at these resources, these *what* which you let people search for- and stores information about those things in perhaps a db file or a mysql database. It may store text, filenames, whatever, file size, whatever.

Yet another step is the user interface, what presents the user with a form perhaps, in which they type things in like 'look for these words'. That thing, looks at what information we stored about our picked out resources, and finds matches between what the user requested, what we know of the stuff, and then where the stuff is- that they may go and see it/download it.

Even crazier, and much more fun, is if you're talking about remotely indexing/cataloguing data seen somewhere, say, in another website to which you only have http access. That's a little(lot) bit like search engines such as google do- on the backend.

Anyway- I don't know if this helps you rewrite your question. Consider trying, you'll get good answers and valuable suggestions here.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://850790]
[choroba]: The theme is "High-end Perl", so both my talks are antitheses: "Low-end Perl" and "How to almost win a T-shirt in a programming contest"
[RonW]: I wonder if that's a CMD.exe limitation or a Windows limitation
[james28909]: what i am trying to accomplish is piping the output of ffmpeg to yet another encryption routine using libsodium. and hopefully once that is done i will be able to forward the packets to discord servers
[RonW]: choroba: Please define "high end" and "low end"
[LanX]: choroba: subversive (as usual ;)
[james28909]: i dunno, if i didnt have so little experience in linux i would swap. but it would be to much of a learning curve for me right now.
[choroba]: Low-end is defined as the Perl that generates millions of income
[choroba]: sorry, that's high-end, of coursse
[choroba]: low end, in my talk, will be code that "we don't touch because it works" and noone knows why
[choroba]: I want to present the most bizzare bugs and misfeatures I met when working for a large financial institution

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (17)
As of 2017-05-22 21:31 GMT
Find Nodes?
    Voting Booth?