looking for ideas for managing snippets collection

[zentara]

Hi, my snippets collection is getting kind of large, and my old method of letting Perl grep through all the files is getting slow and "hard drive intensive". So I have a preliminary design using swish++ and then dumping the files returned from swish into a Tk listbox, and then you can browse the file with the search word highlighted.

So I figure this has been done before, and was wondering if there are better ways. (Before I get too deep into it :-)

The only drawbacks I see is swish++ likes to ignore alot of short common words, which are Perl keywords...like for, if ,etc; and it is limited to single word searches.

So has anyone else found a method which can limit the search to 1 file, like swish++ ?

My first inclination was just to "cat" all the files together in a big text file, and just search it. But then what would be the best way to compress and search it, retaining the original filenames?

My intuition tells me I want to convert the text to some compressed binary reperesentation, to make the searching faster, and the file smaller.

So I would appreciate any ideas and pointers which anyone may have.(before I go sailing off in the wrong direction :-) ) TIA

[cyocum]

I am not familar with swish++ but there is a perl solution that might take some coding but will give you good results in the end. It is called Plucene and is a Perl port of the Java Lucene search program. I hope this helps.

[Anomynous Monk]

"Letting Perl grep through the files"? How about letting grep grep through the files? It's hard to imagine you could have that much in the way of snippets.

[zentara]

It's hard to imagine you could have that much in the way of snippets.

Well over the last few years, I've accumulated almost 256 Megs of snippets, docs, and html related to Perl. I used to be a "compulsive garage shopping junkie", but now I invest in collecting Perl. :-)

[artist]

As a learning approach, you can build the index, as you go and use the scripts. While working with various files, I use 'etags' from emacs.