I'm glad to see you're not all talk, that you're actually doing something.
Btw... How's that going? Going to have something ready to show us Real Soon Now?
Between the mind which plans and the hands which build, there must be a mediator... and this mediator must be the heart.
| [reply] |
It was too much hassle. We couldn't get access to a full database dump, but had to spider the site incrementally. The PerlMonks powers-that-be wouldn't grant access to node rep, which would have been highly useful in improving relevancy and user satisfaction; presumably if we had acquired access, that would have necessitated another round of spidering. Dmitri dropped out.
I'm no longer a consultant but have taken a full-time position, so I have less fungible time in general, and I'm disinclined to spend it fighting institutional resistance at PerlMonks to a custom-coded search. Nevertheless, my current work assignment is to add several features to KinoSearch and get a release out the door, so in some sense I'll continue to contribute indirectly.
Google is already an accepted alternative around here, and it provides a significant improvement over both simple Search and SuperSearch. Naturally I would prefer to see a custom-coded search powered by an open-source full-text indexer like KS, Xapian, or Swish, but life is full of compromises, and one of the gods chatterboxed me a while back that he didn't want the maintenance burden and preferred Google. Changing the Search box so that it uses Google rather than the current engine would be a decent compromise.
| [reply] |
Thanks for the update. I agree that it is discouraging to encounter "institutional resistance" when you're trying to do something good for the community. I just had a chat with one of the gods (Petruchio) and we came up with an approach which may be sufficient for you (or whomever) and acceptable to the powers, wrt privacy and load. Namely:
Periodically (perhaps once per day) a database dump will be cut which includes only publicly visible data, in SQL. This file will be available at a static address, and projects such as yours can download it and load it into their databases. This will not only be much easier than trying to spider the database through the XML-based 'API's, but also put much less strain on our server resources.
I for one have wanted such a thing for a long time, myself.
Petruchio suggests that the demand for it preceed the supply: "Get a project going, and people [meaning gods] will do what's needed to make it happen." (quoted without permission ;-)
The PerlMonks powers-that-be wouldn't grant access to node rep, which would have been highly useful in improving relevancy and user satisfaction
Yeah, that just isn't going to happen, ever. But it really should not be considered a project killer.
Between the mind which plans and the hands which build, there must be a mediator... and this mediator must be the heart.
| [reply] |
Maybe you didn't mean it, but your post comes across as condescending, especially given creamygoodness' reasonable explanation of why that project has not proceeded.
Elda Taluta; Sarks Sark; Ark Arks
| [reply] |
