If you loaded it during initialisation of mod_perl, you would just have it in memory at all times instead. Which probably isn't quite the best answer, either.
Any thoughts on putting it in a database instead? Or, if your software can't do that, you could write a wrapper that creates the model, loads it into memory, dumps it to a database, and then your CGI code could use that database instead?
| [reply] |
The Storable, its create automaticaly by module AI::Categorizer::Learner::NaiveBayes
Thanks for the help :)
| [reply] |
One idea might be to write a seperate daemon that your CGI script connects to via sockets. The daemon would then execute queries on the model on your script's behalf and return the results. While this introduces the overhead of communication with the daemon, it saves constantly re-loading the large file (keeping it in memory only once no matter how many simultaneous queries there are from the CGI script) so it should be a fairly substantial overall win. It has the added advantage that should you need to scale this application up further in the future it should be trivial to move the daemon to a seperate machine. | [reply] |
A separate server process which the mod_perl Apache processes talk to is probably the "ultimate" answer. When some nice person names the perfect CPAN module to implement that kind of server, I'll certainly bookmark it myself (and give you a grateful thought).
Otherwise, I have used the PerlModule command in a <IfModule mod_perl.c> stanza in httpd.conf.
I didn't need to do more. The Perl module is read in at Apache start and the memory is shared with sub Apache processes.
| [reply]
[d/l] |
| [reply] |
A database server really isn't the right solution to his problem. His Storable file is (as he said in a reply to your earlier post) the saved state from a module that almost certainly won't map well onto a relational database, and even if he could find some workable schema it would entail a re-write of the CPAN module he is using.
Of course, he could save it in a BLOB field in the database, but this leaves him much worse off than he was before: now he is not only loading the file into memory every time again, but he also has to transfer it out of the database which is going to have a significant amount of overhead.
Databases are a good solution to a wide variety of programming problems, but this unfortunately isn't one of them. (Unless I'm missing something -- if I am, would you mind explaining in more detail how your proposed solution would work?)
Edited to add: I might be mis-interpreting "database" as "relational database", but even using some non-relational DB like BerkeleyDB isn't going to leave him any better off than he is now, at least as far as I can see.
Edit again: Well, you could write it as a stored procedure in PL/Perl in Postgres and it would probably work, but that doesn't make it a good idea.
| [reply] |
You could migrate the Storable structure to a DBM::Deep data file, then you'd be able access parts of it without loading it all into memory. | [reply] |
