http://www.perlmonks.org?node_id=194007


in reply to Big database queries

i'd suggest using a stored procedure. for one, the SQL in the stored procedure is precompiled, thus avoiding the need for the database to parse the query and create an execution plan every single time; secondly, since all the processing is done on the database, you could return back only the filtered results, thus cutting down on network traffic between the database server and the client machine....
i'd also suggest using recordset paging techniques so that u only need to show 20-25 records to the user at a time , and provide "PREVIOUS" and "NEXT" buttons. this way, u dont have to fetch millions of records , just 25 records at a time....that should help.