Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

Re: Database input speed question

by Corion (Pope)
on Jul 31, 2003 at 15:24 UTC ( #279611=note: print w/replies, xml ) Need Help??

in reply to Database input speed question

The simple answer is, don't use DBI. Every decent database has a specialized "loader" program, that bypasses SQL completely and pumps in the raw data for each table from a data file. The various loader programs vary in syntax and features, but for Oracle, have a look at sqlloader, Sybase and MSSQL have a similar program.

The problem is now that you can't have computed fields in your database, as the loader will most likely not have the capabilities, but you can always load the database first and then update the fields that need (Perl) computation.

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web

Replies are listed 'Best First'.
Re: Re: Database input speed question
by CountZero (Bishop) on Jul 31, 2003 at 19:29 UTC

    The loader may not have the intelligence to do computed fields, but Perl does. As most likely you will have to write a Perl script anyhow to reformat your data into a format acceptable to the bulkloader, you can take that opportunity to fill the computed fields.


    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://279611]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2018-07-21 14:05 GMT
Find Nodes?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

    Results (449 votes). Check out past polls.