http://www.perlmonks.org?node_id=390108

tradez has asked for the wisdom of the Perl Monks concerning the following question:

Hello fellow monks! Been a while but I am back in the monk game with a new question for my favorite monastery. I am coding a script to take HUGE ammounts of data and place it in a postgres database, more specifically a 4 column table within it, in a simple structure. I have been toying with the "copy" functionality from SQL and I have found it to work incredibly well from the command line, and extremely fast. However, when attempting to do this from a perl script via DBI, i get some strange errors. Please consider the following
my $copySQL = "copy snapshot_$formName (name, value, collect_date, omp_id) from ?" my $sth = $dbh->prepare($copySQL); $sth->execute($db_input);
FYI, $db_input is a variable created in the foreach loop that creates the input file, it is explicit in its file location, and again this exact SQL format works perfectly when logged into psql. The errors I get look like this:
DBD::Pg::st execute failed: ERROR: missing data for column "collect_d +ate" at ./getSnapshot.pl line 180, <FORM_OUT> line 103. Couldn't execute copy snapshot_cco (name, value, collect_date, omp_id) + FROM '/tmp/snapshot_mil-newberlin1_cco.input' statement: 7 at ./getS +napshot.pl line 180, <FORM_OUT> line 103.
The file I am inputting looks like this:
csno,100,20040908,32 alarm1.equip[1] y 20040908 32 alarm1.equip[10] y 20040908 32 alarm1.equip[11] y 20040908 32
The truly strange thing about this is that it fails on line 102 or 103 everytime, the first 100 or so make it through just fine. So whattyah say brethren, can yah help me out?


Tradez
"Never underestimate the predictability of stupidity"
- Bullet Tooth Tony, Snatch (2001)