Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

DBD::ORacle: Not able to insert into Database

by slayedbylucifer (Scribe)
on Jul 25, 2012 at 11:22 UTC ( #983630=perlquestion: print w/ replies, xml ) Need Help??
slayedbylucifer has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I ma trying to uplaod a CSV into Oracle Databse, the problem is, my script gets stuck and I have to kill it with CTRL+C.. Here is hte csv:

# cat test.csv vm,farm,dmz name1,farm1,z1 name2,farm2,z2 name3,farm3,z3

and here is the code:

#!/usr/bin/perl -w use strict; use DBI; open (my $FH, '<', "test.csv") or die "Cannot open test.csv: $! \n"; my $dbh = DBI->connect ("dbi:Oracle:host=<hostname>;sid=<SID>", '<user +name>', ',password.', { RaiseError => 1, AutoCommit => 1, TraceLevel +=> 4}) or die "Canot create Databse Handle: $DBI::err +str() \n"; my $table = "CLUSTER_MAPPING"; my @array = <$FH> ; close $FH; shift @array; ### this is to remove the column headers my $sth = $dbh->prepare ("INSERT INTO $table (name, farm, dmz) VALUES +(?,?,?)"); while (<@array>) { my @row = split /,/; my $col1_value = $row[0]; my $var1 = $row[1]; $var1 =~ s/"//g; $var1 =~ s/^ //g; my $col2_value = $var1; my $col3_value = $row[2]; $sth->execute ($col1_value, $col2_value, $col3_value); } $sth->finish(); $dbh->disconnect();

here is the debug output from the script ( i have used level 2 as seen in AutoCommit => 2):

DBI::db=HASH(0x228e560) trace level set to 0x0/2 (DBI @ 0x0/0) in +DBI 1.622-ithread (pid 15876) -> STORE for DBD::Oracle::db (DBI::db=HASH(0x228e560)~INNER 'Usern +ame' '<username>') thr#21f5010 <- STORE= 1 at DBI.pm line 713 -> connected in DBD::_::db for DBD::Oracle::db (DBI::db=HASH(0x228 +e5f0)~0x228e560 'dbi:Oracle:host=<hostname>;sid=<SID>' '<username>' * +*** HASH(0x2401778)) thr#21f5010 <- connected= undef at DBI.pm line 720 -> STORE for DBD::Oracle::db (DBI::db=HASH(0x228e560)~INNER 'dbi_c +onnect_closure' CODE(0x228de28)) thr#21f5010 <- STORE= 1 at DBI.pm line 729 -> prepare for DBD::Oracle::db (DBI::db=HASH(0x228e5f0)~0x228e560 +'INSERT INTO CLUSTER_MAPPING (name, farm, dmz) VALUES (?,?,?)') thr#2 +1f5010 dbd_preparse scanned 3 distinct placeholders <- prepare= DBI::st=HASH(0x228e938) at upload.pl line 17 -> execute for DBD::Oracle::st (DBI::st=HASH(0x228e938)~0x228e980 +'name1' 'farm1' 'z1') thr#21f5010 dbd_bind_ph(1): bind :p1 <== 'name1' (type 0 (DEFAULT (varchar))) dbd_rebind_ph_char() (1): bind :p1 <== 'name1' (size 28/32/0, ptype 5( +VARCHAR), otype 1 ) dbd_bind_ph(1): bind :p2 <== 'farm1' (type 0 (DEFAULT (varchar))) dbd_rebind_ph_char() (1): bind :p2 <== 'farm1' (size 25/32/0, ptype 5( +VARCHAR), otype 1 ) dbd_bind_ph(1): bind :p3 <== 'z1' (type 0 (DEFAULT (varchar))) dbd_rebind_ph_char() (1): bind :p3 <== 'z1' (size 24/32/0, ptype 5(VAR +CHAR), otype 1 ) dbd_st_execute INSERT (out0, lob0)... Statement Execute Mode is 32 (COMMIT_ON_SUCCESS)

After this, the script gets stuck at inserting the 1st record and then I have to kill it.

Could you tell me what I am doing wrong here. I think I have missed something obvious but not able to figure out that.

-Thanks.

Comment on DBD::ORacle: Not able to insert into Database
Select or Download Code
Re: DBD::ORacle: Not able to insert into Database
by Anonymous Monk on Jul 25, 2012 at 11:45 UTC
    This is wrong
    my @array = <$FH> ; .. while (<@array>) ..
Re: DBD::ORacle: Not able to insert into Database
by roboticus (Canon) on Jul 25, 2012 at 11:51 UTC

    slayedbylucifer:

    A couple things:

    • Your while loop contains '<' and '>' which aren't necessary: Those tell perl to read from a filehandle. But you've already read your data into the array. Just remove the angle brackets, and that should clear up the first problem.
    • You're not chomping the lines you're reading, so you may be adding newlines to your database that you probably don't want.

    Those are the only problems I see. However, you might want to rearrange your code to remove the unnecessary array. You can do so like this (untested):

    #!/usr/bin/perl -w use strict; use DBI; open (my $FH, '<', "test.csv") or die "Cannot open test.csv: $! \n"; my $dbh = DBI->connect ("dbi:Oracle:host=<hostname>;sid=<SID>", '<username>', ',password.', { RaiseError => 1, AutoCommit => 1, TraceLev +el => 4 } ) or die "Canot create Databse Handle: $DBI::errstr() \n"; my $sth = $dbh->prepare ("INSERT INTO CLUSTER_MAPPING (name, farm, dmz +) VALUES (?,?,?)"); <$FH>; ### this is to remove the column headers while (<$FH>) { my @row = split /,/; my $col1_value = $row[0]; my $var1 = $row[1]; $var1 =~ s/"//g; $var1 =~ s/^ //g; my $col2_value = $var1; my $col3_value = $row[2]; $sth->execute ($col1_value, $col2_value, $col3_value); } # If this is the end of your program, these lines are unnecessary, as +it'll all be # cleaned up at the end of the program. But if you're continuing to d +o something # else in your program, then by all means leave them in. $sth->finish(); $dbh->disconnect(); close $FH;

    I also removed the unnecessary variable $table as I couldn't see the purpose for it.

    You can do further simplifications, as well. There's no need to create new variables for the columns, you can just operate on the values in place and use them, like so:

    while (<$FH>) { my @row = split /,/; $row[1] =~ s/"//g; $row[1] =~ s/^ //g; $sth->execute (@row); }

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

      roboticus, I ma working on your suggestion. Will let you know how it goes. Thanks for your time.
      Roboticus, I have cleaned up my code as you suggested. but my problem is still there. the records are not going into the database. It gets stuck at inserting the 1st record and then I have to kill the script with CTRL+C. the debug output I have mentioned in my original post also does not proceed beyond that point. Thanks for your time.

        Can we see the cleaned up version?

        What happens if you hit ctrl/d instead of ctrl/c?

      Roboticus, Finally I got it working. there were quite a few issues apart from the cleanliness:

      - First, The chomp part that you said not working here although it was essential. Please find my discussion on below threads: http://stackoverflow.com/questions/11645696/perls-chomp-chomp-is-removing-the-whole-word-instead-of-the-newline Chomp is removing the whole word instead of the newline

      - I had this chomping problem way before I created this thread. I have documented solution on the stackoverflow one. will post my solution on perlmonks as well

      - Second, for whatever reason, I had to unassign "primary Key" column by logging to the SQL developer. Although my entries were not duplicates...still..I had to do it

      - Third, I was uploading the values in a wrong way. The DBI documentation led to use the "$dbh->quote" method to upload strings.

      - Finally, I used Text::CSV to retrieve individual values of my CSV and then uploaded them using the "$dbh->quote".

      Here is the csv:

      vm,farm,dmz name1,farm1,z1 name2,farm2,z2 name3,farm3,z3

      and here is my code:

      #!/usr/bin/perl -w use strict; use Text::CSV; use DBI; my $csv = Text::CSV->new ( { binary => 1 } ) or die "Cannot use CSV: ".Text::CSV->error_diag (); open my $fh, "<:encoding(utf8)", "test.csv" or die " $!"; <$fh>; ### this is to remove the column headers. my $dbh = DBI->connect ("dbi:Oracle:host=<hostname>;sid=<SID>;port=152 +6", 'username', 'password', { RaiseError => 1, AutoCommit => 1, TraceLevel + => 0 }) or die "Cannot create Database Handle: $DBI::e +rrstr()"; while ( my $row = $csv->getline ($fh)) { my $col1 = $dbh->quote ("$row->[0]"); my $col2 = $dbh->quote ("$row->[1]"); my $col3 = $dbh->quote ("$row->[2]"); $dbh->do ("INSERT INTO CLUSTER_MAPPING (VM, FARM, DMZ) VALUES +($col1, $col2, $col3)"); } $dbh->disconnect(); close $fh;

      Thank you very much for your advice and time.

        1. The "chomp issue" was that you expected chomp to return the chomped string but it instead returns the number of characters chomped.
        2. I don't believe you needed to remove the primary key. I think you had some other issue but that you changed more than one thing at once. If the column values are unique or you are going to select via that column you want an index or unique index. As you started with problems attempting to read from a file with <> I suspect you somehow where reading from stdin and hence it looked like your script was hung. That is why I suggested pressing ctrl/d.
        3. It was suggested you inserted the values as parameters and that is still the best way to do it. As you have it you are having to build a SQL string each time when in fact before your while loop you could simply do one prepare with placeholders and then only call execute inside the while loop. It would also avoid all those quote calls.

Re: DBD::ORacle: Not able to insert into Database
by thargas (Chaplain) on Jul 25, 2012 at 13:50 UTC
    If you're using CSV, you'll probably be better off using Text::CSV

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://983630]
Approved by marto
Front-paged by MidLifeXis
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2014-07-12 17:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (240 votes), past polls