Importing files with perl::dbi

[Anonymous Monk]

what is the best way to import a file into either a text or blob field in a sql database using perl::dbi?

[httptech]

use DBI;

my $dsn = "DBI:mysql:foobar";
my $user = "foobar";
my $pass = "baz";
my $textfile = "testing.txt";

open (TEXT, "$textfile") or die "Couldn't open textfile: $!";
$dbh = DBI->connect($dsn, $user, $pass,{ RaiseError => 1})
   or die "Could not connect to database! $DBI::errstr";
{ 
  local $/ = undef; 
  $dbh->do("INSERT INTO foo (bar) values (?)", undef, <TEXT>);
}
$dbh->disconnect;
close TEXT;

[johannz]

Looking at the O'Reilly Perl/DBI book(pg 153, 'Inserting and Updating LONG/LOB columns'), I think it's better if you use bind params with a datatype argument. I have not tried the following code, but it should work:

use  DBI qw(:sql_types);
my $dsn = "DBI:Oracle:foobar";
       my $user = "foobar";
       my $pass = "baz";
       my $textfile = "testing.txt";

       open (TEXT, "$textfile") or die "Couldn't open textfile: $!";
       my $longText = join('', <TEXT>);
       close(TEXT);
       $dbh = DBI->connect($dsn, $user, $pass,{ RaiseError => 1})
          or die "Could not connect to database! $DBI::errstr";

$sth = $$dbh->prepare(
    'INSERT INTO table_name (key_name, long_description) VALUES (?, ?)
+';
$sth->bind_param( 1, 42);
$sth->bind_param(2, $longText, SQL_BLOB);
$sth->execute();

[httptech]

My solution already uses bind params in the "do" method. I don't think it's necessarily faster to "prepare" and "execute" than to just "do".

However, it is faster to undef $/ and then pass <TEXT> as a scalar than to run "join" on every line and pass the intermediate variable.

[johannz]

While normally just using a "do" would be correct, in this case since we are talking about a BLOB, using bind with a datatype is safer. By that I mean, when you execute a "do" with bind parameters inline, you are leaving it up the the DBI interface to figure out the datatype. Sometime it will get it right, but there is no guarentee. By using a "bind_param" call with a datatype you are giving the DBI a strong hint as to how to handle the data. To quote again from the Perl DBI book( pg. 154):

Passing SQL_BLOB as the optional TYPE parameter to bind_param() gives the driver a strong hint that you're binding to a LONG/LOB type. Some drivers don't need the hint but it's always a good idea to include it.

I keep quoting this book since the authors are Alligator Descartes and Tim Bunce, both of whom have been heavily involved in the design of the DBI interface. I tend to believe what they say about how to use the interface.
Your second point about undef-ing $/ and passing in <TEXT> I will concede. On examples I tend to be more verbose since it makes debugging easier since you can easily look at the value of a varible. Looking at a filehandle is distinctly harder. :-)

[PipTigger]

I'm totally a MySQL rookie but I would recommend using a text (upto 64k) or longtext (upto 4GB) if I was reading any textual file because the text types are searchable within SQL with their own flavor of regexes. Although they're not as nice as Perl's, they may be faster on their own data (especially if it's large data like longtext) because it won't have to all be duplicated in memory into a scalar for Perl to examine.

A blob is basically for binary data as I understand it. If you wanted to store image files (or sounds) in your database, that would probably be the best place to store them. I guess a simple answer is: If the file you wish to import contains binary data, use the blob type and use text for anything else. I hope this helps. TTFN & Shalom.

-PipTigger

p.s. Initiate Nail Removal Immediately!

[chromatic]

Assuming you already have your table created, open the file, read it into a variable, prepare a SQL statement (INSERT INTO table VALUES ('filename', ?)), call execute and pass the variable containing the file, and check to see that the insert succeeded.

It's really no different from inserting any other data into a database.