Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling


by Generoso (Parson)
on Oct 14, 2013 at 04:51 UTC ( #1058128=note: print w/replies, xml ) Need Help??

in reply to perl DBI LOAD DATA INFILE

Sorry I was not logged in. Here is the entire article

MySQL Database

How do I import delimited data into MySQL?

Applies to: Grid System

If you have data that you need to bring into your MySQL database, there are a few ways to do it. Exporting data out of mysql is another topic, described here.

1. Using the LOAD DATA INFILE SQL statement

For security reasons, no one has the mysql FILE priv, which means you cannot "LOAD DATA INFILE". You can, however, use a "LOAD DATA LOCAL INFILE" statement as long as you have a mysql prompt on our system and have uploaded the data file to your account here first.

The "LOAD DATA LOCAL INFILE" statement will only work from a MySQL prompt on our local system. It will not work from any web-based tool such as phpMyAdmin, and will never pull a file in directly off your own computer.

To import a file this way, first upload your data file to your home directory on our system with FTP or SCP. Then get a shell prompt on our system, and then a MySQL Monitor prompt so that you can issue the SQL that will import your file.

For example, suppose you have a data file named importfile.csv that contains 3 comma separated columns of data on each line. You want to import this textfile into your MySQL table named test_table, which has 3 columns that are named field1, field2 and field3.

To import the datafile, first upload it to your home directory, so that the file is now located at /importfile.csv on our local system. Then you type the following SQL at the mysql prompt:

LOAD DATA LOCAL INFILE '/importfile.csv'
INTO TABLE test_table
(field1, filed2, field3);

The above SQL statement tells the MySQL server to find your INFILE on the LOCAL filesystem, to read each line of the file as a separate row, to treat any comma character as a column delimiter, and to put it into your MySQL test_table as columns field1, field2, and field3 respectively. Many of the above SQL clauses are optional and you should read the MySQL documentation on the proper use of this statement.

2. Using a script to parse and import the file

You can also write a script in any programming language that can connect to MySQL (such as PHP) to open your data file, break it up into an array of lines that each represent a row of data, split each line up by the delimiter character (such as a comma ',', tab '\t', semicolon ';', space ' ', etc.), and then perform invididual MySQL INSERT queries (one INSERT for each line) to insert all your data from the file into the appropriate table fields.

Such scripts are not difficult to write in less than 15 lines and can import data from text files just as effectively as a LOAD DATA LOCAL INFILE command. A working example script written in PHP appears below in the Annotations.

3. Importing a mysqldump

If your data file actually comes from another MySQL database, and not from Excel or any other source, then the most direct way to export and import your data would be to dump out your table or entire MySQL database on the original database server using the mysqldump command, FTP the resulting dump file to your account here, and then import the dump file at a shell prompt.

For instructions on creating the dumpfile using the mysqldump command, see this FAQ. For instructions on how to import a dump made with mysqldump, see this FAQ.

Here is an example of creating a data file and then importing it with LOAD DATA syntax.

mulder@modwest:/$ echo 'a b c d' > testfile
mulder@modwest:/$ mysql

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3979754 to server version: 4.0.15-Max-log

mysql> CREATE TABLE ´test´ (
-> ´fe´ VARCHAR( 2 ),
-> ´fi´ VARCHAR( 2 ),
-> ´fo´ VARCHAR( 2 ),
-> ´fum´ VARCHAR( 2 )
-> );
Query OK, 0 rows affected (0.00 sec)

mysql> load data LOCAL infile '/testfile' into table test
fields terminated by ' ' lines terminated by '\n';
Query OK, 1 row affected (0.02 sec)
Records: 1 Deleted: 0 Skipped: 0 Warnings: 0

mysql> select * from test;
| fe | fi | fo | fum |
| a | b | c | d |
1 row in set (0.00 sec)

mysql> load data LOCAL infile '/testfile' into table test
fields terminated by ' ' lines terminated by '\n' (fum, fo, fi, fe);

Query OK, 1 row affected (0.02 sec)
Records: 1 Deleted: 0 Skipped: 0 Warnings: 0

mysql> select * from test;
| fe | fi | fo | fum |
| a | b | c | d |
| d | c | b | a |
2 rows in set (0.00 sec)

mysql> exit

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1058128]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2017-09-21 05:23 GMT
Find Nodes?
    Voting Booth?
    During the recent solar eclipse, I:

    Results (242 votes). Check out past polls.