Re^3: How to optimise " csv into mysql " using TEXT:CSV_XS and DBD::Mysql

[Corion]

This is the approach I use with a different database which basically can only do bulk loads, because single INSERTs are very slow. The approach is:

1. Read input data using Text::CSV_XS
2. Perform cleanup on each row
3. Add relevant columns (source_file, source_date)
4. Write output TSV file in the correct encoding (Text::CSV_XS, this time writing tab-separated output)
5. Issue bulk load statement
6. Check that the number of rows read in 1, written in 4 and retrieved from the database are as expected

[taiko]

As I don't need to add/delete columns, I guess I just can make a cleanup using INFILE edit as I'm doing now and then LOAD INTO as you proposed. There is no real added value of using TEXT:CSV_XS, or am I missing something? Actually, if I look to my code, I'm using TEXT::CSV_XS for no reason :). Maybe just wanted to be fancy :)