comment on

This is driving me insane...

I have a perl script that digests some info and loads it into mysql and it was running happily, albiet a bit slowly, inserting line by line using DBI.

While researching mysql optimazation I found out that I should be writing the data to a file and bulk loading it with "load data infile...".

When I got that running, I found that I getting 3557 records as opposed to the 3599 I was expecting and the last record was monged (all defaults or nulls). I check the tmp file and it had 3599 lines, I printed out the command from the perl script and appened it to mysql -e and got 3599 records.

I thought, maybe, it was the pipe not clearing before the script moved on, so I tried to find out how flush things, but couldn't find anything. I even tried running a commit...

I slept on it and this morning decided that since it works from the command line, I'd just rip out the DBI stuff and run the command line version with back ticks... 3557 records!

ARGHHHHHHHHHHHHHHHHH!

I saw something simular in "Unable to load entire CSV file into DB", but didn't see a resolution or any ideas I though applied to me...

Can anyone help me, before I go insane?

Thanks,

Jimbus

Never moon a werewolf!

In reply to DBI/load data infile issue by jimbus

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Do you know where your variables are?
	PerlMonks