Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^4: Best method to load a csv file.

by BravoTwoZero (Scribe)
on Dec 17, 2004 at 16:02 UTC ( #415677=note: print w/replies, xml ) Need Help??


in reply to Re^3: Best method to load a csv file.
in thread Best method to load a csv file.

Plus, you can always call your database import utility from the same perl program that modifies your data when it's complete. With that number of records, I'd still write it out to a text file, though, before calling the external db load utility.

Amatuers discuss tactics. Professionals discuss logistics. And... my cat's breath smells like cat food.

Replies are listed 'Best First'.
Re^5: Best method to load a csv file.
by samgold (Scribe) on Dec 17, 2004 at 16:08 UTC
    That's true. So back to the question of parsing the csv file, which do you think is better or faster? The way I am doing it now with Text::ParseWords or DBI::CSV or Text::csv?

    Thanks,
    Sam
      Use Text::xSV or Text::CSV_XS. The other modules tend to have subtle bugs in them.

      My question is why aren't you loading the data into temp tables, then using PL/SQL to clean the data up later? That's what I did when I had to do something similar on a daily basis and it was quite fast ...

      Being right, does not endow the right to be rude; politeness costs nothing.
      Being unknowing, is not the same as being stupid.
      Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
      Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

        I needed a fast way to load the data. By fast I mean something developed quickly. Now I have some time to go back and look for a better way of doing it. I may just do that, my PL/SQL skill are bit better now and I am more comfortable using it.

        Thanks,
        Sam

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://415677]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (2)
As of 2021-01-19 05:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?