Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re: using the split command

by InfiniteSilence (Curate)
on Jan 26, 2012 at 23:35 UTC ( #950240=note: print w/ replies, xml ) Need Help??

in reply to using the split command

Maybe my brain's just not working, but I don't see how what you are proposing to do will help you with your goal. If you are looking to take data from a production database and archive it and then subsequently remove that data from the production data there is no need to convert to CSV. Database tables should have keys. Move the data from A->B and then cache all of the keys using DBM. Confirm all data was moved to B. If true, then go ahead and iterate through all of the keys and delete them from the production database. That's if you need to do this in Perl.

I wouldn't do this in Perl at all however. I would create a backup database or schema in my database and use SQL in a transaction to do this work.

create table foo; create table foo (id text, name varchar(15)); insert into foo values ('a','foo'); insert into foo values ('b','foo'); insert into foo values ('c','foo'); create table backupfoo (id text, name varchar(15)); insert into backupfoo (select * from foo); delete from foo; -- you would obviously use a WHERE clause to restrict + what you are deleting >>> 3 records copied
Now, you can just push everything from the backup table to an archive offline (example uses PostgreSQL).

pg_dump -d YOURDATABASE -t backupfoo > mybackup.sql

Then you can reload that data in a different database entirely.

psql -d YOURDATABASEARCHIVE -f mybackup.sql

You could script this or use a CPAN module to script this.

Celebrate Intellectual Diversity

Comment on Re: using the split command
Select or Download Code

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://950240]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (8)
As of 2014-08-29 15:04 GMT
Find Nodes?
    Voting Booth?

    The best computer themed movie is:

    Results (281 votes), past polls