Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Speed up file write taking weeks

by dave_the_m (Monsignor)
on Jun 29, 2019 at 14:56 UTC ( [id://11102120]=note: print w/replies, xml ) Need Help??


in reply to Speed up file write taking weeks

If we assume that writing 1.7 trillion records takes two weeks, then that's 1.4 million records per second, which is a reasonable number, assuming that the fields are of a decent size. So while you might be able to speed up things a bit, I'd be surprised if you could reduce it from weeks to days. You'd have to paraellelise it, with multiple disks and CPUs

Dave.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11102120]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (3)
As of 2024-04-20 01:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found