Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^3: sorting very large text files (slander)

by erix (Prior)
on Dec 21, 2009 at 20:39 UTC ( [id://813757]=note: print w/replies, xml ) Need Help??

Help for this page

Select Code to Download


  1. or download this
    Result:
      System sort: 59 minutes.
      Postgres load (6m) + select (59m): 66 minutes
    
  2. or download this
    $ ls -lh dna2.txt 
    -rw-rw-r-- 1 xxxxx xxxxx 12G Dec 21 16:38 dna2.txt
    ...
    real    59m12.569s
    
  3. or download this
    $ time echo "
           create index dna_test_seq_idx on dna_test (seq)
    " | psql -d test
    CREATE INDEX
    real    63m20.451s
    
  4. or download this
    
    $ time echo "
    ...
      ->  Seq Scan on dna_test  (cost=0.00..3872406.68 rows=181261568 widt
    +h=62) (actual time=17.966..65802.621 rows=181261572 loops=1)
    Total runtime: 3463580.243 ms
    

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://813757]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (7)
As of 2024-04-23 12:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found