Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: index for large text file

by GrandFather (Sage)
on Mar 28, 2011 at 09:32 UTC ( #895872=note: print w/replies, xml ) Need Help??


in reply to index for large text file

Would this data be better stored in a database? To answer that you need to think about how the files are generated and used. If you look them up much more often than they are generated a database may be very worth while. If relatively small numbers of records change from time to time that may be another good reason to use a database. If you have control over generation of the current file that will make using a database easier.

True laziness is hard work

Replies are listed 'Best First'.
Re^2: index for large text file
by Anonymous Monk on May 16, 2013 at 23:08 UTC
    I do not think a database would be appropriate considering this file type (FASTQ) can easily have 200,000,000 records (200,000,000 X 4 lines). We (biologists) can often have 100's if not 1,000s of these files.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://895872]
help
Chatterbox?
[ambrus]: Corion: which coordinate system is the upside down one?
[Corion]: ambrus: "the other" ;) To me, "normal" for GUIs means (0,0) is left-upper corner of the monitor/window. But for Prima, the lower-left corner it is ;)
[Corion]: Which would be OK for mathematics and paper, for me ;)
[ambrus]: Ok, so Prima has a postscript-like coordinate system

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (8)
As of 2016-12-09 10:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:













    Results (150 votes). Check out past polls.