Re: index for large text file

[GrandFather]

Would this data be better stored in a database? To answer that you need to think about how the files are generated and used. If you look them up much more often than they are generated a database may be very worth while. If relatively small numbers of records change from time to time that may be another good reason to use a database. If you have control over generation of the current file that will make using a database easier.

[Anonymous Monk]

I do not think a database would be appropriate considering this file type (FASTQ) can easily have 200,000,000 records (200,000,000 X 4 lines). We (biologists) can often have 100's if not 1,000s of these files.