Re^2: Performance Trap - Opening/Closing Files Inside a Loop

graff,
Presumably, the files need to be appended in the order encountered. Part of the long story unmentioned is a lot of guarded responses to my inquiries for additional information. A cut | sort | uniq might not be a bad idea to pre-process the file to get a list of unique file names though.

Cheers - L~R

Comment on Re^2: Performance Trap - Opening/Closing Files Inside a Loop

Replies are listed 'Best First'.
Re^3: Performance Trap - Opening/Closing Files Inside a Loop by graff (Chancellor) on Dec 10, 2004 at 04:05 UTC
I wonder if all java programmers are this cagey/evasive about describing their problem sets... Even so, now we're just talking about a two-stage sort: `## let's suppose the file names are in column 3 of "table.txt": perl -pe 's/^/$.,/' table.txt \| sort -t , -k 4,4 -k 1,1n \| cut -f2- -d +, \| splitter.pl` [download] where "splitter.pl" is a version of your suggested script that assumes lines are pre-sorted by output file name -- so it really needs only one output file handle open at any one time. By pre-pending the original line numbers before sorting, and using the line numbers as a secondary sort field, the (presumably) intended result is achieved. (update: if the original table has file names in column 3, and a perl script prepends a line number to each line, then the primary sort column has to be 4, not 3.)	[reply] [d/l]
Re^4: Performance Trap - Opening/Closing Files Inside a Loop by Limbic~Region (Chancellor) on Dec 10, 2004 at 15:43 UTC
graff, I wonder if all java programmers are this cagey/evasive about describing their problem sets I doubt it. There are people in all trades and positions that like to keep what they do a secret in fear that if people know then their importance and value might be diminished. Since my job title has nothing to do with programming, I am assuming solving such a "simple" task with an "inferior" language would be a huge eyesore. Anyway - it is the Java Developer's problem - not mine. I just wanted to present the problem here so that if there was an superior approach that I was missing I could learn it. Cheers - L~R	[reply]