in reply to Spliting file + removing column
Your code is a bit too much for me to “grok” on my first cup of coffee, but when I am presented with a problem like this, I try to identify what are the different kinds of input-lines my program may expect (I see two...), and to write a handler sub for each one. I will use a regular-expression to parse the lines as specifically as possible. And, I will aggressively code this part of the program so that it will die if it encounters anything that does not conform to my specifications because, in a file of 1,000,000 records, that’ll be the only possible way to find the problem. (The fact that “the program seems to run to completion” doesn’t cut it...)
My next concern, once I have verified that I can correctly read the entire file without any untimely deaths, will be to consider the order in which the various types of records may occur. For this, I use “finite-state machine (FSM)” logic, where the program initializes, say, $state = 'INITIAL_STATE';, and then examines and changes the value of $state to indicate where it is and what it has seen recently.
This state-driven logic constructs some kind of in-memory data structure, and, at the appropriate point ($state) in time, does something with it.