Improvements also very welcome. Much of this code was written in quick and dirty mode, sensible re factoring need not apply. While anything is possible, this is the second part of two scripts, the first writes the file this one reads at an earlier point, the one with '|' as divider so at least that is unlikely. Thanks for the better design suggestions! I'm thinking that the best chance for a 'change' as smoking gun is a significant jump in data size. This is of course complicated by the time it takes to process a file of large size (hours--- companion takes the longest portion).
"Never try to teach a pig to sing...it wastes your time and it annoys the pig."