Preprocessing input files is not something to be afraid of. Of course, it depends on the flow of your data but it is sometimes more efficient to split the job into two distinct, simpler parts than to try and code one more complex solution.

It seems as if you have a (approximately) a set of client system running something like a data entry or data processing system. You are taking the results from these client systems and loading/processing them into some master program. Even if you are just swapping files among systems, a preprocessing step could be very helpful (your milage may vary). The benefit is that, at a certain point in the flow of data, the data will all look exactly the same regardless of originating/destination platform. That can greatly simplify further processing downstream of that point.

Its worth looking at more closely.

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.