|Problems? Is your data what you think it is?|
Thinking out loud (was: Re^2: The Eternal "filter.pl")by Voronich (Hermit)
|on Aug 29, 2011 at 14:00 UTC||Need Help??|
There was a time when I would have set out to do exactly that. But the problem I find with that approach is that it very quickly races past the point of diminishing returns, eventually becoming like J2EE where you have a two line program and 50k of 'configuration'. But it does force me to think about what I do and don't want to abstract away.
I definitely want to abstract the simple "parse to records" logic into something that I can iterate across because the semantics are there and clean in perl already.
But the iteration driver itself can be tricky and I'm not sure it shouldn't just have a few different permutations:
Nah, see that's already messy as hell. Well ok. Maybe there's a reasonable way to make that work; or perhaps it's just not as bad as the description. (Actually I suspect that they're all degenerate cases of the same construct.) I'll have to see once I get everything else out of the way.
Having thought about this for some time now I'm pretty sure I want the actual "filtering function" to be a plain perl function that takes a pair of records. Perl actually does code well and there's no reason for me to abstract everything SO far that I end up having to write a programming language.
Because of that, I'm fighting even with the idea of whether or not a record parsing construct should provide column/datatype information. What good would it REALLY do? The filter function is likely to be very specific to the task at hand, so documentation (in the form of well-named variables, etc) can very effectively be contained therein and operations on the data would require that I re-interpret it. That also sounds a lot like writing a programming language (which I'm sure is fun, but I haven't come up with a good reason to do yet.)
I think I may have thought my way as far as I'm going to think without writing more code.