Ansi has asked for the wisdom of the Perl Monks concerning the following question:
I have very little expereince of Perl, so in the first instance please excuse my ignorance. All I'm looking for to start is some quick pointers on how to write a CSV handling application using perl without reinventing the wheel.The app in question I've already written albeit in MsAccess about ten years ago; I believe it's some subset of VB but I'm not too sure. I write it solely by using the help files that MsAceess provides but beyond that I don't know much about the language used but I imagine the methodology wouldbe the same nomatter what the language
I have done some reading on Perl but in the first instance I really want to avoid learning much more than I need to produce the app thatI need. To that end I'mjust after some general pointers of what modules etc I should be using
In brief, the program handles and manipulates text files, here's a rough flow chart of what happens
CSVaddress file (one address record per line) is imported
Look for postcode in each address/line (which is a predicatable alphanumeric format)
Postcode is then looked up in a two column table for a corresponding value
Corresponding value is added as an additonal field to the address line
If no postcode is found, then each field is checked against a different two column table for a matching post town andcorresponding value which is then appended tothe address line/record
Thereafter there are some sql sorting routines which involve selecting and sorting the "corresponding value" fieldi, and adding sequential counts etc and producing some reports
This is all a little simplified; there is a bit more to it than this, but the process above is the raison d'etre of the app.
What woud be helpful in the first instance is some general guidance on the best way of handling the above. I'm mindful ofthe fact that if handled incorrectly memory could be an issue. The address file can have (in theory) over a million records though in practice 100,000 is very large and mostly it's around 10,000 records. The 2 x two column tables are both relatively small, no more than 1500 rows
So, what modules/methodology would be best to sort through the address records, then cycle through the fields to find the value which would then be used to cycle throogh one or both two column table to find the value toadd back to the address record. I'll leave the sorting and everything else for later. In the first instance if I canget it to output the postcode that's a good start. When I wrote this initially i did it all using babysteps anyway
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: Parsing/manipulating CSV files
by CountZero (Bishop) on Oct 20, 2011 at 22:10 UTC | |
Re: Parsing/manipulating CSV files
by tospo (Hermit) on Oct 20, 2011 at 22:39 UTC | |
by Anonymous Monk on Oct 21, 2011 at 11:00 UTC | |
by Tux (Canon) on Oct 21, 2011 at 13:17 UTC | |
Re: Parsing/manipulating CSV files
by Anonymous Monk on Oct 20, 2011 at 23:11 UTC | |
Re: Parsing/manipulating CSV files
by mrstlee (Beadle) on Oct 21, 2011 at 15:22 UTC | |
by Ansi (Initiate) on Oct 21, 2011 at 18:07 UTC | |
by mrstlee (Beadle) on Oct 22, 2011 at 12:32 UTC |