Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re: What's the biggest piece of work you've done alone with Perl?

by gawatkins (Monsignor)
on Apr 22, 2003 at 22:29 UTC ( #252390=note: print w/replies, xml ) Need Help??

in reply to What's the biggest piece of work you've done alone with Perl?

The largest single piece of code that I have written takes the back end database files from our Avaya CMS server, renames them, inserts them to .xls files, and writes them to a database.

The first part involves the renaming the files from their default name of chrXXX.XXX where the X's are numbers. One of our remote reporting teams likes to receive them as csv files, so I simply open the directory and copy the files to a directory with the extension csv.

The second part was a bit more difficult, I have been using the Spreadsheet::WriteExcel::Big CPAN module to write the files to an excel spreadsheet (for one manager that only uses MS Excel) containing all of the call records each day. I had some difficulties, but solved them by chopping the files at about 14000 lines.

The last piece involved writing the daily call records to a MS SQL database, then a MYSQL database for our local reporting team to generate reports from.

My main strategies involved trying to go out of my way to get everyone and what format they wanted and to help out local reporting guy who was spending all his time doing this task manually. As for the modules, I found Spreadsheet::WriteExcel::Big to be very good.

My biggest mistake was using MS SQl as a db the first time. The tables csv files have 55 columns, somewhere between 20,000 and 30,000 records each day. I had a terrible time with the DBI, each record was taking around 4 second to write. Finally I wised up and was able to write the all the records in just a couple of hours each day to the MYSQL db with the exact same piece of code, go figure.


Greg W

  • Comment on Re: What's the biggest piece of work you've done alone with Perl?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://252390]
[stevieb]: shmem thanks for the 'insight' :P
[shmem]: good thing that Sun already took "OpenWindows", otherwise I'd not stop to shudder imagining an "OpenWindows" from MS
[shmem]: more garbage in, more garbage out that would be
[stevieb]: I found that win10 broke a C# library I was using for one project while enhancing tests for a Perl dist, which breaks other Perl dists, and I'm about to throw my hands up on berrybrew. win2k12 broke one thing, win10 breaks something...
[stevieb]: ...unrelated which requires replacing a lot of code and a whole lib. I'm about to go nix only ffs
[shmem]: stevieb: what you're doing sounds afwully complex. Too much for me this evening to provide brighter insight ;-)
[stevieb]: I don't even own a Windows computer. Both my girl and I have a laptop each with Linux. I'm supporting Windows in some of my projects and I can't even guage whether it's worth it or not.
[stevieb]: shmem It's something I desired to have years ago, which is why I took over berrybrew. Cross-platform build/test automation locally, or over the network Test::BrewBuild
[shmem]: sounds good.
[shmem]: but I'm crumbling smaller stones. remember...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (5)
As of 2017-03-28 22:04 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (342 votes). Check out past polls.