A Perl Success Story

[TedYoung]

So, I never thought anyone would care about some of my programming stories, even those that were wildly successful because of Perl. But, over the last few years, I have seen an interest in Perl Success Stories here at the monastery; perhaps because it is reassuring to see that other people find Perl helpful, and maybe it is that some of these stories can be quite interesting. So, here I offer one of my earlier stories, hopefully one you will find inspiring and amazing, or at least funny.

Perl is a great friend to know. He is there when you need him. When you don't, he steps back and quietly waits. He doesn't get jealous while you are checking out that new scripting language to see if it isn’t any good. He doesn't get upset when you choose another language for a project. He just hangs out, always there if you need any help.

Our company just picked up a new client. They had an aging system, on the verge of failure. They needed system upgrading quick. They had several home-grown applications running against a central database server. The applications were written in a language/environment called SQL Windows 5. SQL Windows was an IDE with similar goals to Visual Basic. It provided drag-and-drop GIU design, fill-in-the-event based development, and built-in database server integration that was never intended for the load they were creating.

SQL Windows wasn't a very nice IDE, either. It may have been OK in its day, but at this time VB 6 was the standard, and in comparison a far better product. For example, the IDE forced you to write code in an outline. You find your window, expand it, find your event, expand it, right-click and insert a new statement, type a little, and so on. Each statement had to be added to this outline. There was no free-text editing of the language within the IDE. I never liked VB much, but I would gladly choose it over this product any day.

So, I successfully upgraded their database to SQLServer. Now I needed to do something about their code base. We chose to port their code to VB6 due to the many similarities in functionality and operation of the two languages. The problem was; I had the file with 70,000 lines of SQLWindows code, a language I knew nothing about.

Most of this file was meta information describing the various properties set for each of the form windows and their contents. It also included definitions for functions and implementations for event handlers, initialization of variables and the structure of classes. Here is a small example:

.head 6 +  Pushbutton: pbDoIt
.head 7 -  Class Child Ref Key: 5
.head 7 -  Class ChildKey: 17
.head 7 -  Class: ADV_frmM_DynSql
.head 7 -  Property Template:
.head 7 -  Class DLL Name:
.head 7 -  Title: Prev
.head 7 -  Window Location and Size
.head 8 -  Left: 0.833"
.head 8 -  Top: 0.06"
.head 8 -  Width:  0.7"
.head 8 -  Width Editable? Class Default
.head 8 -  Height: 0.536"
.head 8 -  Height Editable? Class Default
.head 7 -  Visible? Class Default
…
.head 5 +  On SAM_Click
.head 6 +  If SalMessageBox( 'Register ' || sName || ' to ' || sJobNum
+ ||
' - ' || SalNumberToStrX( nSession, 0 ) || '?', 'Notice', MB_WarningYe
+sNo ) = IDYES
.head 7 -  Set nDefault = SalListQuerySelection ( lbDefaults )
.head 7 +  If nDefault = LB_Err
.head 8 -  Call SalMessageBox( 'Select a default first.', 'Notice', MB
+_WarningOk )
.head 8 -  Return FALSE

The .head tells the level of nesting of this line in the outline. The +/- tells whether the outline item was expanded in the IDE (doesn't matter to me).

It was obvious that most of the meta information could be converted directly to VB. Things like the form windows, their sizes, their contents, the size, location, and other properties of their contents, menus, event handlers, variables, functions and classes could all be ported. If I could convert everything except the code itself, it would save me an enormous amount of time. In fact, I wanted to convert their entire code base (70K LOC) in a week. Also, I wanted something that could be used on other SQLW projects, since they had several other applications that would need to be ported later.

The only thing standing in my way was I didn't know the entire structure of the file, nor did I want to learn it. I wanted to be able to pull out bits and pieces as I needed them, ignoring the rest. Unfortunately, it would be very hard to parse if I could not determine what lines were meta data and what were lines of actual code on a per-line basis.

Here is where I got lucky. It turns out that each line of code in the language always starts with a comment char (!), or one of eleven keywords. I hit the jackpot!

I whipped out Perl and wrote a script to convert the code to… XML! You are probably wondering "Oh, why did he have to shoe-horn XML into this mess?". It is an odd choice. The outline-oriented structure of the code base lent itself to XML very well, and the declarative nature of XSLT made it very easy to generate all of the VB files needed without having to do complex data queries/lookups. There were some minor issues. Normally in XML whitespace is not as significant as it is in code, so it can be a bit trying to get code to format they way you want when generating it with XSLTs.

Regardless, after a couple of days of tweaking the stylesheets, I could start porting all of the commented-out SQLW code I included in my VB files. I few helper libraries made that a trivial task. I realized I could have gone a step further and used Perl to convert the majority of SQLW syntax to VB too.

I never explicitly planned to use Perl in this project. It was something that just came up. And when I needed it, it was there and it saved the day (and months of development time).

I have studied and used scores of languages from C to VB, Java to Python, Ruby, PHP, even more esoteric ones like Prolog and LISP. None has ever been a side-kick to me like Perl. I program full-time in Perl now. Aside from giving me a source of income, Perl is there when I need an laugh (often in the form of an obfu, or crazy feature), and when I need a distraction (anyone for golf?), and often saves my @$$ when I screw up. Thanks Larry!

PS: I will try to make my next Perl Success Story a little shorter. :-)

[derby]

Very nice ... are you able to quantify the cost savings of using the perl-based conversion versus a full manual conversion? That would be something to see!

[TedYoung]

It would be nice if I could. In all, we converted over 100K LOC over three applications, with a minimal amount of hand coding. Where we really saved time was in short term bug support. Had we redisigned everything by hand, surely, we would have had many many more bugs. An off the cuff guess would be on the order of 6 months to do it by hand, instead of just over a week with the conversion.

[derby]

There's nothing wrong with off-the-cuff guesses and if I had to put a dollar figure on your savings, it would be something like this:

• a month is 4 weeks
• a week is 40 hours
• assume one engineer
• assume $100 labor rate
• cost for six months - 96,000 (6*4*40*1*100)
• cost for one week - 4,000 (40*1*100)
• savings - 92,000 (96,000 - 4,000)

To me, these are the types of numbers you take to management not only to show the ROI on perl (and maybe convince them to send a bit of that savings to the Perl Foundation) but more importantly to show the ROI on you (and maybe convince them to spend a bit of that savings on you).

[aufflick]

Translating runtime code from one environment to another using XSLT - that's crazy talk!!