Especially true in the web environment, there are those rare times when your boss/manager comes to you and asks to implement a piece of important something in a short space of time, approximately as much time as that normally allotted to a secret FBI agent to annihilate a criminal cell – normally a venture that takes from somewhere around 12 to 24 hours.

However, programming is certainly no secret FBI bust. It’s a unique endeavor of it’s own kind, and I believe that at one point in your career as a professional (or not so :) Perl hacker, you’ve come to appreciate the glory and perils that come with it. As the author of this post Rapid Rate of Development points out, we are lucky to have Perl at our disposal as it truly is a language well suited for ‘rapid development’.

And as the saying goes: “When times are tough, Perl gets going!” (for the record, I made this quote up ;-]). Just the other day I had a business development/project manager guy come to my cubicle and ask me for a ‘favor’. In brief, he outlined that there was a need for a script/mechanism that would allow them to do a job that would normally take much longer to accomplish and at a significant cost to the company (around $150,000 as per his estimate). Seeing as the job could be accomplished with ‘relative’ ease in Perl, I gave him an affirmative response.

However, in the back of my mind I was beginning to get a little nervous. My main concern was whether I could finish off the scripting within 12 hours. Also note that I had had already quite a number of projects/tasks piled up on my desk that also required my attention (at my company, the rate of projects per developer has increased somewhat over the past year as we had to go through some budget freeze – and a few cuts – and couldn’t hire any additional developers). This only complicated things, as I had to accomplish task in a near simultaneous fashion.

So, the only reasonable approach I could take is calm down and get to the development. In the 5 hours time frame, I managed to come up with 500-600 lines of code that seemed to do the job. Although, I was told that this could be a ‘throw-away’ script, I had to exercise due diligence to design and implement the code right as it had to run for at least a full-week non-stop to complete the job. This development was a combination of defensive coding and extreme programming. However, I didn’t have much time to spend on proper documentation and thorough design analysis as I would wish to. In the end, I was delighted that the final cut seemed to run smoothly and do the job right (actually, I left the script running for the night and will spend some time today to gauge it’s performance). Yet, while my management seem to be ecstatic about the whole thing, I feel a little burned out and exhausted (I still have to do a lot of coding for other projects).

Did you have similar moments in your career as a Perl hacker? It would be interesting for us (me in particular :) to know “just how did you cope?”. Is it reasonable to turn away such requests or pursue them with vigor?

I thank you for your participation in advance! ;)

$"=q;grep;;$,=q"grep";for(`find . -name ".saves*~"`){s;$/;;;/(.*-(\d+) +-.*)$/; $_=<a HREF="/"> "," $2 </a>;`@$_ +`?{print"+ $1"}:{print"- $1"}&&`rm $1`; print$\;}

Replies are listed 'Best First'.
Re: Development at the speed of thought.
by atcroft (Abbot) on Jun 07, 2002 at 15:43 UTC

    Sorry, but several things you mentioned shot up very large, very loud, very gawdy red warning flags for me.

    • He had a dollar estimate. This is something that has been done before, and likely will be done again, too.
    • "would normally take much longer to accomplish" - Again, a process that is going to be done again, most likely.
    • It "could be a 'throw-away' script" - how many times have we heard that, only to see them still in use much beyond their expected lifetime?
    • You mentioned freezes and cuts over the past year, and having to severely multi-project.
    Given the above, I don't see this "throw-away" script getting tossed out anytime soon. From the latter of the points above, I would guess it may be sometime before you can clear enough from your plate to rework it, and by the time you get to the point of suggesting it be reworked, I expect you may very well hear, "well, it's worked okay so far-why put the time/cost into redoing something that already works?"

    As to being called upon to develop solutions/tools "in three minutes or we're all dead" (to quote Adm. Kirk in STII:TWOK), I think we all get called upon to do that, whether we like that kind of pressure or not. I applaude that you were able to maintain good techniques while under this kind of time pressure.

    As to how I cope, I considered two examples:

    • Recently, because of unplanned equipment changes, I had to rework a system for managing a service on a machine to handle the service being split between multiple machines, and managed to do so in about 14 hours (after being awake all day)... although the only bug I missed took 2 days to catch because of circumstances necessary for it to happen.
    • A few years ago, I had to develop an alternative system for managing users in a service, which I managed in about 10-11 hours at the time (I was still very much a novice then).
    After both, I was mentally exhausted and slightly out of it, and so tried to catch up on sleep and recharge what mind I had left.

    As to the question, "is it reasonable to turn away such requests," I believe it is reasonable to turn it away or possibly suggest it be held, if possible, when the goal is too unreasonable, or when the project has too little solidity to have any direction, or when to drop what you are in to jump into that would be of more detrimental than the outcome is worth.

    (And yes, I follow Mr. Scott's example about over-estimating the time required for something, although maybe not to his factor of 4. Still would like the neural interface Mr. Barkley had in one of the ST:TNG episodes, though.)

    Just the $0.02 of an egg (which, when you factor in inflation, may not be worth that much). I look forward to the other comments on this thread.

Re: Development at the speed of thought.
by dws (Chancellor) on Jun 07, 2002 at 19:43 UTC
    Did you have similar moments in your career as a Perl hacker? It would be interesting for us to know “just how did you cope?”. Is it reasonable to turn away such requests or pursue them with vigor?

    If you're competent you can expect a lot of "special" requests throughout your career. Pulling one off can be great for visibility, but the results can be a mixed bag for a couple of reasons:

    • If the company starts to see you as the person to go to in emergencies of a certain type, they may be reluctant to let you out of the box you're in.
    • If the company has a "sticky fingers" theory of code ownership, having your fingerprints on a bunch of solutions means you're stuck supporting them.
    • An occassional firefight is fun, but a string of them can be a real pain if you're trying to get other stuff done. And you boss can get really unhappy when his/her objectives take a hit because you're servicing other special requests.

    Having survived many "special" requests, here's the approach I've settled on:

    Keep Your Boss in the Loop. If you go off your boss's radar, you risk making him/her look bad. It can be really embarrasing to be a manager in a meeting with other managers and your boss, and have it come out that one of your folks is working on something you don't know about. It can make you look like an incompetent. So keep your boss in the loop. A boss who is in the loop may shut down the request, perhaps for reasons you don't have visibility on. That's life.

    Let People Know the Tradeoffs. Once you have an idea of the effort to satisfy the special request, let your boss know. I've found that it especially helps to phrase it in terms of "X, Y, and Z" will be delayed for N days while I do this," since it reminds a (possibly busy and distracted) boss that your time isn't free. And let people who're depending on you for X, Y, and Z know. This gives them the opportunity to escalate, rather than blindsiding them. You may be upstream of stuff that's more important than you realize.

    Leave a Solution that Doesn't Depend on You. To avoid perpetual maintenance resposibility, work the system to have people designated to take over your solution. Train them if necessary, and make sure you're leaving them with documentation that is adequate for them. Sometimes, sadly, this means not doing something in Perl when nobody else knows Perl. :(

    Credit Where Credit is Due. When you do pull a rabbit out of your hat, take care to visibly thank the people who help (e.g., email to their manager(s), cc'ing them.) This keeps you from looking like a prima donna, helps encourage helpfulness, and gives visibility to others who deserve it. And what goes around comes around. I've gotten great support later from people who I've taken extra steps to acknowledge.

    Write an "After Action" Report. To help keep corporate memory from forgetting the past, write a short memo explaining what you were asked to do, how you did it, how long it took, what the tradeoffs were (i.e., what projects slipped or took a hit), and what could have been done to avoid the problem in the first place. Try to be neutral and avoid finger pointing. (Think of this as something that you'd like the boss to pull out of their files when review time comes.) Get this report to your boss, and, if politically possible, to their boss. Here's a good place to thank other people who were helpful.

      Brilliant list.. ++.

      There's one thing I'd add to your "Let people know the tradeoffs" phase, though: Make the requestor handle the negotiations with the people whose projects are being bumped.

      'Special' projects are often a symptom of bad communication among management. Officially, management is supposed to allocate resources (including time and labor), and everyone everyone is supposed to buy into the same plan. Unfortunately, it doesn't always work that way. Personal and political issues can keep a group of managers from reaching an agreement, and when that happens, the things they can't agree on tend to get shoved under the rug.

      Asking developers to do 'special' projects is one way to shove things under the rug. In effect, a 'special' project can force the developer to make decisions that management was either unwilling or unable to.

      That's bad news.

      First of all, the developer doesn't officially have the authority or information to make those kinds of decisions. Then there's the fact that doing the project might put some other manager's nose out of joint. Remember, if all of management was behind the project, it would come to you as an assignment from your own boss.

      So don't let managers use you as the scapegoat in the cases where they can't do their own jobs. Make them do all the legwork and negotiations necessary to buy you the time and resources you need. That's what they get paid for.

      If the project is legit, the requestor will be able to buy you whatever time and resources you need, and everyone will be happy. Everyone will feel involved, and a lot of peripheral work will stay off of your shoulders. Having to get permission to cut in front of everyone else in your work queue will also keep people from deciding that you're a convenient source of free labor. Everyone involved will be reminded that lots of people want your time. That will cut down on frivolous requests and 'Spanish Inquisition' pressure tactics (having the client in the room or on the phone when making the request, so you can't possibly say no).

      'Write an "After Action" Report'
      sounds an awful lot like an "After Action Review" or AAR as the U.S. Army is so fond of calling them.
      * Mr. Muskrat thinks that either:
      1) you are from a military background, or
      2) one of your teachers was.

      Who says that programmers can't work in the Marketing Department?
      Or is that who says that Marketing people can't program?
Re: Development at the speed of thought.
by BlueBlazerRegular (Friar) on Jun 07, 2002 at 18:40 UTC
    atcroft has it right. I would add one other item - you mentioned:

    I had a business development/project manager guy come to my cubicle and ask me for a ‘favor’

    Watch out for this. Having people who aren't in your chain of command give you work that negatively impacts your progress on your 'official' work will most likely upset your boss. And when your boss gets upset - well, if you are lucky, he yells at the favor asker, if your aren't lucky, he yells at you.

    And what do you get from the 'favor asker'? A heartfelt thanks (maybe) and a reputation as being an IT guy who can handle these little jobs. So they come and ask for help on this new thing they are working on. Worse yet, they tell their friends, who also now come to you. Of course, one of the reasons they are coming to you is that IT has told them that it'll take 6 months before they get their report due to the layoffs, hiring freezes, and not coincidently - the fact that half your time is now spent working on these side issues.

    And since Perl is a great tool for these little programs (or not so little programs), the Perl programmer gets the reputation (for good or ill).

    One way to handle these things is to make them go through your boss for everything. Say that you're swamped with these other tasks (which you are) and that everything now has to go through him/her. You'll need your boss's agreement (which they should be happy to give, since it gets you back to working on the real work). And it may upset a few, but remember who does your evaluations and determines raises...

      One way to handle these things is to make them go through your boss for everything.

      I had this kind of problem where I used to work. I was one of the managers in question and we told our staff to do exactly that. What happened was we had two types of staff. Those that were too timid to say no and those that "liked to impress" and would say yes to everything anyway. It was hard work trying to beat off other department heads to try and get control of the dapartment. We would have them activly sneeking in when you were out of the room in order to give your staff work.

      ---If it doesn't fit use a bigger hammer
Re: Development at the speed of thought.
by rattusillegitimus (Friar) on Jun 07, 2002 at 18:29 UTC

    Hrm, shades of my past life...

    I've had a few similar moments in my career (though they're mercifully few these days). I've never had the luxury of doing anything but pursuing the requests with vigor (and sleep-deprivation), though. As one of the two main developers of our company's product, I once had to fly down to the office (I telecommuted from another state at the time) and spend several days only taking 1-2 hours off each day to sleep, eat, or shower. We were handed a sudden deadline to demo the product to the new CEO, and almost none of the features he'd been promised were ready when we started. We got most of it done, albeit poorly in many cases, and coped by bringing beer to the demo. When they woke me up after the demo, they said the CEO had been pleased. ;)

    My problem with such requests has always been that the requestor never seems to get it through his/her pointy head that there is often a very big difference between doing something impossibly fast, and doing it right. This is doubly infuriating when the time crunch is caused by a marketron telling a potential client that a feature the developers have never heard of is a standard part of the software.

    Fortunately for all involved, my relatively strong work ethic has always made me chunk out the solutions in the nick of time. And despite my low frustration tolerance, I've managed to avoid doing or saying anything drastic thus far.

    In the end, I usually took a little time at the beginning to get a handle on the best approach to the problem, then cried havoc! and let slip the dogs of war. And by giving up some of my free or sleep time, I usually managed to get a decent solution written, or one that had the hooks in it to be converted to something more broadly useful.

    - rattus, random hairballs from the nest

      We had similar experiences at my last job. We used to have a salesman who did like to sell stuff we didnt have, requiring us the developers to produces it out of thin air. One particular case was the legendary "Currency Converter" The final instant product was a bit of perl and javascript that pinched the currency rates from a less than official exchange rate page that we found. The converter still exists over 4 years later. It used to update the rates automaticaly, but the source site changed their format and it broke. Now it just gets updated when someone remembers.
      And what happened to the saleman in question? Well he finished up as sales director of course. :-}

      ---If it doesn't fit use a bigger hammer
        I used to work at the same place too but didn't get hit by that one as much. It seemed my job description detailed my responsibility to deliver prototypes as core business functions. You know the one. . . . here Simon, see if this works. . . . . thats nice, add a login to it cos I've promised a big client it will work by tomorrow afternoon.

        Always a laugh. Almost as much as the old - give the team work without telling me and then drop me in it with the M.D cos I'm not keeping your promises - trick :)

        Oh and of course, theres the one where a different manager walztes into your office and demands to know why your prototype (a.k.a business solution) is so cr*p and proceeds to attempt to berate you in front of your co-workers. Naturally you are receptive to this, even though he managed to wipe your new design off a meeting room whiteboard that you and the design team worked till 2am on and its STILL NOT his fault :)

        Or maybe I'm just bitter :P
Re: Development at the speed of thought.
by cjf (Parson) on Jun 07, 2002 at 14:49 UTC
    I had to exercise due diligence to design and implement the code right as it had to run for at least a full-week non-stop to complete the job.

    An entire week? That doesn't sound right. Are you sure Perl was the best choice for the job? Could an extra couple hours of optimizing have saved a couple days of execution time?

    Is it reasonable to turn away such requests or pursue them with vigor?

    Just make sure they know the disadvantages. Security, reliability, maintainability, and many other things are reduced by rushed development. Sometimes it's worth it, sometimes it's not.

      Unfortunately no, there are factors outside of the script that come into play. Althought, I can't disclose a lot about the particular project, but the script 'analyzes' certain websites and indexes data in the database (in a dozen specially designed tables).

      I agree with you on that rapid development may leave a number of holes uncovered. Althought, this being a 'throw-away' script (ment to run once and may be easily discarded once the job is done), any possibility of a break in security etc. is minimized ;). However, I too feel somewhat uneasy whenever I have to do a job like that ;/.

      $"=q;grep;;$,=q"grep";for(`find . -name ".saves*~"`){s;$/;;;/(.*-(\d+) +-.*)$/; $_=["ps -e -o pid | "," $2 | "," -v "," "];`@$_`?{print"+ $1"}:{print" +- $1"}&&`rm $1`; print$\;}
        Althought, this being a 'throw-away' script

        In my experience there's no such thing as a throw-away script. They always find a way to stick around until long after they were supposed to be discarded. A little extra planning and testing can save you a lot of time down the road (consider why you use strict).

        any possibility of a break in security etc. is minimized

        A security policy requiring basic security practices for all scripts should be followed even when deadlines are rushed. Placing a CGI script online without basic measures like taint checking (yes, that's an oversimplification, I know) should not be allowed to happen. Security is a tradeoff, but basic checks are almost always worth it.

        Althought, this being a 'throw-away' script (ment to run once and may be easily discarded once the job is done)

        ROTFLMAO :-) Please, Please, Please return to this thread and follow up when management inevitably asks you to add to, modify, or otherwise maintain this "throw-away" script. I'll be conservative and bet on 3 months or less, but I wouldn't be surprised if it happened before the first week-long run is even over :-)