Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Reactionary Coding—One-Shot Programs

by John M. Dlugosz (Monsignor)
on Jun 13, 2001 at 22:36 UTC ( #88152=perlmeditation: print w/replies, xml ) Need Help??

Yesterday I was asked if I knew of a tool to do something that was a lot of work to do by hand (arranging a huge number of files onto several properly-sized CD images to offload them). I said, “Easy enough to do in Perl.” and helped him through it.

However, as a software engineer I normally think about robust solutions and self-contained shipping products. Furthermore, Perl has evolved toward programming in the large, and those are the techniques I normally use.

So, this Meditation is about what I noticed in breaking with that tendency, and how writing a special-purpose program that will run once is fundimentally different. This is becoming more forign to us as software becomes more of an engineering discipline.

I call this Reactionary as a double entendre: it is a reaction (solution) to a specific problem, and is seeking to undo our revolutions in techniques.

So, the first thing I noticed concerned the need to iterate over all the filenames in the source directory tree. Do I teach him to use File::Find or otherwise write some other sub to do that, and take command-line arguments to specify the starting directory and options? No! I generated the list using dir /fsb on my 4DOS/NT Shell and gave him the list. He doesn't have that shell, but he can use this file of fully-quallified file names as input with the magical <> operator.

That's what got me thinking that this is different from normal programming. Programs tend to become more complex with time, so we try to make it general in the first place, with an overall architecture that can handle unexpected changes. Not providing a way to change my input parameter is quite contrary to that!

Likewise, robustness is the next thing to go out the window. Make sure the destination directory exists? Who cares—if it doesn't work I'll run it again, and once it runs once, I don't need it anymore.

However, my friend caught me thinking large on another step. Changing the source name to the dest name, I was going to pull out the proper File:: module to break it into parts, and figure out the relative path of interest. He reminded me that we know the source path as a constant in the program! A simple s/// to change the first part of the name is sufficient. In a “real” program this would give me the willies. But for solving a specific problem, it's simple and correct and hard to get wrong and won't need debugging.

So, I was reminded through this exersize that I don't always have to write reusable, robust, code. Sometimes a brittle highly specific solution is better, and there is a lot of mentality to overcome in order to write in that style.

But, I still use strict and so on. Hard-coding things and assuming things about the run is not the same as writing a golf-ish one-liner. They are different concepts, and can be applied independantly. Code well, just allow assumptions about the specific run to afect your code.

Happy Coding,

Replies are listed 'Best First'.
Re: Reactionary Coding—One-Shot Programs
by Sifmole (Chaplain) on Jun 13, 2001 at 22:54 UTC
    The problem that seems to inevitable come up, at least for me, is that once I do a "cool little on-off-er" it seems to be desired more than once. "Hey, I heard you did this little tool for so-and-so. I have to do ALMOST exactly the same thing, but need to point here....", then another, and another......

    So what was once intended to be a one-off script gets a life of its own without my intent. Then I have to go back and start making it a more real critter.

Re: Reactionary Coding—One-Shot Programs
by dragonchild (Archbishop) on Jun 13, 2001 at 23:06 UTC
    I would say that if you have a problem that comes up as a one-off-type problem, code a one-off-type solution. But, save that solution somewhere. Maybe, you play with it and hack it into a more general solution. Maybe not. But, you save it in your personal scripts directory with some notes as to what it was used for.

    Six months later, someone will come up and ask for a slight modification. Then, when there's two times you want to run it, make it a generic mini-application.

    Once is a one-off ... twice is a mini-app ... three times is a solution. :-)

      I agree. It might get "reused" in that having done that before I can write another similar one real quick. Or, if it's very similar, I can dig up the script and edit it, changing the names and such. Just because it's specialized doesn't mean it has to be bad code. So source and dest names, etc. can be noted as constants at the top of the script, and assumptions can be documented as comments.


Re (tilly) 1: Reactionary Coding-One-Shot Programs
by tilly (Archbishop) on Jun 14, 2001 at 04:13 UTC
    There are differences, but one piece of advice.

    Software of all sizes has bugs. For one-offs it is very important to have a backup of whatever you are working on. If you get it wrong, just make another copy and try again until you are happy.

    Just the confidence of knowing that you can try again makes a world of difference...

Re: Reactionary Coding-One-Shot Programs
by xphase_work (Pilgrim) on Jun 13, 2001 at 23:16 UTC
    My job consists of a mixture of both methods of writing perl. I write scripts that edit ada code, generate reports from information within the ada code, and generate reports from output of other programs. The normal development phase of perl scripts here is that the software leads realize they need something, ask for a script that generates the information that they need, and then expect the reports(or edits) done as quickly as possible.

    What I try to do is write a one-shot that does what they want, but is open enough so that I can then use it as a base for a complete program that will be ready when they realize that they needed it more then once.

    I find that this approach works well because it keeps the requesting lead happy, helps make maintaining the code easier, and it allows each lead to use it.

    The only time that I don't write code that I can easly translate into a full program is when it will only be used as a one-liner called from a shell script with perl -e.


Re: Reactionary Coding—One-Shot Programs
by BooK (Curate) on Jun 14, 2001 at 20:05 UTC

    I do from time to time code one-shot programs. Perl is the perfect language for this, as it gives me powerful functions to do what I need (whether it's changing a few lines, calculate a string transformation or extract useful lines).

    As others noted, the problem is in how will these tools evolve over time... But you get to solve your present problem now. If you discover it's a larger problem, you can always go back to the keyboard and start to think about a more general purpose program.

    For example, here is a one-shot done this afternoon in a quick session, that is aimed to extract lines from a special text export of FireWall-1 logs... The only goal was to be a little faster than the FireWall-1 log tool for problem investigation. And to use simple (and regular) expressions to match lines.

    #!/usr/bin/perl -w use strict; # the various fields my @field = qw/id date time if fw type action service src dst proto ru +le sport reason/; my $i = 0; my %field = map { ($_, $i++) } @field; # create the filter my %filtre = (@ARGV); my @filtre; while(my ($k, $v) = each %filtre) { # better use a while than a for lo +op push @filtre, '$data['.$field{$k}."]=~m/$v/i"; } my $filtre = join ' && ', @filtre; # open the file my $file = "fw01.log"; open F, $file || die "Error: $file $!\n"; $\= "\n"; print join"--\t--", @field; while(<F>) { my @data = (/"(.*?)"/g); print join"\t",@data if eval $filtre; }

    It has -w and use strict;, but the filename is hardcoded and the file is not closed! I translated the comments from French, but they were here from the start. And the most important ones are missing... because the filter creation process was clear to me!

    This ugly script allows for nice combinations, like: src action drop service "^23|telnet" which will show all telnet connections to that were dropped by the firewall.

    If I had to write a bigger and more general purpose script, I'd probably use closures to create filter subroutines...

    Update: Quick and dirty scripts can be very ugly... This one for example had a bug that prevents you to use several conditions... each should not be used in a for, but in a while loop.

Re: Reactionary Coding—One-Shot Ugly Programs that do the trick
by stuffy (Monk) on Jun 14, 2001 at 05:24 UTC
    I recently wrote a (one shot) program at work. I am not a programmer, but this program does make life a little easier. I showed it to my boss to see what he thinks, and he mentioned that other groups will probably want to use something like this too. My problem is that I don't have the skill necessary to write a program such as that to be easily changed to work with the formats that other groups use. I would have to basically change all my If statments. My point, I would probably make all my scripts easlily useable in other areas if I knew what I was doing. But I guess that will come with time, right now I'm happy just getting my programs to work the way I want them to, even if they are ugly.

    That's my story, and I'm sticking to it, unless I'm wrong in which case I will probably change it ;~)
Re: Reactionary Coding?One-Shot Programs
by the_slycer (Chaplain) on Jun 14, 2001 at 08:04 UTC
    I very rarely code once run scripts.
    Rather, I very rarely code as though the scripts will be run once (as most of my scripts are run only once).
    Hrm, let me clarify. For example, in a script like the above I would probably start with the absolute path, get it working using that, then switch that to $ARGV[0]. Why? So that the next time that I need to write something similar, though not quite exactly the same, it's there for me with very little modification.

    This is of course more do to laziness. Code always gets reused in my (small) experience. Even if it's not the whole thing, at least you've got something you can snag bits out of.
Re: Reactionary Coding?One-Shot Programs
by ariels (Curate) on Jun 17, 2001 at 15:56 UTC
    The "extreme" (no pun or reference intended!) example of this would be writing "one-liners" -- running perl -[np] -e ... from your command line. I use that for massaging data files and for extracting simple statistics. Sure, I could modularise the statistics and write it just once, but looking up the exact interface for a counter-that-also-keeps-average-value will take longer than recoding it. Maintainability isn't an issue here: if you're asking a question about a log file in order to find out if it's an interesting question at all, you're most likely going to decide you don't care about the answer.

    Similarly, we sometimes run programs for days on end, only to discover a small bug in output. We'll fix the bugs, no question, but we don't want to lose the CPU time. If a small script can fix the output, we want it! (However, as tilly points out above, you can only do this if you've a backup; perl -pi.bk -e ... is invaluable here). The "patch" code will never run again, as the bugs in the program have been fixed -- it only exists to correct a single output file!

    This is bioinformatics: some of our files are >1GB in size, so text editors are not an option; it's either sed, awk or Perl.

    In both cases, you want to write code iteratively (the first 5 versions never work), run it once, and throw it away.

    Of course, I always hide away a copy of the script in my home directory; I never know when I'll want to steal some stuff from an old script.

      I think there could be an intermediate form, or something for "one line programmers in training". That is, a small editor window or a button on your general editor to say "run it NOW" without having to make up a file name and save it, but still be able to easily edit and retry.

      Perhaps a perl script that when run from the command shell, pops up this window and saves the stuff in a temp file implicitly; output when 'run' goes to the same shell window (and context) that you started from. Run it again and you get a window that remembers the contents from last time, so you can alter and try again. Maybe I'll play around with that and post it if I come up with something worth keeping.


Re: Reactionary Coding—One-Shot Programs
by sierrathedog04 (Hermit) on Jun 15, 2001 at 22:53 UTC
    Optimally arranging n files onto m CDs so as to minimize wasted space and preserve certain file dependencies is a classic zero-one linear programming/simplex problem. Don't write one-shot logic to do it. Use CPAN's PDL::Opt::Simplex module instead.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://88152]
Approved by root
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (7)
As of 2023-06-01 10:20 GMT
Find Nodes?
    Voting Booth?

    No recent polls found