Re^3: Thorny design problem

[Tanktalus]

I'd be very careful about this. Because if you look for similarities, there likely will be many. I realise that the idiosyncracies probably outweigh the similarities, but even encapsulating those few similarities (which, of course, won't be similar among all companies, but probably a significant subset will share one similarity while another subset shares another similarity, etc.) can pay dividends.

For example, what I have done in a similar (yet completely dissimilar) situation ... where I have a bunch of tasks to perform, each one is almost completely unique from the other, and I have a task manager object that reorders the tasks (order can be important for me - not likely for you, but the manager could do other things to help, such as figure out that some companies have been updated recently enough to skip, or something) and then executes them serially.

The manager object is actually almost completely uninteresting here.

package TaskManager;

use Task;

sub new {
  # create.
  # receive task objects or task names, store them via add_task.
}

sub add_task {
  # receive task objects or task names
  my $self = shift;
  push @{$self->{_tasks}}, map {
    ref $_ ? $_ : Task::find_task($_)
  } @_;
}

sub execute_tasks {
  my $self = shift;

  # do we need to do them?
  my @tasks = grep {
    $_->needed()
  } @{$self->{_tasks}}

  for my $t (@tasks)
  {
    $t->perform_task();
  }
}

1;

For the most part, that's it. I skipped the sorting part, for example. And strict/warnings - they're in the real module. And then, the Task module looks like this:

package Task;

sub new {
  # creation of generic object.
}

my %task_cache;
sub find_task {
  my $task = shift;
  $task = shift if $task eq __PACKAGE__; # can call as Task-> or Task:
+:

  return $task_cache{$task} if $task_cache{$task};

  (my $mod_name = $task . '.pm') =~ s{::}{/};
  eval { require $mod_name };
  if ($@) { 
    # handle error - return error, die, whatever.
  }
  $task_cache{$task} = $mod_name->new(@_);
}

sub needed
{
  #default - yes.
  1
}

sub perform_task
{
  # default... can't.
  my $self = shift;
  die "Forgot to override perform_task in " . ref($self) . "\n";
}

1;

Now I didn't really show you how to reuse stuff. But what I've started here is a framework. A framework upon which you can build further frameworks. For example:

package Task::WWW;

use WWW::Mechanize;

sub perform_task
{
  my $self = shift;
  # set up proxy, ...
  my $www = WWW::Mechanize->new($self->get_hostname());
  $self->handle_page($www);
}
1;

Now you just need to derive your HTTP-gathering info from Task::WWW, override two simpler functions, and can rest assured that if your application needs to change proxies for http requests, they all go through this code and can be easily modified. I realise that LWP::Simple probably uses environment variables for this, but it's just an example. I'm sure the more imaginative out there can find another use for this type of abstraction. Perhaps you now need to authenticate to get out of your company intranet. Or something equally silly. Whatever. By abstracting out the common parts, you not only can save time in adding new requirements like that, but you can concentrate on smaller pieces of the overall puzzle at a time in each function in each module.

You'll have well over 100 modules this way, but each one will be simpler.

(I think I need to formalise this and put it on CPAN...)