Old NumberCrunching Modules

[Anonymous Monk]

I was thinking about asking yesterday, and ran across this "elitist" posting which bothered me.

http://www.perlmonks.org/?node_id=986846

I don't have anything in CPAN (yet), but I've been writing perl since the early 1980's. I even did a partial port of perl to QNX-2.x which I never passed on (it was just commenting out all the networking). Twenty five lines to a function, I've had many functions run to 10 or more pages. I'll chalk that up to engineering versus computer science, but others may disagree. And it has little to do with the topic anyway.

Math::Brent has been around a long time (1995 or so), and it apparently works just like Brent's method for minimization does in every other implementation. The code looks like a perl version of FORTRAN-II, but so does Numerical Recipes (pick your language, they all look like FORTRAN_II to me). And seeing as I grew up with FORTRAN_IV, looking at FORTRAN_II (in Perl, C, Pascal or whatever) isn't a problem.

There have to be lots of potential Perl modules implementing old FORTRAN code, and the biggest problem with most of these is getting the documentation correct. The code (if implemented properly) just works.

I run Debian Linux, and there is the concept of a non-maintainer upload. There has been a bug filed against Math::Brent for 4 years, which is correct. There is a documentation error in Math::Brent, and to fix it requires the deletion of 4 characters in the POD. Does some similar mechanism exist in CPAN?

I am working on a module (or set) to analyze what is nominally Map data produced by GPS, but could be things like surface chemistry as a function of position.

Part of what I need to do is Cross Validation, and I don't see how I can make use of Math::Brent in its non-OO form. Consequently I am re-implementing Math::Brent in an object oriented form. Having read about non-derivative optimization of 1D functions a bit, I am thinking about a modification (thanks to Crenshaw).

Sorry, too long.

Whenever I get this Map thing working, I could easily make an update to Math::Brent to fix the documentation, which would get rid of a 4 year old bug. Is there some accepted practice about re-implementing functional code as OO? Math::Brent::OO? If I make a change to how Brent works, what should a person do? Option to Math::Brent::OO->new(), or a new module?

The idea of changing Brent, a person needs to test. There are many modules in Test::, I don't see any involved with verifying how floating point code works on different architectures. I have a bunch of test functions and results assembled. Suggestions?

[tobyink]

Yes, it's possible for you to upload a new version of Math::Brent to CPAN. It will be marked as an unauthorized version though, and will not be indexed. This means that although it will become part of CPAN, and carried by all CPAN mirrors, the cpan command-line client will not find it. (It can still be installed via the command-line client; you just need to provide the full path to the file, a la install JOEBLOGGS/Math-Brent-1.234.tar.gz.)

However, if you're making large scale changes to it, such that your version and the original have incompatible APIs, then it serves end users better for your module to have a different name.

[davido]

Attempt to contact the author (likely to fail, since the module is sooooo old). Contact the PAUSE folks to request a transfer of ownership or maintainership. If it needs to be on CPAN now, upload an unauthorized version (bump the version number too). Once maintainership is granted, bump the version again and upload (so that it gets indexed).

Your OO module is a new beast entirely, and deserves its own name. Math::Brent::OO or Math::Brent::Class, possibly. That way you don't break an existing module (and unlikely as it may be, a dependency on that module) by converting it from functions to an OO interface. The [e-mail] mailing list is a good place to vet module names. Be sure in your new module to document clearly how it differs from the old one; Simpler OO interface, more thorough test suite, portability as a design goal, etc.

[jgamble]

I agree that you need to acquire ownership if the original author is unavailable now. Double-check that by posting to [e-mail] mailing list to make certain.

Then fix the bug and documentation error before re-writing and re-factoring code. It's only one extra version upload, and it lets the module's users know that you are making changes and lets them prepare for those changes.

I also have ex-Fortran code in Perl, and the sensible process there is to make the basic, obvious changes (change loops to use the '..' operator; fix variable names to something meaningful instead of single letters; lexicalize function variables that are only found within an individual loop down to that loop).

(Looking at Math-Brent's code, it looks like the original author already did some of that, although it seems in some cases he has multiple variables in play where an array would be more useful).

Then consider OO-ing the code.

[Athanasius]

Sorry, but this:

I've had many functions run to 10 or more pages. I'll chalk that up to engineering versus computer science, but others may disagree.

touched a nerve. So:

<rant>

I used to work for a company offering streaming video to hand-held devices. I worked mostly on the codec, but at one point I was given the job of refactoring the top-level code of the video player.

This code was mostly a single function stretching over several pages, and had been written and maintained by a single programmer. Took me a while to figure out what it was doing, but I was able to re-factor it into reasonably-sized functions. And in the process I discovered redundancies which the author hadn’t noticed. (And no-one else would be likely to notice them, since no-one else could really follow his code!)

Now to the point: What has any of this to do with “engineering versus computer science”? Who would rate a civil engineer who designs a bridge to carry the needed loads but uses materials which wear out in a few months? (Except, of course, in special circumstances, such as in wartime.) Well then, why would any software engineer prefer monolithic spaghetti-code to modular code which is clear, maintainable, and extensible? I confess — I just don’t get it.

In my opinion, the whole engineering vs. computer science contest is a myth. The two sides come at problems from different angles, but as a rule they end up in the same place: namely, the realm of good craftsmanship and common sense.

</rant>

Just my 2¢.

[davido]

I can't agree more with the tirade against monolithic functions that span several pages. A few months ago I was working on refactoring some code where many functions reached or exceeded around 200 lines. And as I began refactoring it was amazing how many places I found that I could generalize what five to fifteen lines were doing in a way that could be reused across several of those functions. The end result was fewer total lines, no functions beyond a single page, and in my opinion, easier to read and maintain (which was my motivation behind refactoring it -- I needed to maintain it and found it to be unmanageable without this sort of sprucing up).

My inspiration was derived from Schwern's Skimmable Code talk (PDF... sorry). I wanted to follow-up to your post to get that link out in the open once again; it's a great resource.

[Tux]

But refactoring for the sake of refactoring - or just to get a shorter method/function body can easily led to just the oposite, as I have sadly seen way too often.

A perfectly sane 400 line function/method stuffed with comment of a process that was a sequential series of logical steps that only got interrupted by serious Exceptions/dies was broken into several parts shattered across modules without proper call-checks and, worse, rewritten in an understanding that was not true, thereby killing the whole process.

Some people say they understand code better when it is rewritten into smaller chunks. That might (very) well be true if any of those chunks is to be reused or makes a perfectly sane unit. If the code however is not (ever) to be shared or reused by other parts, refactoring just because of size most of the times leads to serious errors. At least in my experience.

.

[jgamble]

... And now that I think of it, rather than writing a stand-alone OO module "Math::Brent::OO" as suggested above, it probably would make more sense to write a sub-class of Math::Polynomial. There's even instructions on how to do it.

So for the sake of the Math::Brent users, just fix the bugs there, and create separately Math::Polynomial::Brent, with methods find_minima(), bracket_minimum(), brent(), and minimise_1d() (yeah, I de-camelcased the function names).

If it sounds like I'm dumping work on you, I'd be happy to help out and write code. Let me know via [e-mail].