in reply to TMTOWTDI... and most of them are wrong

The presumption here is that those languages that seek to restrict the idioms and modes of use to some subset of those possible, can and will force the programmer to produce more correct code with less bugs.

History does not support this presumption.

Whilst compile-time type-checking, declarative method and attribute visibility, compile-time bounds checking and a host of other compiler enforced restrictions will often allow many trivial source code and logic errors to be detected early, these are often the same errors that would be detected early anyway, whether through rudimentiary unit or system testing.

Some other diminutions of programmer responsibility can pragmatically reduce the scope for programmer error, especially in those areas that require the programmer to remember to perform certain, often rote, mechanical actions across functional boundaries. Eg. Memory management. Cleanup of memory is often called for at points in the code well away from the allocation of that memory. This is true for procedural code and even more so for OO code. This type of responsibility often requires a quite long-term overview of the flow of the code, and the life of the allocated objects. And it is often well divorced from their instanciation and the main points of use. These kinds of responsibilities benefits from the computers ability to perform rote actions reliably and rigourously. Hence, automated memory management (GC) is usually a big win.

However, many other areas of a programmers skills are less ameniable to automation, and in many cases, those types of compiler enforced restrictions can actually inhibit the programmer from pursuing and achieving elegant solutions.

For example: I've been trying to get to grips with FP languages. These mostly have a strong element of type safety. This is usually seen as a good thing, and when writing code that fits into "known algorithms", I would agree that it does, but there is a downside. It tends to also enforce a bottom-up approach to coding!

It is very difficult to write top-down code where, in the early stages, many elements of the solution one is seeking are unknown. With a relaxed language like Perl, it is very easy to "mock-up" the lower levels of a program as you don't have to specify the types, or numbers of the parameters of functions. Indeed, you don't even have to write the functions (methods) so long as they never actually get called. This can be a great advantage as it allows you to use a very pragmatic approach of "at this point in the code I need to determine if xyz() is true. So you simply code that. If the logic of the code means that xyz() never actually gets called, then you do not even have to code xyz().

This allows you to start with the top level premise of the code and, what basically amounts to , pseudo-code your solution, in a top-down manor. Often, you will never get around to coding xyz() because having mocked up the top level of the code and "proof-read" it (with perl -c prog), you realise that the logic of xyz() is better incorporated into pqr(); or that there is no way that you will have access to the parameters that you would require to determine xyz() at the point in the code where you coded it; or that actually there is no need to determine xyz(), because it's results naturally fall out of logic that precedes the point where you need it.

I arrived at a maxim about 15 years ago that has stood me in very good stead ever since:

Define top-down; Refine bottom-up.

Start with what you know: Available inputs and required outputs and define--as a function--what you need to transform the former to the latter. Within that, repeat the same strategy.Decide what you need to produce the return value and what forms your inputs at that level. And so on down.

The beauty of Perl, is that it makes doing this very easy. The absence of parameter and return typing mean that you can easily mock up a function that takes nothing and returns a (hardcoded) something that is exactly what the calling code needs.

sub doSomething{ return 'Whatever's needed'; }

This truely makes for rapid prototyping. Defining functions is so cheap that is encourages you to do so even if you are 95% sure that you will later discard it. And once you know you will keep the function, you can, slowly, refine the constraints that you impose upon the parameters, as those constraints become clear. And, if the application warrents it (ie. It's more than a one-time script, or the demonstration of the prototype incites further investment of time and/or money), then you can refine the constraints, bottom-up, to achieve the level of reliability commensurate with the application.

Of course, how much energy will get expended on that refinement depends upon two things: The thoroughness of the programmer; and the perceived value of the time expended.

When it comes to "trusting the programmer", it is just as possible to do only the minimum required to satisfy the compiler in an 'enforcing' language, as it is in a relaxed one like Perl, and the assumption that code that meets the minimal requirements of the enforcing language is "good enough", is a very bad assumption.

Even with the best will in the world, the programmer can just as easily believe that he has understood the requirements of the application, when s/he has actually misunderstood them, when using an enforcing language, as when using a relaxed language. Indeed, I would say that the enforcing language has the ability to delude both the programmer and his customer into believing that if it compiles, it must be correct. This is, and will always be, a bad assumption.

For the sake and sanity of both parties, the only thing that can be trusted, is an accurate specification and correct tests of complience with that specification. Any other criteria for judging the correctness of code will always be subject to debate, misinterpretation and misuse.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Replies are listed 'Best First'.
FP languages have a bottom-up bias?
by tmoertel (Chaplain) on Jun 27, 2005 at 18:10 UTC
    Regarding your claim about FP languages:
    [FP languages] mostly have a strong element of type safety. This is usually seen as a good thing [...], but there is a downside. It tends to also enforce a bottom-up approach to coding! It is very difficult to write top-down code where, in the early stages, many elements of the solution one is seeking are unknown.
    Are you extrapolating too much from personal experience? Because you find top-down coding in FP languages difficult doesn't mean that it is in general. For instance, I work almost exclusively top down when I code in Haskell, and it seems effortless. Could there be another explanation for your experience?

      Yep! There are several possible explainations I guess.

    • I've been reading the wrong "worked examples".
    • I (still) haven't done enough Haskell to have extablished a good approach to programming with it.

      Perhaps you could show me the steps you would take to tackling the sort of problem I have been trying to get to grips with?

      For the purpose of the exercise, I decided to index the words in a bunch of files. To get a start on the exercise, in Perl, I decided to just count the words, here simply defined as whitespace delimited groups of characters, and write the results to a file. This allows me to get something going and I can then work on refining the definition of a "word" and and decide how I am going to store the index.

      My first attempt at writing this program (from scratch, just now) was:

      #! perl -slw use strict; use G; ## Expands wildcards arguments as would be done by most *nix sh +ells. our $INDEX ||= 'words.index'; my %index; if( -e $INDEX ) { open INDEX, '<', $INDEX or die "$INDEX: $!"; m[^([^:]+):(\d+)$] and $index{ $1 } = $2 while <INDEX>; close INDEX; } while( my $file = @ARGV ) { open FILE, '<', $file or warn "$file:$!" and next; while( <FILE> ) { $index{ $_ }++ for split ' '; } close $file; } my( $key, $val ); open INDEX, '>', $INDEX or die "$INDEX : $!"; print INDEX "$key:$val" while ( $key, $val ) = each %index; close INDEX;

      I then ran a quick sanity check:

      P:\test>perl -c windex-1.pl windex-1.pl syntax OK

      So far, so good. Now try it out:

      P:\test>windex-1.pl *.pl 710:No such file or directory at P:\test\windex-1.pl line 15. 710:No such file or directory at P:\test\windex-1.pl line 15. 710:No such file or directory at P:\test\windex-1.pl line 15. 710:No such file or directory at P:\test\windex-1.pl line 15. ...

      Whoops! A quick look and I see I missed out the keyword pop in line 14:

      while( my $file = pop @ARGV ) {

      Try again:

      P:\test>windex-1.pl *.pl 306836-trietest.pl:Permission denied at P:\test\windex-1.pl line 15, < +FILE> line 83342.

      Okay, I've got 1 file with bad permissions in the directory. And what ended up in words.index?

      P:\test>type words.index expletive:2 $midVal:30 9f:8 proven:2 AT:10 happpen:2 inconsequential:2 perverted:2 $x;:24 TTGGGTCAGCGAATGTCACATTTGGAATGGGAATCGGATGATGGGGGCGA:2 ++$x):2 greif:2 yvvvvvvvvvvhvevwvvy:2 @exclude_keys:2 $hh2,:2 scrapped:2 classe:2 sod:2 $array[2]->set($_);:2 Cleaner,:2 $expand_ratio,:8 CACGTCTCCACCCGAAGACTGTGGAATGCTACTATTAAAATACTATTTTT:2 cdpr:2 BLANK_NODE:4 paws:6 "\n$_:2 ...

      Okay. The word definition is ecclectic, but is seems to be doing what I set out to do, now I can start to refine it. Not a bad start for 10 minutes coding.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
        Like I wrote in my earlier post, I almost always work top down. So, to take up your challenge, I will start with a top-level outline: read in the inputs, process them somehow, and then show the results. Since I don't know what it means (yet) to perform each of those tasks, I will use simple placeholders like id to represent each stage of the outline:
        module Main where main = interact $ showOutput . process . readInput showOutput = id process = id readInput = id
        This is legal Haskell that compiles and executes. It doesn't do much, however, merely echoing its input:
        $ echo alpha beta | ghc -e main WordIndex.hs
        alpha beta
        Still, it shows how easy it is to start sketching a solution at a high level.

        Next, working down, I will flesh out the outline. I won't worry about reading input from files or writing output to an index just yet. For now, I will focus on the core word-counting problem, adding logic to do just that:

        module Main where import qualified Data.Map as Map main = interact $ showOutput . process . readInput showOutput = unlines . Map.foldWithKey show1 [] process = foldl accumulate Map.empty readInput = words show1 key count = (unwords [key, show count] :) accumulate hash key = Map.insertWith (+) key 1 hash
        Notice that I did not need to alter the outline. All I did was add low-level detail to it. Now I have counts:
        $ echo alpha beta | ghc -e main WordIndex.hs
        alpha 1
        beta 1

        Continuing to refine the lower-level functionality, I will return to the I/O tasks. Like before, I will add lower-level functions, leaving the top-level outline largely unchanged:

        module Main (main) where import Control.Monad import qualified Data.Map as Map import System.Environment main = control $ showOutput . process . readInput showOutput = unlines . Map.foldWithKey show1 [] process = foldl accumulate Map.empty readInput = words show1 key count = (unwords [key, show count] :) accumulate hash key = Map.insertWith (+) key 1 hash control f = readFiles >>= writeIndex . f readFiles = getArgs >>= liftM concat . mapM readFile writeIndex = writeFile "words.index"

        Let me recap. I started with a top-level, executable outline and worked down from there. With two simple iterations I was able to add the counting functionality and then the I/O functionality. Now I have a complete solution that is largely equivalent to your code (ignoring details that don't matter for our top-down coding discussion).

        As a demonstration, I will compile the code and use it to index itself:

        $ ghc -O2 --make -o WordIndex WordIndex.hs
        $ ./WordIndex *.hs
        $ cat words.index
        "words.index" 1
        $ 1
        (+) 1
        (main) 1
        (unwords 1
        . 5
        1 1
        :) 1
        = 9
        >>= 2
        Control.Monad 1
        Data.Map 1
        Main 1
        Map 1
        Map.empty 1
        ...
        

        There you have it: top-down functional programming, or at least the way I do it in Haskell. I hope you will trust me that it was quick and easy. (It took much longer to write this explanation than the code.)

        Cheers,
        Tom