Re: Appropriate amount of abstraction

[Mutant]

As everyone has already said, it depends on whether you're interested in doing the job, or learning how something works. There's a good chance I'll never use assembler level code in my professional career, but learning how this lower level works gives me a deeper understanding of the higher levels.

The only case where you really need to use lower levels is for efficiecny reasons. The higher the level of abstraction, the worse the efficiency. However, on the occasions when you do require maximum efficiency (and these cases are far more rare than a lot of people think), then it's good to be able to drop into the lower level.

Another point, though, is that the more mature a level of abstraction is, the smarter it is about being efficient. Early 3GL languages were probably amazingly inefficient, which is why heavy use of inline assembler was necessary. This is now the case with things like Class::DBI. But maybe in 10 years time, we'll very rarely need to write SQL statements, and the DB abstraction layers will be smart enough to optimise even complex queries for us automatically.

The ultimate goal of Information Technology is to reach the highest level of abstraction possible. If this goal is ever achieved, then it won't just be SQL we won't need to worry about, it'll also be Perl.

[gunzip]

The ultimate goal of Information Technology is to reach the highest level of abstraction possible. If this goal is ever achieved, then it won't just be SQL we won't need to worry about, it'll also be Perl.

Then no-one will be able to understand how anything works. Layer upon layer of abstraction usually decreases efficiency and increases the number of different places you have to look to understand how something works. There is a law of diminishing returns at work and beyond a certain point you end-up with Java lego code, which is how many new students now enter the software industry. I got into Perl after rejecting this facet of Java but Perl can end-up the same if you keep adding more and more layers of abstraction.

[hardburn]

There are good and bad abstractions. Speaking purely in terms of computational efficiency, a good abstraction gives an optimizer more information about what you're trying to do. Optimization is basically about reconginzing that there are multiple ways of compling a given bit of code, and the Programmer shouldn't have to worry about what the best way is.

Speaking in terms of programmer efficiency, a good abstraction will allow the most work done with the least amount of code. A good language will aim for the easiest way to program something to also be the most computationally efficient. For instance, a Perl programmer who has a large ammount of data indexed by a string will often automatically reach for a hash. Which is great, because a hash is likely the most computationally efficient way, too.

Now, Class::DBI gets tripped up because it can't get enough information to create the SQL. Consider this Law-of-Demeter-breaking snippet for getting through a few joined tables:

my $table3 = $table1->table2->table3;

If we were writing raw SQL, we could get this in a single SQL statement with a three-table join (which is hairy, but possible). But because Class::DBI has to fetch each table individually, a lot more SQL would be needed (I'd have to dig through the source to know just how much, but it's definately more than a hand-optimized version).

So this is a case where the abstraction makes it easy on the programmer, but hard on the computer. Maybe that's OK, and maybe it isn't. It all depends.

[Mutant]

There's a difference between needing to understand how something works in order to use it, and needing to understand it in order to build and maintain it. I'm sure so long as the lower levels use Perl, then people will need to know Perl (besides just for personal fulfillment). But that number could be a lot smaller than it currently is. (I'm not trying to predict the demise of Perl here, I'm just saying that if the goal of a much higher level of abstraction was achieved, this is how it could potentially be.)

There are also good and bad implementations of higher level abstraction. Someone might try to implement a Natural English programming language that compiled to Perl code, but unless they made their abstraction layer extremely smart, it probably wouldn't work very well. But that's no reason to cast out the idea of having a Natural English programming language. If it could be implemented well, it would be something that would probably be very useful. The difference between a good and bad abstraction layer is how smart the layer is. If it's good at understanding what the programmer (or user) is trying to do, and optimising it, then it's much more likely to be successful.

Sometimes, levels of abstraction are attempted too early. That is, we needed one or more intermediate levels before the higher one was possible. For example, if someone had tried to implement Class:DBI without DBI, it might've been a huge failure. Again, this isn't a reason to say Class:DBI is a level of abstraction too high, just that we need to wait until the intermediate levels are completed (something which can take a long time - they have to be mature) until the next level is possible.

I'm sure there are plenty of attempts to create a level of abstraction that are implemented poorly, or are just before their time. Perl tends to be good at not rushing towards higher levels.