Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

What obfuscation can teach

by LassiLantar (Monk)
on Nov 04, 2007 at 08:48 UTC ( #648850=perlmeditation: print w/replies, xml ) Need Help??

I've only recently taken up obfuscation, having cobbled together a JAPH in the past few days. In general, I consider myself at least a competent programmer when it comes to straightforward code. My comments are comprehensible and I have legible style, I eschew global variables, &c, &c. But with Java as my first "real" language (and the one I work in most at the moment), I know little of the full power and feature-set of Perl.

Writing this first JAPH actually taught me a lot, though not directly in the writing. Naturally, as a first attempt, my mind wandered to ASCII codes, and I was forced to learn of ord and chr. In order to obscure things further, I played a bit with eval. Whee.

Most of the things I learned didn't end up in the JAPH itself. Instead, I learned them through studying (and, once or twice, understanding) a number of other nodes from the obfuscation section. They're not exciting; most are just built-in functions whose syntax I was finally forced to learn and got to see a few (very) creative uses of. A few, off the top of my head: splice, split, system, command-line switches such as W, n, T and e, reverse, and fork.

This was like a fun exercise in school which secretly (don't let the kids know!) teaches some core skills. I'm sure the next time I sit down to write an actual, useful script, I'll remember some of these functions and use them. It will save me time, and it will probably make the code shorter and more legible. Obfuscation has made me clearer and more efficient? Bizarre...

Learning from Obfuscation suggests rightfully that obfuscation lives in the "evil dark corners of the Perl." But a major component to successful education is that people be interested, engaged and having fun with what they're learning. Our XP system is one simple way we amuse and encourage ourselves to keep working hard at hacking, teaching and learning. Obfuscation is another. It's a learning tool, a self-challenge, a mental exercise. It provides puzzles to create, take apart, reassemble, solve. It strengthens the mind like working out strengthens the body.

But surely: writing regular code does all these things too. So how is the mental process of writing code that belongs buried in deep dark corners different than writing ordinary code? What does obfu do for you? What has it taught you, and how has it made you a better hacker?

Edit: Corrected the node number of my japh.

Replies are listed 'Best First'.
An obfuscated parable (was Re: What obfuscation can teach)
by roboticus (Chancellor) on Nov 04, 2007 at 12:17 UTC

    ++ nicely stated. The only thing I want to comment on:

    But surely: writing regular code does all these things too. So how is the mental process of writing code that belongs buried in deep dark corners different than writing ordinary code? What does obfu do for you? What has it taught you, and how has it made you a better hacker?
    I don't believe that writing regular code necessarily does all these things. When you're writing code to solve problems, many times you'll stop learning in a particular direction once you find an idiom that solves your immediate needs. Then, when you encounter a situation where that idiom isn't the best fit, a lot of times you'll rearrange the code to make it fit better. As long as you get the solution, it feels like a fine solution. Only when it takes a lot of time and/or effort to make the idiom fit do you look around for a new idiom.

    The problem with writing regular code is twofold: First, you'll spend a *lot* more time with a small toolbox of idioms for your coding. Second, as your toolbox helps define your abilities, it also affects your vision. You can't easily see how restricted you are for making solutions.

    So, while you go about writing regular code, you might easily come up with solutions to all the tasks you encounter. But you're missing the opportunity to solve them better. Every second you spend writing code that you don't need to write is wasted. That time could be better spent thinking about the problem, improving your design, or working on the next project.

    A horribly contrived example: Suppose in some bizarro universe, multiplication isn't widely known. It's not unheard of, but it's only used by some elite few. So, if you need to know the area of a square of a given height and width, you might write some code like:

    my $area = 0; for my $h (1..$height) { $area = $area + $width; }
    Now, you might be perfectly happy with that. After all, it's simple, it's purpose is clear, and it's a common idiom (remember, we're in bizzaro world). Now you're working on some code written by one of the "elitists". You encounter the code:

    my $area = $height * $width;
    You rebel: Sure it's less typing, but it's such an odd looking statement! No-one is going to know what that means when they see it. They're going to have to carry a cheatsheet around with them so they can look it up to remember what the heck it does.

    But after working on the codebase with the "elitists", you start to find it a natural idiom, and forget exactly what the fuss was about. Eventually, unknowingly, you become an elitist--but you now call it "being an experienced programmer". Later, when a "noob" comes by, he asks "What the heck is that? Why should I bother learning that? You guys should stop using such wierd code constructs and use stuff that we noobs understand? Why are you overcomplicating things?" (1)

    You sigh, and try to explain. Some will get it immediately, some will take a while, some will have to have it beaten into their heads with a cluebat, and some will move into management roles.

    Back to the subject at hand: Writing (or studying) obfuscations help you see more dark corners of the language more quickly, and helps you think in new ways. This helps you as a programmer: When you decompose your big problem into small bite-sized pieces, you'll be able to select the "correct" pieces to break the problem into. (I.e., the largest "trivial" chunks you can.) Ultimately, you'll make a better, smaller, more maintainable (but not by newbies) program.

    I don't write obfus myself, as I just don't have that much free time (between my son, my job and my myriad other hobbies). But I do take 'em apart sometimes because they're so educational.


    Notes: (1) Sound familiar? ;^)

      Kudos to you and the original poster; well-stated and thoughtful. Just to add a touch of gilding to the lily...

      I've read (can't recall where) discussions of the difference between what we might call exploratory programming and task-driven programming. In one case the goal is to learn something (feature of the language, a new technique, how a published algorithm works, a design pattern, etc.); in the other the goal is to produce a concrete result (typically under time pressure). While the two goals can mix, they usually are pursued separately under different mental states, which don't mix. A person engaged in one will typically be frustrated by interaction with a person engaged in the other (e.g., thinking of the other as "goofing off" or "being a stick-in-the-mud").

      I believe that it's important to have a well-balanced mixture of the two. Although my job involves the creation of code, I feel a day has been lost if I don't learn something (and today I'm usually glad for what I learned yesterday).

      Your example of multiplication touches on another interesting contrast: concepts of the problem domain vs. concepts of the programming language. Theoretical mathematics and applied mathematics have developed in a complex dance in which each has taken by turns inspiration and results from the other. Similarly, if the idiom you presented were commonly-enough used, one would expect the DRY principle to drive it into a function. If the function were commonly-enough used, the equivalent operator would become more acceptable. If either became common enough, the concept might "leak" back into the subject area.

      All of which is just a round-about way of saying that having fun can be a powerful way to learn (if one is not a stick-in-the-mud).

      The darker side of all this is the general "dumbing down" of culture, at least in the US. The media present 15-second sound-bite oversimplifications of complex events/stories. Some employers want to hire inexpensive labor and use them as plug-replaceable factory-line units, even for information-intensive jobs. People of learning and adequate vocabulary are mocked as "eggheads" (or less-flattering terms) instead of respected.

      My favorite answer to the above is Dijkstra's response to someone who challenged formal programming techniques as being too difficult for use in the education of "the average programmer". He replied, "What about the education of the average brain-surgeon?"

      In medicine, we don't restrict physicians to the narrow range of techniques available to the average nurse's assistant (despite the apparent desires of the insurance industry to second-guess every medical decision and to control treatment by tying payment to their approval of treatment/medication). Similarly, we shouldn't restrict all programmers to the techniques of noobs.

      How to get that distinction recognized is another problem.

      Excellent! ++!

      When I wrote the chunk you quoted at the top of your post, I knew there was an answer like this one but wasn't sure of what it was. Indeed, I was rather sad, because I figured that nobody would figure it out and I would never know. The idea that writing ordinary code tunnels your vision is something that will stick with me. It's something that I've noticed before in other guises - after all, it happens to me all the time when working on actual projects - and never generalized.

      In particular, I see it in retrospect when attempting to extend previous work. Having written some code X that accomplishes a task Y, I look back on it and think "Hmm, this should be extended to perform Y+", where Y+ is some related but additional task. When writing X I was following a misguided version of the YAGNI principle and had left that question for later. Interestingly, this holds even if X is the most logical way to accomplish Y alone. If I had considered this all from the beginning, I may have instead written Z, which is the most logical way to accomplish Y only when considering that we may also have to do Y+ (or other related tasks) as well.

Re: What obfuscation can teach
by dragonchild (Archbishop) on Nov 04, 2007 at 19:59 UTC
    All excellent points. I would point out a few things (in no particular order):
    • Obfuscation (and golfing, my personal passion) are dual-purpose programs. You are not only making sure that your program correctly solves the problem, but you're also attempting to have the code satisfy a constraint. This helps us write maintainable code because we become used to thinking about providing two deliverables in one package - a program and a work of code.
    • The key to mastering any toolkit is to understand the limits of that toolkit. Given that a programming language is nothing more than a toolkit, obfuscation and golfing provide an excellent way of exploring the oft-hidden areas. This gives us confidence that there are tools we haven't learned yet. While I don't use the inner workings of various built-ins that I've learned from golf, I have learned to look at another unexplored area - CPAN. There have been a rash of questions over the past few weeks regarding list manipulations. Many monks have provided answers using code they wrote. All of my answers involved List::MoreUtils.
    • As with any other skill, the more uses we put it to, the more varied practice we get. As practice is the key to mastery, the more practice the better. I would be hesitant to hire any developer who doesn't choose to exercise their programming skills in non-work endeavors. They just won't have the practice in.

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: What obfuscation can teach
by vrk (Chaplain) on Nov 04, 2007 at 17:19 UTC

    There is a difference between creative programming and problem oriented programming. Both are creative activities in one sense, but the means and the goal are entirely different.

    In "problem oriented programming", you have a particular, application specific goal you want to reach. You have to think in terms of the application domain and the application (or at least it's highly beneficial to do it that way), and somehow find a program that solves the problem you have. The end is more important than the means.

    In "creative programming", the goal is fixed. Whether it is fixed to "a JAPH" or "a better plotting algorithm" doesn't matter, because the important thing is the means. You try to find not only a program, but the program, the best there is (with respect to criteria you restrict yourself with). The end doesn't matter as long as the journey is enlightening.

    Writing JAPHs lets you focus on technique and methodology. The produce is irrelevant; no-one uses other people's JAPHs, except if they embark on the same journey. The real value is what you go through when writing it.

    print "Just Another Perl Adept\n";

Re: What obfuscation can teach
by technojosh (Priest) on Nov 05, 2007 at 22:28 UTC
    Although I have yet to post up any of my own obfuscations or JAPHs to this site, they are some of the most interesting learning tools I find here.

    Mind you, if I have a specific question, there's better places to look for answers.

    I try to learn from everyone's obfus by just straining my n00bish mind to attempt to figure parts of them out. I find it's usually a bit more challenging than whatever the solution to my problem-du-jour might be...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://648850]
Approved by moritz
Front-paged by moritz
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (5)
As of 2018-01-22 20:44 GMT
Find Nodes?
    Voting Booth?
    How did you see in the new year?

    Results (237 votes). Check out past polls.