|
in reply to Reinvent the wheel!
Thanks to everyone for feedback. It looks easier to respond in one place, so here it is.
@ Zentara: I agree that there has to be some limit where you stop trying to do it all yourself. And there's no danger of me ending up in Assembly: Even the little bit of C that I've done was enough to make me (as Lawliet and starX say) very, very grateful for Perl.
@ Perrin: I wouldn't quite say that I'm bored. Let's all agree not to put it that way to my employer. Still, your advice is good. I have tried to "do more than I have to" with the code I write. (For example, I doubt that many 100 line utility scripts provide command-line parsing (via Getopt::Long) and a full man page (via Pod::Usage).) I actually do that for the same reason I tried this "reinventing the wheel" experiment: to push myself, to learn something new. Nevertheless, you may be right that my time would be better spent on new work.
@ Chromatic: I hear what you're saying, but much of what you wrote doesn't straightforwardly apply to me. (I admit my case is unusual.) For example you write:
Writing code for didactic purposes may be appropriate, if you learn something from it. Did you seek code review? Exploration testing? Adherence to the relevant standards? Algorithmic analysis? Did you compare the results to results from a widely-used, well-tested alternative?
As I said in my original post: I am the only person writing code in any sense where I work. And quite frankly I'm a complete amateur (and novice). It sounds like you are primarily thinking of professional, full-time programmers. (A reasonable enough focus for you, but it's not who I am.) Still, to respond to your questions:
- Nope, no code review. I don't really have anyone to ask. See below for why I'm not going to ask here as far as this experiment goes.
- I have no idea what exploration testing means in this context. So, I guess I didn't do it. I'm only just learning how to write tests at all. Yeah, yeah, I know, get out the torches and the pitchforks. (Coincidentally, I bought your book on testing about a week ago because I figured it was high time I learned.)
- There are standards for filesystem traversal? Cool. Who knew? Nope, I didn't look at those either.
- I do have an inkling of what algorithmic analysis means in this context, but nope, I didn't do that either. Again, I have no training or background which would allow me to analyze an algorithm in any very substantive way. I thought about what I needed to do, tried things, looked at how they failed, tried some more things, and so on. However, I probably should have done some algorithmic analysis. When I run my code against a fuller directory (eg, my $HOME), it wets the bed with a whole lot of "Deep recursion on subroutine main::walker" errors. During development, I only ran it on a smaller sub-directory. Damn.
- And, finally, I am in the process of comparing my results and code with File::Find. I certainly appreciate File::Find more than I did two weeks ago.
But all of that said, I think you missed one crucial point: I wasn't doing this intending to release it upon a production environment. I was doing it to learn. I was going to test it on the blogs at my work but only to compare with File::Find. All my actual scripts for work use File::Find or File::Find::Rule. I'm dumb, but I'm not stupid.
@ Grandfather: I have one module written for my work code, and you're right that I should continue to work on that. I understand OO Perl well enough to use other people's OO modules comfortably, but for my own code, I find procedural (with a little bit of functional) more natural so far. I should probably work on that too.
I wholeheartedly agree that teaching is a good way to learn, and I do actually answer questions on the rare occasion here, at Stackoverflow and on the Perl Beginners mail list. It's a great way for me to solidify my understanding of things. I also read a lot of posts to see how people approach things. I think I will pass on the invitation to post my broken wheel to the Cool Uses for Perl part of the site. (See above on "Deep recursion" errors. Damn.)
So where am I left? Well, I'm less thrilled with my experiment, now that my wheel doesn't roll properly except on small streets. I also think people poked some good holes in my reasoning. On the other hand, I do feel a lot more comfortable with callbacks and code references. Maybe this wasn't the best way to spend last week, but I'm not yet convinced it was a complete waste of the week either. |
Re^2: Reinvent the wheel!
by chromatic (Archbishop) on Mar 22, 2009 at 00:38 UTC
|
It sounds like you are primarily thinking of professional, full-time programmers.
Not at all.
perrin had a pithy quote about opportunity costs. I believe that the goal of programming is to solve problems, not to produce code. If you can solve problems without writing code, so much the better. Not everyone agrees.
I do believe that an effective learning technique is doing (which is one reason I believe the testing notebook works in it current format), but I also believe that unguided doing is a poor use of time. It's trivial to write a simple HTTP server, if you ignore most of the HTTP specification -- but as a learning technique, writing it by sniffing socket traffic from a single web browser on a single operating system is a poor way of understanding how HTTP works. There's no substitute for reading the relevant RFCs.
If you're only playing around and throw away the code after you've finished, that's fine -- but ignoring all of the knowledge available about what works, what doesn't, and which ideas seemed good but turned out to have problems means you're shortchanging yourself (and anyone who relies on the code) out of false hubris and false laziness.
Writing good software is difficult, especially if it interacts with other users and other software. There are no shortcuts besides well-honed good habits. I want to encourage people to develop good habits and discourage them from developing bad habits.
| [reply] |
|
|
I believe that the goal of programming is to solve problems, not to produce code. If you can solve problems without writing code, so much the better.
The single biggest contribution to reliability, security and maintainability, is using less code to solve the problem.
Note: using, not writing. You may succeed in writing less code by using a one or more modules, but if the modules are over-engineered, too generic, grossly hierarchical, or otherwise verbose, then you can still end up with more code than you need. And that's bad all round.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] |
|
|
| [reply] |
|
|
chromatic,
If I had to come up with a number, I would wager that 75% of the code I write is "throw away" code. I do not work as a professional programmer and think of code as my creative outlet. It is easy to see from a
few links that my primary pursuit is knowledge. As an artist, it is often important to me not to be influenced by existing solutions when solving a problem.
Here is where I make a clear distinction and was happy that you wrote this node. I very seldom have the hubris to believe I have created something superior. I go back and examine very closely existing wheels. I run my code through their test suites. I discover what bad assumptions I made and how others addressed them. I spend way more time on a problem than is required to solve it so I might squeeze the very essence out of it and learn as much as it has to offer.
This isn't the only way that I "waste" my time learning. I spend a lot of time answering questions here and on #perl IRC. I see the common elements of problems over and over again. I see how others responding solve the problem. I learn from what they have to offer and often will summarize the differences in the solutions and why one may be more advantageous over the others. It has been my experience that well worn wheels are able to accomplish the job for 98% of the general problems - there is still 2% that require going beyond existing wheels.
I spend this 75% of the time doing "pure" research so that I can do my best the 25% of writing "applied" solutions. I do make money from programming (hobby vocation) and I do solve real world problems (for myself, for my employer and for complete strangers). This includes very large complex applications and not just 50 line scripts.
After nearly 7 years of programming in perl, I think I would still be a novice if I only wrote code when I needed to.
Note: All the numbers in this node are made up WAGs
| [reply] |
|
|
This isn't the only way that I "waste" my time learning.
Learning isn't a waste of time, not in and of itself. Undisciplined learning is.
I'll try a different metaphor. Playing scales and arpeggios on the piano (or Chopin's Études) is usually a good exercise, but if you slouch or rest your wrists on your knees and have poor fingering technique, they're lousy, wasted exercises. If you let yourself slip up and hit the wrong keys, you're only hurting yourself.
In a similar way, well-focused research is often valuable. I know you make a sharp distinction (as do I) between experimental, research code and code you intend to maintain. That's a very important point. If discipline in practice and research is important (and it is), it's because it helps to establish and maintain good habits for when you use your skills for real.
| [reply] |
|
|
|
|
|
|
Ok, all fair enough and a good challenge to me. However, you keep talking about webservers and the relevant standards or RFCs. In my case, though, what was I supposed to read? The only standards I can think of are things line the Unix or Linux file system hierarchy standard, and that's not relevant to my concern. (At least, if it is, the connection isn't obvious to me.)
I ended up doing this because I was rereading Intermediate Perl, and I kept thinking about the general problem that file system crawling presents: What do you do when (1) you need to drill through a structure, (2) you have no way to predict in advance how far down it goes, and (3) at each branch the items you find may be a simple thing (a file) or a complex thing (a directory containing zero or more directories and zero or more files)? How do you build a map of such a structure in code? Once you have the map, how do you reorganize it for printing? Perhaps you don't want to print it, but you want to extract one piece of information (the byte count) about one of the types of thing in your map (the files). How do you do that most efficiently? The majority of what I do involves files and folders: scanning them for specific types of things, checking their size, updating them, etc. So it's a problem I need to care about.
If I understand you correctly, you're saying that my time would have been better spent reading through the code in File::Find. Is that your point? If not, then I would be curious to know what you would recommend I do. But, please, no more webservers. I'm not writing one. I promise.
| [reply] |
|
|
In my case, though, what was I supposed to read?
Fair enough. In your case, a good POSIX or Unix book which describes how filesystems work. In particular, some of the grottiest code in File::Find detects such weirdnesses as recursive symbolic and hard links that can turn a nice tree structure into a complex graph with cyclic relationships. That's a common enough situation that will not be obvious to most autodidacts trying to deal with this sort of problem.
How about dealing with orphaned symlinks? Non-regular files? Mount points? Overlays?
It's also a very common problem in graph traversal, of which "How do I find files under this directory according to some pattern?" is a refinement.
I kept thinking about the general problem that file system crawling presents....
Of course. This is a good example and learning opportunity. By no means do I mean to suggest otherwise. Yet a robust solution to the problem is far more complex than you might suspect.
Perhaps I'm reacting poorly to the phrase "reinventing the wheel", which connotes to me that you've examined the existing solutions and found them lacking in some means. (In my experience, the people who claim that they have really haven't, not in any detail.) Perhaps I've misunderstood you.
If I understand you correctly, you're saying that my time would have been better spent reading through the code in File::Find. Is that your point?
In specific, I can't recommend reading that code. It's not great. It does, however, address a lot of problems that occur in the real world that a robust filesystem walker does need to address. What's a good way of agreeing to the general question but backing off from the specifics?
| [reply] |
|
|
Re^2: Reinvent the wheel!
by gwadej (Chaplain) on Mar 23, 2009 at 13:58 UTC
|
To me, the most important point of your post is
And, finally, I am in the process of comparing my results and code with File::Find. I certainly appreciate File::Find more than I did two weeks ago.
Reinventing the wheel by going in a random direction can cause extremely bad habits. Taking your version and comparing it with a working version helps to moderate that effect.
Unlike some of the advice here, I think this is a valid and very effective approach to learning, as long as you don't forget the production wheels that you can learn from. You also have the opportunity to rewrite your code after the comparison to see if you really learned what you believe you learned from File::Find.
As professional programmers, it's easy to lose track of the need to practice. Rewriting code that is not needed for production can help you learn and help keep your basic coding skills from getting rusty.
It also helps to remind us of the sense of accomplishment that we used to get from making something small with our own two hands. When working on big production systems with multiple programmers and man-decades of work, it's hard to remember the simple pleasure of making the computer do something for me, as opposed to keeping the system up and functional.
Good luck in your explorations and don't forget to share your experiences with your fellow monks.
| [reply] |
|
|
| [reply] |
|
|
In my experience, the needs of a production system often tend towards minimal changes to keep things running. I've known many people who eventually reach a point where they don't really program as much as they serve the system.
This can involve extreme knowledge of the various libraries and configurations needed by the production system, and a loss of basic programming ability. (In some cases, this can be so extreme that they can no longer write a standalone program.
This may very well be a function of my particular career experience.
I find that working outside production environments allows me to exercise a different set of programming muscles. This also allows me to try things that might fail where the risk is lower.
I find this kind of programming practice both a good exercise and a good way to learn new things. It also helps me re-examine basic assumptions once in a while to keep them from solidifying into personal dogma.
I have also used reinventing the wheel, on purpose as a kind of Kata to practice approaching problems from a new direction.
| [reply] |
|
|
|
|
| |
|
Re^2: Reinvent the wheel!
by GrandFather (Saint) on Mar 21, 2009 at 23:54 UTC
|
Well, if not to CUFP, then at least post your wheel, broken or otherwise, in this thread. An important part of the learning process (and perhaps the hardest) is working through the criticism of your answer, but it may be the most rewarding in the long run.
Comparing your code with File::Find won't pick up on stylistic issues or preconceived notions that you may have and that will come back to bite you later on. Many good coding techniques look so natural that you don't even notice them unless they are pointed out. As a trivial example, missing strictures (use strict; use warnings;) won't stand out in your code if you are not accustomed to including them in every script you write. However strictures often make a huge difference to code quality and the ability to identify and eliminate bugs early. If you even just receive one answer that provides a tip on the order of 'always use strictures', posting the code is well worth the dent to your self esteem. ;)
True laziness is hard work
| [reply] |
|
|
