RFC: How to succeed with your Perl homework

[angiehope]

So you're studying biology, linguistics or management and taking a programming class? You're comfortable working on your Windows PC, using Word documents, Excel spreadsheets or Access databases. Maybe you've already used VB.Net or created an HTML page. And now you're supposed to write a Perl script under Unix/Linux..

1. Preparations

1.1. Editor

Most Unix/Linux systems offer you two possibilities to edit your script: emacs and vi. Both use different menus and keystrokes than other Windows or Linux applications. If you already know how to touch-type, vi will be more comfortable, once you've learned how to use it. Open a terminal window and enter "vimtutor" to get started. If you don't touch-type or have little time left, I would recommend using Emacs. easymacs will make things much easier for you - allowing you to use Emacs the same way as any other software you might already know. However, it consumes 70 MB of disk space (version 2.0, uncompressed), so check your disk quota before trying to install it. (Open a terminal window and enter "quota -v" to see how much disk space you're allowed to use.) Otherwise, I would recommend Effective Emacs by Steve Yegge and the Emacs tutorial (start Emacs and click on "Help > Emacs tutorial" or just type <Ctrl-h> t.

1.2. Version Control

Version Control software permits you to manage different versions of your script. You can easily change back to an earlier version of your script when you notice that it stopped working after saving your 99th version at 2 am in the morning...
Personally, I'm using subversion, because it's available under various operating systems and can be used for single and team projects.
You can find out how to set up subversion as a single user reading the High Speed Tutorial.
An additional benefit of version control: if you enter descriptive messages when checking in a new version, the log file produced by svn log will be a valuable record of how you created your script. If you ever need to prove that you really did your homework yourself, this might help you.

2. Writing your script

2.1. Make an outline

So you've set up version control, started emacs and used <Ctrl-x> f homework.perl to create your first Perl file. Stop. If you just start typing Perl commands (or pasting them from another web site), you'll end up with a mess of code later.
Take your assignment text and either print it out and use a text marker or copy the relevant phrases/words into your file that describe what you're supposed to do. This might look like:

# author: Your name
# created on:
# description: solution for assignment no. 
# read command line option -file
# open your text input file
# read each line inside your file
# and: replace string "NP" by string "nominal phrase"
# or: check for string "ACGT"
# open your text output file
# write your changed or found lines to your output file
# close your text output file
# close your text input file

The "#" at the beginning of the line tells Perl to ignore those lines as comments. Sometimes, your assignment will be clearly written, so you just have to extract the relevant text, sometimes you'll have to think of an appropriate description yourself. In each case, your outline will be a good basis, should you need to explain what your script is doing.

2.2. At the beginning

At the top of your text file, I would recommend adding the following lines:

#!/usr/bin/perl

use strict;
use warnings;

use Data::Dumper;

The first line makes it possible to run your Perl script on its own, i.e. "./homework.perl" and not "perl ./homework.perl". All you need to do is to type "which perl" into your terminal to find out the correct path to your Perl binary, e.g. "/usr/bin/perl" or "/usr/local/bin/perl" and then "chmod 755" to tell Unix that your file is an executable script.
use strict; offers you a lot of helpful advice on commands that might cause trouble when you run your Perl script and use warnings; provides more informative error messages.
use Data::Dumper;: print Dumper [your Perl variable] will help you tremendously, when you need to examine what type of data you are passing around inside your script.

2.3. One step at a time

Try to solve one task in your outline at a time, i.e. first make sure that your command line arguments get into your script, then look at how to best read your data file and then at how you process/change it. Make sure that each step works by printing out the results (or part of them) on the command line.

2.4. Don't repeat yourself

Especially if you're in a hurry, it might be tempting to write an entire list of statements.

$temp_fahrenheit_1 = 32 + $temp_c_1 * 1.8;
$temp_fahrenheit_2 = 32 + $temp_c_2 * 1.8;
...

You'll save yourself many lines of code (and lots of work) if you delegate this type of work to a subroutine:

sub c_to_fahrenheit {
    my $temp_in_c = shift @_;
    my $temp_in_f = 32 + $temp_in_c + 1.8;
    return($temp_in_f);
}
$temp_fahrenheit_1 = c_to_fahrenheit($temp_c_1);

If you've got to apply your function to an entire array of values, things get even easier:

@temperatures_in_c = (5,10,15,20);
@temp_in_f = map(c_to_fahrenheit($_),@temp_in_c);

This applies your subroutine to every value in your array.

2.5. Using references

Unlike other scripting languages, Perl uses references, i.e. variables that contain links to another variable. If you want to send data to a subroutine (or back) that is more complex than a simple number, string or an array, this is extremely useful.
You can create a reference, pass it to a subroutine and unpack it there:

do something(\@myarray,\%myhash,\$mystring);
sub do_something { 
    my @localarray = @{shift @_};
    my %localhash = %{shift @_};
    my $localstring = ${shift @_};
    ...
}

If you suspect that your references might not work as intended, do use print Dumper @_; at the beginning of our subroutine to look at your variables.

2.6. Keep it simple!

There is an entire chapter in "Programming Perl" devoted to arrays, hashes, arrays of arrays, hashes of hashes, hashes of arrays ... . If you need to store your data in a complex data structure before writing it back into another file, don't let it get to intricate. Put a note into your outline (or draw on a piece of paper) how your data structure should look like.

%course = (code => $code; 
    weekday => $day, 
    time => $time, 
    title => $title, 
    location => location);

will work better than

    $date{weekday} = $day;
    $date{time} = $time;
    $course{"date"} = %date;
    ...

2.7. Test your regular expressions

As soon as you're told to replace part of a string by another string, you'll start using regular expressions.
A regular expression is a string that contains a pattern you can use to find or extract specific sections in your data. You can find a first introduction at perlrequick.
However, if you just want to replace one string by another and store the result in a new variable, here's a quick solution:

$testphrase = "The colours of the rainbow are red, yellow, green, blue
+, indigo, and violet";
($newphrase = $oldphrase) =~s/colour/color/;

3. Practice, practice, practice

3.1. Perl at home

Even if you don't want to program for a living, you won't get far using Perl if you don't practice. And you won't enjoy practicing Perl if you can't connect it to what you do outside programming.
As soon as you decide to install Linux on your computer, you can get Perl and most of the modules and programming tools that interest you by using the package manager of your distribution, for example (Ubuntu. If you want to use Perl on your computer at home without leaving Windows or OS X, the easiest solution would be to install <a href=http://www.activestate.com/activeperl/downloads">ActivePerl and a text editor of your choice. There's a modified version of Emacs for Windows and OS X by Vincent Goulet, which contains lots of other useful Emacs modules. vim also provides Windows and OS X versions, and an excellent Perl support plugin.
Another solution would be to use Padre a dedicated Perl editor packaged with Strawberry Perl. In case you want to take your Perl homework home with you, write your script under Windows and expect the result to work on a Linux machine inside your computing lab, that's the version I would recommend.

3.2. Read code

You want to know how a certain Perl module works, for example the BioPerl module used to process gene data, Lingua::Stem to find word stems in several languages or Finance::Quote that collects stock quotes and fund prices from various sources on the net? Download the source code and try to understand what it does, what data is collected and in which files it is processed. EasyEclipse and the Eclipse Perl Integration plugin give you an excellent tool to understand how the parts of a complex Perl module are connected to each other.
Don't panic .. Perl might look and feel different from what you've gotten to know so far, but once you have become used to it, you'll appreciate its reliability and speed -- at least when it comes to process large amount of data in the background while you might go off and have a coffee :-) ...
Update Applied corrections suggested by moritz and toolic (on Data::Dumper).
Thank you!

[luis.roca]

Thank you for putting this together. I like the way this starts out with "Preparations" and the whole idea behind "How to Succeed With Your Homework". I think I would have liked to see you follow through in that direction instead of mixing in notes on regular expressions and something a little more advanced like references.

When I began reading I was expecting your tutorial to be organized something along the lines of :

"How to Succeed With Your Perl Homework"

1. Tools
   Discussing the text editors and version control software.
2. Resources
   PerlIntro was a nice suggestion as well as studying modules. A few more examples like books and links to good beginner tutorials on perlmonks.org would be a good addition.
3. Prepare Your Work Area
   Creating the outline and what to include at the top of the file.
4. While Your Writing Code
   I think the "One Step At A Time", "Keep It Simple" and "Don't Repeat Yourself" would be better here.

*Then, I think if you really want to include them, place the notes on regular expressions and references in a separate "Miscellaneous" category. It's not that they aren't good tips. I just prefer the simplicity of 1. Get the right tools, 2. Gather good resources, 3. Prepare your work area, 4. While you're cooking

Overall I love the tutorial and the thought you put into it. I think this is going to be really valuable for people like me and especially those who are in their first month of learning Perl. I do think with some of these changes it can give it more clarity in direction and make it stronger.

Thanks again. Nice work!

[toolic]

use Data::Dumper; is optional. It's not part of the Perl core.

Data::Dumper is a Core module; just ask corelist:

corelist Data::Dumper

Data::Dumper was first released with perl 5.005

It should already be installed.

No how-to guide is complete without a mention of the free online Perl documentation, i.e., perlintro

%course = {code => $code; 
    weekday => $day, 
    time => $time, 
    title => $title, 
    location => location};

Did you really want those &gt;?

[JavaFan]

No how-to guide is complete without a mention of the free online Perl documentation

I would say, if you're mentioning documentation, point out that the documentation comes with Perl. You'll have all the documentation, even if your network is down - or if you're behind a firewall. No need to depend on a remote side being up either.

[talexb]

To offer a different perspective, I love git and I'm very happy with vim as my editor. I also prefer common::sense rather than strict and warnings.

And as far as version control goes, save early and save often, kids. There's nothing like getting a script 95% working .. then trying Something Different somewhere, breaking the script, repairing it, breaking it again, fixing it again .. and pretty soon you wish you could go back to 2am to the version that mostly worked.

Finally, when you're new to Perl, it's tempting just to write everything yourself. You'll burn up lots of cycles that way -- check CPAN instead and see if there's a module that does that already. You never know until you look.

[Tux]

I personally would never ever advice a novice perl programmer to use common::sense, Modern::Perl or whatever module that replaces use strict; use warnings; and use autodie; for several reasons. The main reasons are:

• They are not part of perl CORE. That implies that the user has to install perl modules, which is most likely not part of their homework.
• They are not likely to show up in any example code in the standard perl docs.
• It does not hint towards what these programmers will see in everyday scripts and examples they see on the internet when they browse for examples.

In fact, it is way more likely that I will remove such declarations in scripts that I receive from others if I have to use it, than that it is likely that I will change my headers to move to whatever replacement for the default tantra is currently seen as best.

Remember that this is a guide for a homework-beginner. Not for a seasoned programmer that knows exactly why (not) to use common::sense.

I think I am a seasoned perl prgrammer, and I use git, elvis (a vi clone), strict, warnings, and more and more autodie.

[chromatic]

Remember that this is a guide for a homework-beginner.

That's exactly why the second page of the Modern Perl book shows how to use Modern::Perl and explains what it does. It's awfully silly to poke fun at all of the boilerplate Java programmers have to convince their enterprise strength IDEs to emit for them before they can write "Hello, world!" when any correct Perl 5 program worth writing in 2010 has several lines of equally opaque boilerplate.

[angiehope]

Hi!
Thank you for your reply.

To offer a different perspective, I love git and I'm very happy with vim as my editor. I also prefer common::sense rather than strict and warnings.

Nothing against vim ... I'm just talking about users who come from a non-Unix background (Windows or OS X). When I first started using Perl under Windows, I was grateful for eshell and the respective output windows provided by Emacs.
In case someone wants to use vim under Windows to edit Perl, I would recommend installing MSYS - the Windows DOS box has too many limits.


Finally, when you're new to Perl, it's tempting just to write everything yourself. You'll burn up lots of cycles that way -- check CPAN instead and see if there's a module that does that already. You never know until you look.

Fine - as soon as students work at home, they're free to download and use any module they want.
In a computer lab, however, restrictions similar to those discussed in Yes, even you can use CPAN apply: even when your disk quota is high enough to install CPAN modules locally, your instructor might want you to solve a given problem yourself.

[halfcountplus]

I'm a vim lover, but I'd take that first bit about "what editor to use on *nix" completely out (or else, do a real survey). You do not have to pick between emacs and vim, and I would not necessarily recommend either of them to people both new to programming and the *nix environment. Why not tell them how to sit in the chair as well -- and that the chair must be on a pedestal with wheels, not four legs? And that you must have a wheel on the mouse? Etc...


There must be close to a dozen GUI editors with syntax highlighting for perl available under linux, and most systems will have one, eg, gedit, available. Those tools are very simple to begin using at once, especially for windows users who will be comfortable with their mouse and menu driven interface. Vim and emacs are great, of course, and no doubt later on you may want to explore the real tish, but gedit is totally sufficient at first and requires no attention, leaving you to focus your mind on perl and not contemplating bells and whistles like "code completion", key macros, six different ways to cut n' paste, etc. Or having to read a tutorial in order to type a script, save it, and load it again without problems. Type. Save. Load. Syntax highlighting. Cut n' paste. Maybe print. That's it. All the common linux desktop environments (gnome = gedit, kde = kate, xfce = mousepad, et. al.) have something standard integrated into them that will do those things via some nice iconic toolbar. Point and click. No installation or tutorial required.

Other than that diversion, this looks like a nice, succinct set of beginning tips. WRT use strict, etc:

use warnings FATAL => qw(all);

Is a great thing to know about. Beginning programmers easily ignore warnings "because they aren't real errors" and miss the significance of, eg, mistakenly using uninitialized variables in the wrong place. Generally when developing something, I don't want it to "keep going anyway" if I get warnings -- it's much easier to stop right there and fix the first problem that arises. And, just like there is no good reason for a program to be unable to run strict, there is no good reason for it to produce any warnings. Start with the good habits and best practices right away and you will never find them a hassle later on.

[MishaMoose]

I find that gvim works quite nicely native under Windows. And I agree that the instructor might well insist on the student solving it on their own. Sometimes we learn more from a re-invented wheel than from simply 'remounting' one.

My best professors made us learn what useful tools did by writing one before they allowed us to use them. Of course that was back in the stone table an dbear skin days

[sundialsvc4]

An excellent post!

One of the most sensible books I ever read said a great deal by just its title:   Algorithms + Data Structures = Programs.   This is a good approach to take.   Consider, separately, (a) how you intend to represent the data, and (b) how you intend to process it.   If you cannot clearly express those two things on paper, then you cannot possibly be ready to express them to a computer.

It helps tremendously to build test-cases while you are building something.   (See: Test::More and all of its many kinfolk.)   When you change a program “anywhere,” you can break something “anywhere else.”   But, if a program that used to say, All tests successful, still does, then at least you know where the problem probably isn’t.

I am also a big fan of writing the documentation, in the source-code, before you actually write the subroutine you’re about to write.   There is always a notepad (and a coffee cup) at my left hand.

Finally, pay very close attention to http://search.cpan.org.   No matter what you’re doing with Perl, it has probably been done before.   Or at least, there is something you can use.   Learn to be very jealous of your own time.   There is definitely a place in the world of computer software development for preparation, forethought, and research.

[SuicideJunkie]

No love for keeping it simple with nano?

Most outlines I expect will need a second pass to fill in more detail before starting on code... although it does depend on how much spoon-feeding is going on in the homework assignment.

I suggest something connecting the outline to the getting started part.
For example, after writing that outline in comments, keep it active, and fill in code between the outline lines. As you fill things in, you end up with commented code, and keep a local reminder of what you meant to do next.

[juster]

++!

I would also like to recommend ErgoEmacs. I have never tried Easymacs but it seems to have similar goals. Like Easymacs, ErgoEmacs tries to make emacs follow now familiar UI standards. As you may have guessed ErgoEmacs also has the goal of following established ergonomic standards. ErgoEmacs gives the same standard key-bindings like C-c, C-x, C-v, C-n, for copying, pasting, etc but also provides alternatives using Alt because it is easier on the pinky finger.

edit:Correction... ErgoEmacs doesn't seem to bind the Ctrl modifier key to those shortcuts, only the Alt key. This is the same as a Mac's shortcuts. This is different from windows which uses Ctrl.

ErgoEmacs also has a windows installer which comes with many packages and bundles MSYS for UNIX utilities needed for emacs functionality. I use Linux and haven't tried this but I will the next time I use Windows.

[sundialsvc4]

Heh.   It always amazes me how people do love to use editors that still rely on obscure control-character sequences and character-mode screens.   And who continue with the “real programmers qr/do|don\'t/ use this-or-that.”   That’s not the point and it never was.   Oh, it might be what you are used to, and it might even be what you prefer.   But programming is a craft that consists mostly of devising a solution to a problem, such that an electronic device can carry it out without thinking.   What language(s), editor(s), debuggers and what-have-you you use, is immaterial to that.

And, fair warning, it is a craft that can only be learned by practice.   I will never forget my first (and only) attempt to throw a clay pot on a wheel.   It looked so easy.   But that (!#@#@!!) blob of clay flew off that spinning disc at considerable speed and hit me squarely in the gut.   My instructor helped me clean up, spoke soothing words, and in a few deft strokes fashioned the pot.

[Your Mother]

What language(s), editor(s), debuggers and what-have-you you use, is immaterial to that.

Perhaps from a philosophical-Turing perspective but not in reality. I was teen hacker, honors math student who fell *completely* away from it and ended up with a liberal arts degree, etc. If I'd only had Java or C and friends, I never would have resumed programming. The only reason I came back was Perl. I don't hack because I have to. I hack because it's fun solving problems this way. And not to start a thread about it because it's personal taste but I might not have come even with Perl if vim had been the only universally available vehicle.

[biohisham]

Kudos for this angiehope...I have dealt with folks coming to Perl from different disciplines. Some exhibit influences of the languages they already use and others are yet to develop thinking along lines of computer programmers. The educational programs are as just varied as the student base they're directed at, this is true for someone who're not self-educating and hence, the types and levels of homeworks may be different since some of these programs go so deep in Perl while others are just skimming through operators and basic Perl stuff. The guidelines you provided unify all these aspects under a general umbrella so tactfully.

Excellent work indeed specially the bit when you spoke about version control, that is an important concept. However, you've limited the scope of the guidelines to only Unix/Linux when you spoke about the editors out there. There are many editors that have simultaneous support for many languages across different platforms and a student maybe using such an editor already in writing another language. So you can suggest that a student may check whether their editor has support for Perl syntax or not

What a novice learning Perl should focus more on is writing correct syntax and following good coding practices, when I am training folks on Perl I get them to use plain notepads first of all to get them familiar with understanding the syntax errors and warnings generated and then acting upon these errors rather than depending on some syntax highlighting features that may distract attentions (for instance, some excellent editor that I use doesn't highlight non-core module names for these modules installed not through PPM or CPAN)

use Data::Dumper;: print Dumper [your Perl variable]

or a [reference to your Perl variable]. Consider the following:

use Data::Dumper;
my @dim_array = ([qw(1 2 3)],[qw(3 2 1)]);
print Dumper(\@dim_array);
print "\n";
print Dumper(@dim_array);

for example the BioPerl module used to process gene data

BioPerl is a collection of many modules that parse a lot of data related to biology and bioinformatics

Finally, emphasis on documenting code properly through exercising oneself in the practice is another point worthy of mentioning, after all, we learn Perl to write code that may be supported by someone else and documenting such code may make maintaining it a lot more easier..

[TomDLux]

My firs course at U of T, early 80s, we had to use ed under a restricted shell ... that's the commands from vi without the full-screen aspect. I think you should make them all do the same, uphill both ways in the snow!

Otherwise, looks great ... though I'm dubious anyone programming for the first time will be ready for map{}.

[raybies]

I know there are better tools, but honestly I still am most comfortable with whatever vi I can find and plain old cvs. It does surprise me, however, how many programmers (most often addicted to a certain studio of the visual sort...) I run into that don't bother with version control at all.