|Pathologically Eclectic Rubbish Lister
BioLion's scratchpadby BioLion (Curate)
|on Jun 16, 2009 at 18:01 UTC
Audience: Perl Monks and Everyone with an interest.
Introduction & Justification: Perl has come to cover many areas of IT and has been dubbed the 'glue' for that matter. Perl has also contributed to Biology, big time, it saved the human genome project and not only that, it has continued to be the mainstay of much bioinformatics munging and analysis, playing no small part in the burgeoning ‘*omics’ sciences.
The increasing number of bioinformatics related Perl problems that seem to be coming up in the Monastery, and the confusing and disparate resources available on the internet contribute a great deal to making BioPerl fearful or at least "perl-plexing"…
PerlMonks plays a great role in the evolution of Perl, it has encouraged many to join up the community and exchange knowledge in a place of utmost cohesion between its members and thus BioPerl coders can be equally encouraged to participate and share their knowledge and code.
Description - what this is and isn't:
So, while this isn't intended to do the job of the extensive BioPerl docs, or many reference points out there, it will hopefully be a starting platform for those looking to delve deeper into using Perl in bioinformatics related tasks and also assisting Monks in becoming more accessible to BioPerl questions: Facilitating the back and forth that makes Perl and the Monastery so special.
It is also to highlight the interesting problems that bioinformaticians have to deal with - not all are BioPerl related(!) and can often involve huge, diverse datasets. And we hope that these sorts of challenges will tempt a few talented programmers to get involved.
Tips on posting bioinformatics type questions in the Monastery:
Please go through the following whenever you notice that your question or parts thereof don't look like how you expected after you have hit the "preview" button and remember; a well formulated question will garner better and quicker response.
How do I post a question effectively? is particularly relevant for specialist, such as bioinformatics, questions. Here we try to highlight the importance of well formulated questions:Not all monks are familiar with biology terms and not all monks are into bioinformatics, so as much as possible, use clear language that describes what your problem is and use biology terms only when relevant, better still, post the part of your Perl code that describes the problem or demonstrate the problem in Perl.
"I have a DNA sequence that I want to BLAST and I tried Bio::Tools::Run::StandAloneBlast but it did not work how can I do that? "
These sorts of questions invite down-voting and confuse monks and their response would be either trying to extract words from you to get you to explain it better, make wild guesses that would confuse you the more or ignore your question rather than BLASTing on you. Better to think about what you are trying to actually do and think about how this is a Perl problem.
This leads to an important point - often overlooked - of providing test data (just enough - 3 cases of input, not the whole file, and if it is in a particular format - say which or provide an example of its layout !), and if you are really stuck, what output you want. This greatly helps people grasp what you are doing and also test any code they produce.
I am trying to convert a string (a DNA sequence) into a series of three-letter sub-strings– to do that I have written the following code but I failed to make the substrings overlap by moving one letter at a time from the original sequence in the forward direction.
Now that seemed like an ideal question, clear wordings, examples of input and desired output and the code involved if any so that testing the respondents code on the provided data is made possible.
Good coding practice:
Many bioinformaticians are new to coding and can be guilty of certain malpractices, so your code should be readable, self-descriptive and properly indented and commented. Good coding practices are critical point checks, they can alert you to avoid potential errors, dangerous coding behavior and enable you reduce debugging time and increase code efficiency and re-usability. And as always, use warnings and strict, check for errors etc… because you never know what this code could be used for! Maybe some IO error means that a potential cancer biomarker is missed (extreme example, but point remains!).
Also - remember that posted nodes can be edited at a later point to encompass suggestions, changes to code, what course was finally decided etc... Remember that it is considered good form to mark any changes with ‘Update:’.
Tips on Answering BioPerl Questions:
Typical problems and solutions:
A frequent problem is the installation of BioPerl, this in itself is not difficult if certain caveats are attended to, if you are familiar with Installing Modules then you are good to go. Note that there is some difference between the BioPerl Installation Requirements on Linux and BioPerl Installation Requirements on Windows and that not all of BioPerl is available on Windows hence you need to add the following repositories to the ActiveState PPM manager.
If anyone can contribute tips for other methods (e.g. Strawberry Perl and cpan?), it would be much appreciated!
The BioPerl suite of modules revolves around sequence acquisition, parsing and retrieval from public databases and automating various tasks related to studying these sequences BioPerl HOWTOs. Think this is simple? Think again - CODE.GOOGLE.COM tells us there are 3,666,478 lines of code to get your head round!
A sequence is just a text string in a certain format (this format is described in the beginning of the text file containing that sequence) that represents either a gene or a protein, the alphabet of the sequence with regard to genes is but a combination of four letters (ACGT) and sometimes U (replaces T) and N (for aNything). A gene represents a sequence too, so doesn't negate the fact that it still has the aforementioned alphabet. ('GATTACA' is a sci-fi movie name that has these four letters). The Protein alphabet, on the other hand, comprise 20 letters.
Often IO problems start with the sequence having non-canonical letters, punctuation, or whitespace left in from reading in the sequence, so perlretut and perlop for help on regexes, and substitutions (s///) which are one way of checking for / replacing naughty characters.
Working with either type of sequences (DNA or protein) can involve:
Modules of Interest: (Module Reviews Needed)
There are also a host of extremely useful modules for general data handling.
Further Insight:If you intend to develop libraries in BioPerl, a grip on Object Oriented Programming is mandatory.
So now you have a good start on the Perl side, but want some data to play with? Much of bioinformatics revolves around the integration of large datasets in an attempt to draw out relationships, ultimately giving biological meaning to observed phenomena.
Fortunately, biology naturally lends itself to informatics, with known hierarchies and interrelations mirroring OO structuring, and the sheer abundance of data makes the challenge interesting. Here are a few possible sources of publicly available data:
Currently available resources:
For both biologist and programmers, here are a few resources for those of you who want to read more.
Any other recommendations are welcome!
Nodes of interest
Many great discussions have taken place in the monastery and this is just to highlight a few of the lessons learned there. Super Search will hopefully lead you to more specific answers too!
Lastly, if you are really interested, there are several good forums / sites that advertise jobs within bioinformatics and related science. Personally, I have found job-hunting to be no easy task, so here is a few of the better things I have stumbled upon:
Further Insight: If you can suggest ideas, invite/offer Modules review or share code addressing a certain aspect of BioPerl feel free to come forward with it.
Mediation / Tutorial i am thinking about RFCing:
Sharing variables and filehandles between processes
Perl cloud computing... (something for the future!)
End of meditation...
Very confusing mudules:
I find for most situations, rolling your own is the best approach:
Some of my favourites
One of the nicer explanations out there : Re: Extend a module
Apart from being node 19000, i just think this is cool : Re: Perl Spots
Others that I will look at again:
Uncategorised as yet:
Re: recursive algorithm for nested data structures
Re: Real Life Perl Exercises
Re: how much memory each Perl variable uses
Re: Dealing with errors in subroutines
extract phrases of n-words length
creating and printing a sliding window
Searching in binary files
Why is this code so much slower than the same algorithm in C?
Balancing Logging and Code Readability
boolean calculation with very large data
Prevent direct acces to object's attributes
Use strict warnings and diagnostics or die
How can I visualize my complex data structure?
module for cluster analysis
Net::Amazon::EC2 loses pound-shebang when base64'd
Graph Algorithm Package
[threads] Sharing object through threads
Managing a graph with large number of nodes
Shared Memory using IPC::Shareable - Can't use an undefined value as an ARRAY reference
forking with Storable and IPC::ShareLite