Hi, all ye Monks.

I'm using Module::Build and have done for years. One thing seems odd to me: If you don't have a required module, it displays a nice message, but it still exits with success. This means you can't do this:

perl Build.pl && ./Build && ./Build test && sudo ./Build install
[download]

... and have it do what you'd think it would do, namely stop after the 'perl Build.pl' stage noticed that the prerequisite modules aren't there.

I'm guessing either a) I'm doing something wrong and it should work that way or b) I'm doing something wrong by trying to use it that way.

So, if I want to install Module::Build packaged modules via a shell script, what's the best way to tell the shell script if the prerequisite modules are or aren't installed?

Thanks!
--Pileofrogs

Hi,

I am trying to make DNS filter where I can send 'A' entry for some of the requested domains and redirect the request to a real DNS server if no domain matched.

I am stucked with the fact that tools like 'nslookup' send un-understandable characters to my perl script so I am unable to match anything and stopped working on the code until I get some help.

Here is my code:

use IO::Socket;
my $server = IO::Socket::INET->new(
            Proto      => 'udp',
            LocalPort => 53,            
            );

die "Couldn't setup server: $@" unless $server;

while ( $server->recv(my $data , 1024, my $flag ) ) {
    print "Got: $data\n";
}
[download]

I am having an issue where mime lite is not including the attachment when it sends out the file. The main part of the script creates the attachment and is quite involved. Simply it creates a tab separated file since some of the data elements contain commas. The script runs and mime lite sends the message but doesn't include the attachment.
Here is the section of my script for MIME::Lite

my $msg = MIME::Lite->new(
     From    => $from,
     To      =>  $lobmail,
     Subject => 'DUA '. $lobn .' Ticket Activity Report',
     Type    => 'multipart/alternative',
     );
$msg->attach (
     Type => 'TEXT',
     Data => "Attached is the ticket activity report for $lobn",
       );
$msg->attach(
      Type => 'text/plain',
      Encoding => 'base64',
      Path => '/home/eric/tmp/lobreport.tsv',
      Filename => 'LobReport.tsv',
      Disposition => 'attachment'
) or die "Unable to add attachment: $!\n";
$msg->send();  
[download]

Hello Monks!!! The Win32::Scheduler module works fine in local machine to schedule a perl file. But when trying to connect to a remote go grid machine, it doesn't, and creates the job in local machine itself. There is a method - SetTargetComputer() where the host machine name is given. But i'm unaware where to give the Go grid machine's password. Here is my code

$scheduler = Win32::TaskScheduler->New();
$scheduler->SetTargetComputer("173.1.105.147");
$tsk="my_task";
%trig=(
       'BeginYear' => 2010,
       'BeginMonth' => 02,
       'BeginDay' => 9,
       'StartHour' => 13,
       'StartMinute' => 06,
       'TriggerType' => $scheduler->TASK_TIME_TRIGGER_DAILY,
       'Type'=>{
               'DaysInterval' => 1,
       },
);

$scheduler->Activate($tsk);
foreach $k (keys %trig) {print "$k=" . $trig{$k} . "\n";}
$scheduler->NewWorkItem($tsk,\%trig);
$scheduler->SetApplicationName("cmd.exe");
$scheduler->SetAccountInformation('usrname','pwd'); #username and pwd 
+for the particular task
$scheduler->Save();
$scheduler->End();
[download]

I just want to schedule a job in remote windows machine using this module. Thank You Monks!!!!!!

I want to access the Values of User_Preferences hash from ::main package.

I need perl monks advice in this

 
#File name: ConfigFileRead.pm
package ConfigFileRead;

use strict;
use warnings;

my %User_Preferences;
my $configFile;
print("Printing from ConfigFileRead.pm...\n");
print("\nIn namespace: ",__PACKAGE__,"...\n");

sub readConfigFile 
{
    print("Printing from readConfigFile()...\n");
    
    my $configFile=shift;
    open CONFIG,$configFile || die "Missing input file name.\n";
    
    while (<CONFIG>) {
        chomp;                  # no newline
        s/#.*//;                # no comments
        s/^\s+//;               # no leading white
        s/\s+$//;               # no trailing white
        next unless length;     # anything left? - skip if the the str
+ing is not having any length.. if the length of the default string un
+der process is having length greater than 0 then proceed with next st
+ep
        my ($var, $value) = split(/\s*=\s*/, $_, 2);
        $User_Preferences{$var} = $value;
    } 

    close(CONFIG);

}

 1;  # don't forget to return a true value from the file
[download]

Config.ini file

#Filenname: config.ini

# set class C net
$NETMASK = '255.255.255.0';
$MTU     = 0x128;
# Brent, please turn on the modem
$DEVICE  = 'cua1';
$RATE    = 115_200;
$MODE    = 'adaptive';

[download]

Main file : main.pl

use strict;
use warnings;

require ConfigFileRead;

&ConfigFileRead::readConfigFile("config.ini");

print("\nIn namespace: ",__PACKAGE__,"...\n");

# How to access the contents of %User_Preferences ?
# Need help

[download]

Dear Monks,

would you recommend to use Catalyst in a context where directed attacks against a web application seem likely?

To be more specific, I am going to create an application which will accept file uploads from users; then, the application will start working on the files, and eventually inform the users that the (transformed) files are ready for download. The contents of these files will be highly sensitive; attempts of industrial espionage to access them (via breaking into the system that I am going to build) would not seem unlikely.

The application shall be embedded into a nice web GUI, and, of course, there will be lots of necessary administrative functions, e.g. changing email addresses, activating or deactivating users etc. To use something like Catalyst for UI, authentication and basic managements tasks would save me the time to write another template- and data-management system.

What would you recommend? Is Catalyst mature enough to be used in such a setting? If not, is there another system that may seem suitable? Or is the best solution to code everything up manually?

Thanks,

 use A;
 use B;
 use C;
[download]

I'd like to create a module (say ABC) which essentially performs the above use statements but runs the modules' import() routines in the module from which ABC is use'd. The latter part is the grotty part I'd like to avoid having to write (without it there's not much point in the whole exercise).

I'm pretty sure that there's a module on CPAN which handles this sort of thing and makes it effortless. I think I saw something closely related to this in the daily CPAN modules list a few months ago, but I can't find it.

Any help is appreciated.

Thanks, Diab

Here's a cheesy (but accurate) example of the technique I am talking about:

#!/usr/bin/perl -w
use Data::Dumper;
use strict;

package Apple;

sub new {
    my $obj  = shift;
    my %self = ( color => 'red' );
    bless \%self, $obj;
}

sub serve {
    return 'sliced';
}


package Orange;

sub new {
    my $obj  = shift;
    my %self = ( color => 'orange' );
    if ( @_ ) {
        $self{apple} = shift;  ## other objects
        $self{shape} = shift;
    }
    bless \%self, $obj;
}

sub serve {
    my $self = shift;
    return ( $self->{apple} ) ? $self->{apple}->serve() : 'squeezed';
}


package main;

my $red = new Apple;
my $yel = new Apple;
$yel->{color} ='yellow';  ## or is it better to pass to the new() for 
+initialization.

my $foo = new Orange();
my $baz = new Orange( $red );  ## is it wise to pass an object ref vs.
+ "use'ing" the package


for my $obj ( $red, $yel, $foo, $baz ) {
    print 'served: ', $obj->serve, "\n";
}

print "\n fruit stand: \n"
    , Dumper($red)
    , Dumper($yel)
    , Dumper($foo)
    , Dumper($baz)
    , "\n";
[download]

Thanks for any guidance or recommendations!

Is this a job for a regular expression? I want to match a character followed by anything else except for that first character. I've tried it with backtraces, but that obviously not possible

$str = '...+...';
$str =~ /(.)[^\1]/;
# And now I had in mind to have $& containing '.+'
[download]

RFC (Perl and Bioinformatics) BioLion * biohisham

Audience: Perl Monks and Everyone with an interest.

Introduction & Justification: Perl has come to cover many areas of IT and has been dubbed the 'glue' for that matter. Perl has also contributed to Biology, big time, it saved the human genome project and not only that, it has continued to be the mainstay of much bioinformatics munging and analysis, playing no small part in the burgeoning ‘*omics’ sciences.

The increasing number of bioinformatics related Perl problems that seem to be coming up in the Monastery, and the confusing and disparate resources available on the internet contribute a great deal to making BioPerl fearful or at least "perl-plexing"…

PerlMonks plays a great role in the evolution of Perl, it has encouraged many to join up the community and exchange knowledge in a place of utmost cohesion between its members and thus BioPerl coders can be equally encouraged to participate and share their knowledge and code.

Description - what this is and isn't:

So, while this isn't intended to do the job of the extensive BioPerl docs, or many reference points out there, it will hopefully be a starting platform for those looking to delve deeper into using Perl in bioinformatics related tasks and also assisting Monks in becoming more accessible to BioPerl questions: Facilitating the back and forth that makes Perl and the Monastery so special.

It is also to highlight the interesting problems that bioinformaticians have to deal with - not all are BioPerl related(!) and can often involve huge, diverse datasets. And we hope that these sorts of challenges will tempt a few talented programmers to get involved.

Tips on posting bioinformatics type questions in the Monastery:

Please go through the following whenever you notice that your question or parts thereof don't look like how you expected after you have hit the "preview" button and remember; a well formulated question will garner better and quicker response.

• How do I post a question effectively?
• Markup in the Monastery
• How do I compose an effective node title?
• Perl::Tidy and Perl::Critic
These last two will help make sure your code is presentable, and their use should be considered Good Coding Practice (see below).

How do I post a question effectively? is particularly relevant for specialist, such as bioinformatics, questions. Here we try to highlight the importance of well formulated questions:

"I have a DNA sequence that I want to BLAST and I tried Bio::Tools::Run::StandAloneBlast but it did not work how can I do that? "
OR
“I am trying to translate my coding sequence, I can work out the tRNA lookup table, but I can’t break up the sequence into codons - any ideas?”

These sorts of questions invite down-voting and confuse monks and their response would be either trying to extract words from you to get you to explain it better, make wild guesses that would confuse you the more or ignore your question rather than BLASTing on you. Better to think about what you are trying to actually do and think about how this is a Perl problem.

This leads to an important point - often overlooked - of providing test data (just enough - 3 cases of input, not the whole file, and if it is in a particular format - say which or provide an example of its layout !), and if you are really stuck, what output you want. This greatly helps people grasp what you are doing and also test any code they produce.

I am trying to convert a string (a DNA sequence) into a series of three-letter sub-strings– to do that I have written the following code but I failed to make the substrings overlap by moving one letter at a time from the original sequence in the forward direction.

#original seq

accgttac

#required output
acc         #first substring
ccg
cgt
tta
tac         #fifth substring
[download]

Here is my non-functioning code

#!/usr/local/bin/perl
use strict;
use warnings;

for(<DATA>){
              print substr ($_,0,3),"\n" ;

              }

__DATA__
accgttac
[download]

Now that seemed like an ideal question, clear wordings, examples of input and desired output and the code involved if any so that testing the respondents code on the provided data is made possible.

Finally, always check to see if your problem hasn’t been answered before - learn to love Super Search and google… There are also links to discussions in the Monastery that may be of interest!

Good coding practice:

Many bioinformaticians are new to coding and can be guilty of certain malpractices, so your code should be readable, self-descriptive and properly indented and commented. Good coding practices are critical point checks, they can alert you to avoid potential errors, dangerous coding behavior and enable you reduce debugging time and increase code efficiency and re-usability. And as always, use warnings and strict, check for errors etc… because you never know what this code could be used for! Maybe some IO error means that a potential cancer biomarker is missed (extreme example, but point remains!).

Also - remember that posted nodes can be edited at a later point to encompass suggestions, changes to code, what course was finally decided etc... Remember that it is considered good form to mark any changes with ‘Update:’.

Tips on Answering BioPerl Questions:

Typical problems and solutions:

A frequent problem is the installation of BioPerl, this in itself is not difficult if certain caveats are attended to, if you are familiar with Installing Modules then you are good to go. Note that there is some difference between the BioPerl Installation Requirements on Linux and BioPerl Installation Requirements on Windows and that not all of BioPerl is available on Windows hence you need to add the following repositories to the ActiveState PPM manager.

• BioPerl-Release Candidates.
• BioPerl-Regular Releases.
• Kobes.
• Bribes.

If anyone can contribute tips for other methods (e.g. Strawberry Perl and cpan?), it would be much appreciated!

The BioPerl suite of modules revolves around sequence acquisition, parsing and retrieval from public databases and automating various tasks related to studying these sequences BioPerl HOWTOs. Think this is simple? Think again - CODE.GOOGLE.COM tells us there are 3,666,478 lines of code to get your head round!

A sequence is just a text string in a certain format (this format is described in the beginning of the text file containing that sequence) that represents either a gene or a protein, the alphabet of the sequence with regard to genes is but a combination of four letters (ACGT) and sometimes U (replaces T) and N (for aNything). A gene represents a sequence too, so doesn't negate the fact that it still has the aforementioned alphabet. ('GATTACA' is a sci-fi movie name that has these four letters). The Protein alphabet, on the other hand, comprise 20 letters.

Often IO problems start with the sequence having non-canonical letters, punctuation, or whitespace left in from reading in the sequence, so perlretut and perlop for help on regexes, and substitutions (s///) which are one way of checking for / replacing naughty characters.

Working with either type of sequences (DNA or protein) can involve:

• sequence comparison (Sequence Alignment): two or more sequences are compared against each other to evaluate how similar they are, and where they are similar.
• Sequence manipulation - in-place modification, concatenation, reverse complimenting, etc…
• BLASTing (Database Search for similar sequences):

Modules of Interest: (Module Reviews Needed)

• Bio::Seq.
• Bio::SeqIO to access sequence files and perform I/O operations
• Bio::DB and Bio::DB::Query to either retrieve a single sequence from a database via its ID or ACCESSION Number or retrieve multiple sequences at a time by query objects containing search terms and criteria specific to the database under investigation.
• Bio::Tools::Run::StandAloneBlast to run the local BLAST version of the famous sequence analysis program.

There are also a host of extremely useful modules for general data handling.

• Math::Random Often used a control.
• Data::Validate Data munging tasks.
• Other Statistical modules and R Modules.
These last ones also highlight that bioinformatics is ultimately about ‘getting an answer’!
• Benchmark, Parallel::ForkManager, Parallel::Forker and Devel::NYTProf.
For those interested in getting the answer slightly faster!

Further Insight:If you intend to develop libraries in BioPerl, a grip on Object Oriented Programming is mandatory.

Got Data?

So now you have a good start on the Perl side, but want some data to play with? Much of bioinformatics revolves around the integration of large datasets in an attempt to draw out relationships, ultimately giving biological meaning to observed phenomena.

Fortunately, biology naturally lends itself to informatics, with known hierarchies and interrelations mirroring OO structuring, and the sheer abundance of data makes the challenge interesting. Here are a few possible sources of publicly available data:

• ENSembl
Data can be accessed using either cpan tools or the perl API
• http://www.biomart.org/
Providing access to a vast number of other databases.
• NCBI EUtilities
Again Perl based and a source of a huge amount of information. They also have a lot of documentation which explain about what the data is...

Currently available resources:

For both biologist and programmers, here are a few resources for those of you who want to read more.

• BioPerl Official Site and HOWTOs.
This is obviously a major hub of information, and with the BioPerl API, a huge source of power. From the outside though, it can be intimidating.
• Beginning Perl for Bioinformatics by James Tisdall. (Review Required).
• http://www.amazon.co.uk/Mastering-Perl-Bioinformatics-James-Tisdall/dp/0596003072/ref=pd_bxgy_b_img_b by James Tisdall
• Bioinformatics Computing by Bryan Bergeron
• Developing Bioinformatics Computer Skills by Cynthia Gibas and Per Jambeck
• BLAST by Ian Korf, Mark Yandell and Joseph Bedell
• Bioinformatics in the Post-Genomic Era Genome, Transcriptome, Proteome, and Information-Based Medicine by Jeff Augen
All of which (and possibly more) are available on O'Reilly's Safari ( thanks to planetscape ).
• Using Perl for Bioinformatics (PDF) (Review Required).
• A Quick Guide for Developing Effective Bioinformatics Programming Skills.
• James Tisdall on Perl.com
• OSC tutorial
• Data Munging on Perl.com
• http://www.bioinfobooks.blogspot.com - others, with reviews
• Courses too…Cold Spring Harbor's PFB
BioLion: this was my first major experience of bioinformatics, beyond university/college short courses, and was superb. The focus is very problem-oriented, and has a heavy emphasis on teaching you to teach yourself, which in the long run is the most important lesson.

Any other recommendations are welcome!

Nodes of interest

Many great discussions have taken place in the monastery and this is just to highlight a few of the lessons learned there. Super Search will hopefully lead you to more specific answers too!

• BioPerl.
• Job Field - Bioinformatist.
• perl's long term place in bioinformatics?.

Jobs

Lastly, if you are really interested, there are several good forums / sites that advertise jobs within bioinformatics and related science. Personally, I have found job-hunting to be no easy task, so here is a few of the better things I have stumbled upon:

• http://www.bioinformatics.fr/jobs.php
• http://www.newscientistjobs.com/jobs/browse/biology_bioinformatics.htm - also naturejobs, sciencejobs etc...
• http://www.123genomics.com/jobs.html

Further Insight: If you can suggest ideas, invite/offer Modules review or share code addressing a certain aspect of BioPerl feel free to come forward with it.

Thanks to planetscape, erix, marto and GrandFather for their contributions

Update: Updated to cover comments as of 8th January 2010

OK, so I have an object. The object keeps a reference to another object and passes unknown methods to that object. Sounds reasonable? The AUTOLOAD looks like this:

sub AUTOLOAD {
    (my $sub = $The::Class::Name::AUTOLOAD) =~ s/^.*:://;
    my $db = shift;
    $db->{dbh}->$sub(@_);
}
[download]

Wrong. Under Unix all subtests pass and then the test script "creates core". Every. Single. One.

I turn the XS module's tracing on, everything looks OK. I add a plethora of debug prints into the XS modules DESTROY method, the last of them at the very last line of the function. Everything gets printed, then it crashes.

I undef the outer object at the end of a test script and put a debug print above and below. The first one is printed, the other is not. OK. So it must be something with the object destruction, but what??? There's nothing special in the outer object, no need to destroy anything explicitely, it doesn't even have it's own DESTROY method!

OK, so the XS module worked with its own (short) tests, maybe something inside got borked by the many tests of the outer one. Let's exit() the test script sooner. Crash. Let's exit() it just after the object creation. Crash! OK, OK, OK, let's undef the attribute explicitely from outside the object.

(in cleanup) Can't call method "DESTROY" on an undefined value at ... 
+at line ...
[download]

• 1. Save the template below as "pod.tmpl".
• 2. Run: postgresql_autodoc -d mydb -l . -t pod
• 3. post-process the pod file like this :

mkdir pod
perl -MIO::File -lpe '
    s/^_CUT: (.*)$// and do { *STDOUT = IO::File->new(">pod/$1"); };
    s/^_DB: (.*)$//  and do { $_ = qx[psql -c "$1"]; s/^/ /gm;    };
    ' mydb.pod
[download]

<tmpl_loop schemas><tmpl_loop tables>
_CUT: <tmpl_if view_definition>view<tmpl_else>table</tmpl_if>_<tmpl_va
+r table>.pod
=head1 NAME

<tmpl_var table>

=head1 DESCRIPTION

<tmpl_var table_comment>

=head1 COLUMNS

<tmpl_loop columns> <tmpl_var column> (<tmpl_var column_type>) - <tmpl
+_var column_comment>
</tmpl_loop>

=head1 SCHEMA

_DB: \d+ <tmpl_var table>

=cut

</tmpl_loop>
</tmpl_loop>
[download]

In 2007, I read the following task on an online puzzle site (exercise 5 here).

Cut a 6 times 6 chessboard to 18 black and 18 white squares. Assemble them to a 6 times 6 board in such a way that each row and column has 3 black and 3 white squares, every row is different, and every column is different. How many boards can you get this way? The rows and columns of the board are numbered, so boards that differ in rotation or flipping count as different.

As discussed on the J wiki, this puzzle is quite easy to solve with brute-force.

In 2008, we decided to post a larger variant of this puzzle on the first round of the Challenge24 contest. The (contest webpage has the full problem set downloadable, this is problem F.) The task was basically the following.

A seven by seven cell table has to be filled in such a way that each row and each column must have exactly two cells with a 0, two cells with a 1, and three cells with a 2 in them; no two rows can be exactly the same; and no two columns can be exactly the same. Given the contents of the first two rows, compute the number of ways the rest of the table can be filled.

With current computers, it's almost impossible to brute force this task by enumerating all possible solutions.

There is one simple observation that cuts down quite some time though.

However, brute forcing takes too much time even after knowing this trick. The first working perl program I wrote takes about twenty minutes on one input. Such a long running time would be unacceptable, as we want to give ten sample inputs, and the contestants would have five hours for the whole contest (reading the tasks, development, running the program, submitting the answer, all this for all eight tasks, with only three people in a team). I didn't despair though, for I already knew that that solution can be optimized a lot.

My final solution is more complicated and much faster: it runs on any one input in at most six seconds (that was back in 2008, computers are twice as fast now). That's fast enough, so we could post this problem. (In fact I was quite surprised on how much faster the program became.)

The remainder of this post explains how my solution works, and shows the code too. I've spoilered this in case you want to try to write a solution yourself.

There is one more notable point that surprised me when I found out during the preparations. This is that there are only fourteen (14) significantly different input data you can give in this problem. (This didn't help the contestants though, for we gave ten significantly different inputs, and this is the kind of contest where the contestants run their programs on their own computers and submit only the result.) There are thirteen possible outputs, for two of the inputs have an equal output for a reason unknown to me.

You may want to compute these fourteen inputs (pairs of two lines): this is much easier than solving the original problem. Here are them in canonical form in case you want to check.

If you have a premium rapidshare account it is real easy to make a batchloader using wget. You only need to insert your premium cookie. Cut and paste the rs urls into a file and start downloading.

I seek employment.
Is Perl the answer? Perhaps.
Email me with work. 
[download]

the snow hides objects
a layer of abstraction
things seem functional
[download]