This is an archived low-energy page for bots and other anonmyous visitors.
Please sign up if you are a human and want to interact.
BioGeek has asked for the wisdom of the Perl Monks concerning the following question:
Hey Monks,
I have a page, which has the following format:
It is a list with (biological) "candidates".
For each candidate, a bunch of information is given, like protein name (in square brackets), a R-score and a GO-score.
For each candidate, I'm interested in it's R-score and protein name. The tricky part is the variable number of occurences of the protein name.
The information can have one of the following formats.
1) R-score and protein name.
2) R-score and more than one protein name.
3) R-score and no protein name.
The code I wrote fetches the R-scores, but I can't come up with a code that matches all possible cases of the protein name.
use strict;
use warnings;
use LWP;
my $R_score = ();
my @protein = ();
my @R_score_protein = ();
$browser = LWP::UserAgent->new();
my $resp = $browser->get(http://www.bork.embl-heidelberg.de
+/g2d/list_hits_disease.pl?U57042:Inflammatory_bowel_disease_7);
my $content_all = $resp->content();
while ( $content_all =~ m{R\-score<\/A>\s=\s(\d\.\d+)\;}g ) {
$R_score = $1;
push( @R_score_protein, $R_score );
Update:To improve readability, I've removed the relevant HTML snipets, and placed them on my BioGeek's scratchpad. |
Re: regex with variable input
by bradcathey (Prior) on Aug 09, 2004 at 20:10 UTC
|
BioGeek, I don't mean to complain, but can simplify your question, a bit, get it down to the basics (I don't want to try to figure out all that HTML). Also, consider <readmore> tags if posting long questions. Thanks, and I look forward to seeing a distilled version.
Update: Escaped my angle brackets per hv
—Brad
"Don't ever take a fence down until you know the reason it was put up. " G. K. Chesterton
| [reply] |
Re: regex with variable input
by spq (Friar) on Aug 09, 2004 at 21:23 UTC
|
First of all, does bork.embl-heidelberg.de offer any sort of batch download, or text based save that could be used?
I'm guessing your looking to do this in a (semi) automated manner, and starting from saving as text from your web browser wont help in the long run, since your scripting the fetch? I also might suggest that you may want to consider contacting the web site maintainers. They may not be thrilled about getting hammered by your script, and may be very open to helping find an easier way to provide this information to you instead.
That being said, I used wget to download the page and wrote this code which seems to capture what you want (this is a quick translation from a command line test, and in no way should be considered production ready or tested).
while (<>) {
chomp;
if(/CANDIDATE (\d+)/){
$cand = $1;
@acc = ()
} # if candidate line
elsif (m{http://www.ncbi.nlm.nih.gov:80/entrez/query.fcgi}
&& m{(\w+)</A>}) {
push(@acc,$1);
} # else if ncbi accession #
elsif (m{R-score</A>\s*=\s*(\S+);}) {
print "$cand\t", join(",",@acc), "\t$1\n";
} # else if R-score, print this candidates info
} # while reading web page
| [reply]
[d/l] |
Re: regex with variable input
by TomDLux (Vicar) on Aug 09, 2004 at 22:12 UTC
|
Collapse the three lines of declaration into one line:
my ( $R_score, @protein, @R_score_protein );
Arrays are by default initialized as empty arrays, so you don't need to set them to (). <em$R_score is a scalar, not an array, so by default it is undef, which acts like the empty string if you try to concatenate to it; but you should initialize it to "" if it might be read before it is assigned.
In any case, you don't need $R_score, since you only use it to transfer a value from $1 into @R_score_protein. Delete the variable and assign $1</em, into the array.
my ( @protein, @R_score_protein );
...
push( @R_score_protein, $1 );
Does 'strict' not complain about that unquoted string? In any case, get that URL out of the code and into a configuration variable at the beggining of the file.
Better yet, provide for the user to override the URl at run time:
if you can process proteins and R scores separately:
while ( $content_all =~ m{R\-score<\/A>\s=\s(\d\.\d+)\;}g ) {
push( @R_score_protein, $1 );
}
while ( $content_all =~ m{\[(NP_\d+)\]}g ) {
push @protein, $1;
}
Better yet, you could move the regular expressions into 'local' variables or 'configuration' variables:
my $protein_re = qr|\[(NP_\d+)\]|;
my $score_re = qr|R-score</A>\s=\s(\d\.\d+)|;
while ( $content_all =~ m{$protein_re} ) .....
while ( $content_all =~ m{$score_re} ) ....
Tidier, more readable, more maintainable.
I suspect you want to associate the protein codes and the R scores, which the version above doesn't provide. You need to break the input into distinct lines, which I'll arrange to have happen magically in the break_into_lines() routine:
| [reply]
[d/l]
[select] |
Re: regex with variable input
by graff (Chancellor) on Aug 10, 2004 at 00:47 UTC
|
I'm interested in it's R-score and protein name. The tricky part is the variable number of occurences of the protein name. ... So I want to fetch ..."
And on fetching the various chunks, what do you need to do with them? How do you want to organize, manipulate or summarize them? Is each "candidate" uniquely identified by the "candidate number"? If so, do you want a simple listing that shows, e.g. candidate-number R-score [protein-name ...]
What you intend to do with the data is an important issue, because it should determine how you set up the appropriate data structure for the task. For example, if the candidate numbers are unique and sequential, you probably want an array of hashes:
my @candidates; # should use a hash instead, if
my $c_id; # the numbering has lots of big gaps
while (<>) {
if ( /CANDIDATE\s+(\d+)/ ) {
$c_id = $1;
}
elsif ( /DNA\s+.Note.\s*(\S*.*)/ ) {
my $proteins = $1;
my @p_set = ();
while ( $proteins =~ /\[(NP\s+\d+)\]/g ) {
push @p_set, $1;
}
$candidates[$c_id]{proteins} = [ @pset ];
}
elsif ( /R-score\s*=\s*([.\d]+)/ ) {
$candidates[$c_id]{rscore} = $1;
}
}
If you're still learning about data structures and references, I'm sure the relevant man pages will supply any other details you may need (perlreftut, perldsc). Use Data::Dumper as well, of course. | [reply]
[d/l]
[select] |
|
|
