a_mere_kat has asked for the wisdom of the Perl Monks concerning the following question:


I have a problem with my "fruity loops" they are not so fruity and sadly seem to loop forever.

I am having some problems embedding loops. I get this problem continuously and I don't know why.

Ok here is the problem I need to compare selected fields from two flat files.

I have extracted the one field I want from each line of the two files and placed it into two arrays.

Then using an if statement inside a for loop embedded in another for loop. This set up should compare each element of one array to every element of another array. The results being output to two flat files. respectivly matches and non matches. Instead of producing what I want it to it just goes into an infinite loop. I want to know why this occurs and how I can get over this problem? If anyone sends me a RTFM advisory they can stuff my output in there I/O and die. I am posting this because I would like help not abuse please.

Below is the code I have written to perform the task Feeble as it may be it should do the job. Would someone out there please be kind enough to point out what may be obvious to them but is sadly not obvious to freshmeat like me. Thankyou for your time.
for ( $i = 1 ; $i <= $#line ; $i++ ) { $k = $line[$i]; for ( $v = 1 ; $v <= $#angel ; $v++ ) { $j = $angel[$v]; if ( $k =~ m/($j)/i ) { push ( @a, $v ); } else { push ( @b, $v ); } } }

update (broquaint): added <code> tags

Replies are listed 'Best First'.
Re: To infinity and beyond: Embedding loops
by antirice (Priest) on Jul 16, 2003 at 10:14 UTC

    If anyone sends me a RTFM advisory they can stuff my output in there I/O and die.

    Well aren't you a ray of sunshine? :P

    Where to begin? Well, it shouldn't loop forever. Although, I expect on a large @line and a large @angel it should seem as if it's taking forever. For instance, suppose both are 10_000 elements long. That means you would have 100_000_000 comparisons. When everything is finally finished, @b will be absolutely enormous. For illustrative purposes, let's suppose @line = @angel and that every element in @line is unique. After the loop, @b will contain scalar(@line)**2 - scalar(@line) elements. Needless to say, that's a lot. Also, you're aware that you're pushing indexes into the array, right? You're skipping index 0 on both arrays with your loops. Your comparison checks to see whether or not $k contains the regex contained in $j. This can be a nightmare from a maintainence standpoint since your program will die anytime $j contains a string with an unmatched parenthesis or left bracket. I doubt this is what you really want. You should at least use \Q and \E if you want to see whether or not $k contains the string in $j. If you just wanted to see if they're equal, you should just use lc($k) eq lc($j). Of course, if you want to see whether or not they're actually equal to one another, there's a better way to do it. Here's some code that should be a bit faster and make @b and @a contain what you actually want (except you get to fill in some blanks):

    # this part is just so you don't get an "requires explicit # declaration" error my @lines = getLines(); my @angel = getAngel(); my @a;my @b; #'s some magic # we create a hash whose keys come from @lines. Why # do this? Since checking an element in a hash is O(1) # it is much faster than searching the array @lines # ...also, on a side node the map is so we do case # insensitive checking when we look for exists my %hash; @hash{map(lc,@lines)} = (); # ok, now we create a loop that will set $_ to the numbers # from 0 to $#angel. We could also do for (@angels) to get # the actual values, but since you pushed the index of # matches into @a and @b, I opted for this option instead. for (0..$#angel) { # ok, since we didn't actually set each hash element to any # value, we can't compare it against anything...BUT, we can # check to see whether or not it exists if (exists $hash{lc($angel[$_])}) { # hey, we have a match, take care of it } else { # darn...not a match :-/ } }

    Since I don't want to stuff your input into my I/O and die, I will not tell you to RTFM. Also, if you have time, don't read perlre, perldata or How can I find the union/difference/intersection of two arrays?.

    Updated: Added comments to the code so it is more understandable to a fellow monk who asked for clarification of how exactly it works.

    The first rule of Perl club is - use Perl
    ith rule of Perl club is - follow rule i - 1 for i > 1

Re: To infinity and beyond: Embedding loops
by Skeeve (Parson) on Jul 16, 2003 at 10:01 UTC
    No infinite loop here. But is this really what you wanted?
    use Data::Dumper; my @line= qw(a b c); my @angel=qw(1 2 b); for ( $i = 1 ; $i <= $#line ; $i++ ) { $k = $line[$i]; for ( $v = 1 ; $v <= $#angel ; $v++ ) { $j = $angel[$v]; if ( $k =~ m/($j)/i ) { push ( @a, $v ); } else { push ( @b, $v ); } } } print Dumper(\@a, \@b);
    prints out:
    $VAR1 = [ 2 ]; $VAR2 = [ 1, 1, 2 ];
    BTW: Arrays start at 0 not at 1...
Re: To infinity and beyond: Embedding loops
by tos (Deacon) on Jul 16, 2003 at 10:09 UTC

    your problem isn't really clear to me. Perhaps it will be helpful if you show examples of your inputfiles and an example of your expected output.

    Be that as it may, your momentary shown code didn't do an infinite loop.

    greetings, tos

Re: To infinity and beyond: Embedding loops
by a_mere_kat (Initiate) on Jul 22, 2003 at 03:43 UTC
    Indeed a big shining ray of sunshine ;). Sorry for the defensive behaviour I have asked for assitance with tasks on other similar sites and some people (I am sure you know the sort) were putting it plainly somewhat less then helpfull. Reading it again it does make me sound (justifiably) like a bit of a fool

    Thankyou very much for all your assistance. You were quite correct my code was in fact not an infinite loop. However it did contain 2 very long arrays. The first array contained some 25000 elements and the second a rather large 46000 elements.

    As a result the script that I wrote took 4 hours to run. Your own which you supplied me with I notice took about 5 seconds!!!! Your Perl kung-fu is mighty indeed Mr Antirice. Thankyou very much for your kind assistance and also the assitance of Aquarium and Skeeve.

    There is one more twist in the tail. My coding problem was not resolved before I was sent the correct data file. It is a deeply embarressing thing to admit but I had been given the wrong data to attempt a comparison on. The files that I was given shared no common IDs. I failed on first principles before I had even begun. I trusted the data I was given. When I did recieve the correct file using your code I was able to resolve the problem very rapidly. For this again I thank you.

      You provided an entertaining piece..

      Developing practice of getting to know the basics of the people you are working with, will be a helpful attempt on your part before screaming. <p

Re: To infinity and beyond: Embedding loops
by aquarium (Curate) on Jul 16, 2003 at 12:09 UTC
    can you please send an excerpt of the files to run this code against?