Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

Trying to make a string generated like google

by ultranerds (Pilgrim)
on Jul 20, 2009 at 10:31 UTC ( #781591=perlquestion: print w/ replies, xml ) Need Help??
ultranerds has asked for the wisdom of the Perl Monks concerning the following question:


I'm trying to make a string show up like:

Some test bla bla .... and now we carry on, with more keywords...

Basically, the string would look something like:

Some test bla bla. We wanna add some more content in here, just as an example - and now we carry on, with more keywords etc

Here is my current code:

my $keywords = q|fait une petite St-Laurent|; my @keywords = split / /, $keywords; $text =~ s|<!--start_tag-->\s*.*?<!--end_tag-->||g; #remove all th +e citations $text =~ s|&lt;|<|g; $text =~ s|&gt;|>|g; $text =~ s|<\s*.*?>||g; #remove any html $text =~ s|\[\s*.*?\]||g; #remove any bbcode tags my $strings; use Data::Dumper; my @split = split / /, $text; my $number = 0; my $hash_count = 0; foreach (@split) { $back .= "$_ "; if (length($back) >= 250) { # print qq|Adding to: \$string->{$number} \n|; $strings->{$number}->{value} = $back; $strings->{$number}->{count} = get_word_count_match($back) +; $back = undef; $number++; } } print Dumper($strings); sub get_word_count_match { my $text = $_[0]; my $total_words_found; foreach (@keywords) { my $count =0; $count++ while $text =~ /\Q$_/g; $total_words_found = $total_words_found + $count; } return $total_words_found || 0 ; }

What this does, is generate a hashref, like so:

user@serverdev admin $ perl test.cgi $VAR1 = { '1' => { 'count' => 3, 'value' => 'large. La température de l\'eau durant +l\'été ne dépasse pas 3 ou 4°c et lorsque le vent se met de la partie + vaut mieux être bien habillé... Cet été, j\'ai fait une petite excursion en kayak de mer dans la régio +n des Bergeronnes (à l\'Est de Tadoussac). ' }, '0' => { 'count' => 2, 'value' => ' Salut Memphre, Je ne connais pas les dates de fin des excursions pour aller voir les +baleines dans le St-Laurent. Mais je ne crois pas qu\'une ballade en +bateau au large en octobre soit très agréable. Déjà, durant l\'été, i +l peut faire très froid au ' }, '2' => { 'count' => 0, 'value' => 'Nous avons vu plusieurs baleines: rorqu +als communs, marsouins et bélugas. C\'est vraiment impressionnant de +voir les baleines de si près! Entre l\'excursion en bateau ou celle e +n kayak, je te conseille fortement celle en kayak. L\'expérience est +beaucoup ' } };

Now, this is exactly what I want.

After this, I need to then grab the "phrases" which has the highest value for "count" .. in this case, it would be 1 and 0.

Can anyone suggest to me how I could accomplish this?

I know I could enter the values into a mySQL table, and then do an ORDER BY to get the 2 highest values - but this is going to be run on about 20 strings per page, so I would imagine that would be a little slow (especially seeing as we are doing about 2 million+ page views a week, and about 200,000 searchs =))



Comment on Trying to make a string generated like google
Select or Download Code
Re: Trying to make a string generated like google
by davorg (Chancellor) on Jul 20, 2009 at 10:39 UTC
    my @sorted_phases = sort { $b->{count} <=> $a->{count} } values %$strings;

    You'll then have an array of your hashes ordered by the "count" value. The ones with the highest count will be at the start of the array.


    See the Copyright notice on my home node.

    Perl training courses

      OMG - your a star! I've been trying to get something like that to work for ages, and you did it in one line - thanks, it doesn't *exactly* what I need :)



      I'm trying to do something a bit more complex now:

      my @sorted_phrases = sort { $b->{sort_number} <=> $a->{sort_number +}, $b->{count} <=> $a->{count} } values %$strings;

      ..instead of just:

      my @sorted_phrases = sort { $b->{count} <=> $a->{count} } values %$strings;

      Basically, what I need to do - is :

      1) Sort by the value "count" as they were done before
      2) If the 2nd array element has a value of "0" for "count", then it will order them by sort_number thereafter.

      I've tried all kinda things - including this messy bit of code - but I can't seem to get it working :(

      my @sorted_phrases_2; # my $i = 0; foreach (my $i = 0; $i <= $#sorted_phrases; $i++) { local $_ = $sorted_phrases[$i]; if ($i == 1) { print "BLA BLA XXX - \"$_->{count}\" " . Dumper($_); if ($_->{count} < 1) { my $chosen_id_now = $sorted_phrases[0]->{sort_number} ++ 1; print qq|<br />Current sort was: $sorted_phrases[0]->{ +sort_number} , and new one is $chosen_id_now <br />|; foreach my $tmp (@sorted_phrases) { if ($tmp->{sort_number} == $chosen_id_now) { push @sorted_phrases_2, $tmp; } } } } else { push @sorted_phrases_2, $_; } } @sorted_phrases = @sorted_phrases_2;

      Thanks for any ideas - as I'm stumped :(



        I've just logged on for the first time for almost a year and saw your unanswered question. It's really quite simple.

        my @sorted_phrases = sort { $b->{count} <=> $a->{count} or $b->{sort_number} <=> $a->{sort_number} } values %$strings;

        It's really worth reading the Perl FAQ. See How do I sort an array by (anything)?


        See the Copyright notice on my home node.

        Perl training courses

Re: Trying to make a string generated like google
by moritz (Cardinal) on Jul 20, 2009 at 10:50 UTC

      We are already using Lucine for the searches, as we have a lot of posts (and this has prooved the most effecient for what we need)

      Thanks for the suggestion though =)



Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://781591]
Approved by targetsmart
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (8)
As of 2014-09-17 15:55 GMT
Find Nodes?
    Voting Booth?

    How do you remember the number of days in each month?

    Results (90 votes), past polls