Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

counting words in string

by Anonymous Monk
on Aug 07, 2018 at 13:31 UTC ( #1219990=perlquestion: print w/replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi perlmonks, nice to meet you. I am practicing my regexes and want to know how many times a word in a string occurs but i am not succeeding. However i did wrote a working example but not by using a regex. Here it is:

use strict; use warnings; my %hash=(); my $re = "iowq john stepy andy anne alic bert stepy anne bert andy ste +p alic andy"; my @names = split(/\s+/,$re); foreach(@names) { $hash{$_}++; } foreach(keys %hash) { print "key: $_ value: $hash{$_} \n"; }

Can someone show me a regex to do all this?

Replies are listed 'Best First'.
Re: counting words in string
by haukex (Canon) on Aug 07, 2018 at 15:12 UTC

    There Is More Than One Way To Do It :-)

    The method you showed is fine. If the input string starts getting long, you might have to watch out for memory usage. Here's a version that uses a regex and a while loop to scan the string, without building an intermediate @names array. I'm using the regex \S+ (one or more non-whitespace characters) to match "names", so that this code should produce the same output as yours regardless of the input string. If you need to match only certain characters in the names, you'd have to tell us more about that (Re: How to ask better questions using Test::More and sample data).

    use warnings; use strict; use Test::More tests=>1; my $str = "iowq john stepy andy anne alic bert stepy anne bert andy st +ep alic andy"; my %names; pos($str)=undef; while ($str=~/\G\s*(\S+)(?:\s+|\z)/gc) { $names{$1}++; } die "failed to parse \$str" unless pos($str)==length($str); is_deeply \%names, { alic => 2, andy => 3, anne => 2, bert => 2, iowq => 1, john => 1, step => 1, stepy => 2 };

      Since I'm a not an expert with perl regex, I start digging in the code of haukex with commenting it's original code and with searching a simpler loop.

      Here we go:

      use warnings; use strict; use Test::More tests=>2; my $str = "iowq john stepy andy anne alic bert stepy anne bert andy st +ep alic andy"; my %names; =for comment pos Returns the offset of where the last m//g search left off for the vari +able in question ($_ is used when the variable is not specified). Note that 0 is a valid match offset. undef indicates that the search position is reset (usually due to matc +h failure, but can also be because no match has yet been run on the s +calar). =cut pos($str)=undef; =for comment https://www.regular-expressions.info/continue.html The position where the last match ended is a "magical" value that is r +emembered separately for each string variable. The position is not associated with any regular expression. This means that you can use \G to make a regex continue in a subject s +tring where another regex left off. If a match attempt fails, the stored position for \G is reset to the s +tart of the string. To avoid this, specify the continuation modifier +/c. =cut while ($str=~/\G #start where the last match ended \s* #match 0 to n space char (\S+) #remember any non space char after that and followed by (?: #start clustering of \s+|\z #1 to n spaces or the end of the string ) #end clustering /gcx) { $names{$1}++; } die "failed to parse \$str" unless pos($str)==length($str); test_it (\%names); %names = (); #Takes a new variable #my $str2 = "iowq john stepy andy anne alic bert stepy anne bert andy +step alic andy"; #or reset pos for the original var pos($str)=undef; my $last; while ($str=~/(\w+)/g) { #print $1, " ", pos $str, "\n"; $names{$1}++; $last = pos $str; } die "failed to parse \$str" unless $last ==length($str); test_it(\%names); sub test_it { my $hr_names = shift; is_deeply $hr_names, { alic => 2, andy => 3, anne => 2, bert => 2, iowq => 1, john => 1, step => 1, stepy => 2 }; }
      I have 3 questions
      • Where is the /c modifier documented in the perldoc ? It is given at the end of pos but I couldn't find no other description
      • How comes that I have to remember pos $str in the second loop ? It's undef after the second loop but not after the first one
      • Are the two loops equivalent or will the second one failed in some situation ?

      Cheers

      François

        pos($str)=undef;

        Note: This may seem like overkill in the simple example I showed, but it might become important if the code is copy and pasted into a larger script, where it's possible that a previous regex operation on $str left its pos set (but we want to start matching at the beginning of the string).

        Where is the /c modifier documented in the perldoc ?

        Hm, you're right that the doc seems a little hard to find. There's a pretty good explanation in perlretut's "Global Matching", and a more complex example of the usage of m/\G.../gc in perlop under "\G assertion".

        How comes that I have to remember pos $str in the second loop ?

        That would be the effect of the /c modifier: your second loop ends when the regex fails to match. The regex failing to match also resets pos, except when /c is used.

        Are the two loops equivalent or will the second one failed in some situation ?

        The way you've written it, no, because \w+ is not the same as \S+: the latter will match any non-word characters too, anything except \s characters.

        If I change your loop's regex to /(\S+)/g, you can get the "failed to parse $str" error, namely when there's whitespace at the end of $str. This is because the last match doesn't reach all the way to the end of the string, while the regex I showed does, because it includes a \s+ after the \S.

        The pos==length check makes sense if you want to make sure your loop went all the way to the end of the string and there's no garbage left at the end that we didn't match, but in this case, the only thing that could possibly be left after matching \S+ is whitespace, which we're not interested in.

        searching a simpler loop

        I showed the m/\G.../gc technique because I wanted to demonstrate it - it's a great tool for certain types of parsing (one example in the first paragraph of this post, or this), but as long as the OP just wants to split on whitespace, it's admittedly overkill. If you wanted to simplify the loop, you could say this (others have shown variations of this):

        my %names; while ($str=~/(\S+)/g) { $names{$1}++; }

        But as soon as the matching rule isn't as simple as \S+, things can get tricky. For example, if the rule is \w+, and the input string is " foo bar *", anything that looks for \w+ only will miss the junk at the end of the string.

Re: counting words in string
by Tux (Abbot) on Aug 07, 2018 at 13:45 UTC

    Simplest I could come up with is (more or less the same what you did)

    $ perl -MDP -wE'my%x;$x{$_}++ for "iowq john stepy andy anne alic bert + stepy anne bert andy step alic andy" =~ m/(\w+)/g;DDumper\%x' { alic => 2, andy => 3, anne => 2, bert => 2, iowq => 1, john => 1, step => 1, stepy => 2 }

    edit: and here the code inside the regex:

    $ perl -MDP -wE'my%x;()="iowq john stepy andy anne alic bert stepy ann +e bert andy step alic andy" =~ m/(\w+)(?{$x{$^N}++})/g;DDumper\%x'

    Enjoy, Have FUN! H.Merijn
Re: counting words in string
by AnomalousMonk (Chancellor) on Aug 07, 2018 at 13:55 UTC

    What's in a name? As soon as you've figured this out (and it may not be easy), the rest of the problem is fairly simple. For clarity, the example below takes a few steps to do what could be done in a single statement.

    c:\@Work\Perl\monks>perl -wMstrict -le "use Data::Dumper; ;; my $rx_name = qr{ \b [[:alpha:]]+ \b }xms; ;; my $str = 'iowq john stepy andy anne alic bert stepy anne bert andy step alic + andy'; ;; my @names = $str =~ m{ $rx_name }xmsg; print Dumper \@names; ;; my %hash; ++$hash{$_} for @names; print Dumper \%hash; " $VAR1 = [ 'iowq', 'john', 'stepy', 'andy', 'anne', 'alic', 'bert', 'stepy', 'anne', 'bert', 'andy', 'step', 'alic', 'andy' ]; $VAR1 = { 'john' => 1, 'stepy' => 2, 'alic' => 2, 'bert' => 2, 'andy' => 3, 'step' => 1, 'anne' => 2, 'iowq' => 1 };


    Give a man a fish:  <%-{-{-{-<

Re: counting words in string
by tybalt89 (Vicar) on Aug 07, 2018 at 20:03 UTC

    Hash free ! :)

    #!/usr/bin/perl -l use strict; use warnings; $_ = "iowq john stepy andy anne alic bert stepy anne bert andy step al +ic andy"; print "$& => ", s/\b$&\b//g while /\w+/;

    Outputs:

    iowq => 1 john => 1 stepy => 2 andy => 3 anne => 2 alic => 2 bert => 2 step => 1
Re: counting words in string
by thanos1983 (Vicar) on Aug 07, 2018 at 14:33 UTC

    Hello Anonymous Monk,

    Sorry I missed understood the question I will update it.

    Another possible way (maybe not so efficient) is to use map and List::MoreUtils::uniq.

    #!/usr/bin/perl use strict; use warnings; use Data::Dumper; use List::MoreUtils qw(uniq); my $re = "iowq john stepy andy anne alic bert stepy anne bert andy ste +p alic andy"; my $i = 0; my %hash = map { ++$i => $_ } uniq split(/\s+/, $re); print Dumper \%hash; __END__ perl test.pl $VAR1 = { '6' => 'alic', '8' => 'step', '1' => 'iowq', '4' => 'andy', '2' => 'john', '7' => 'bert', '5' => 'anne', '3' => 'stepy' };

    There is a similar question asked in the forum, see here Count duplicates in array..

    #!/usr/bin/perl use strict; use warnings; use Data::Dumper; my %hash; my $re = "iowq john stepy andy anne alic bert stepy anne bert andy ste +p alic andy"; my @names = split(/\s+/, $re); map($hash{$_}++, @names); # $hash{$_}++ foreach @names; # alternative instead of map print Dumper \%hash; __END__ $ perl test.pl $VAR1 = { 'bert' => 2, 'anne' => 2, 'alic' => 2, 'john' => 1, 'andy' => 3, 'step' => 1, 'iowq' => 1, 'stepy' => 2 };

    Hope this helps, BR.

    Seeking for Perl wisdom...on the process of learning...not there...yet!
      map($hash{$_}++, @names);

      You can also use  grep($hash{$_}++, @names); as an alternative to map. (But why would you? :)


      Give a man a fish:  <%-{-{-{-<

Re: counting words in string
by Anonymous Monk on Aug 08, 2018 at 08:53 UTC

    Thank you all for responding, you've been mot helpful.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1219990]
Approved by marto
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (7)
As of 2018-10-22 08:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    When I need money for a bigger acquisition, I usually ...














    Results (119 votes). Check out past polls.

    Notices?