Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
I couldn't find the stuff I had done before, probably was pretty sad Perl since I wrote it back '96. Here, however is some stuff to start with that I hacked just now. (I'll continue with this, but probably in my own direction.) The thesaurus file is merely a list of keywords followed by synonyms. The newWords should probably be appended to the thesaurus file with an appropriate # comment for each word.

A thought, it might be possible to get some first take synonym solutions by parsing the returned text from a dictionary site.

#!/usr/bin/perl -w use strict; use warnings; my $thesaurusPath = ""; my $thesaurusFile = "thesaurus.nav"; my $ignore = '//'; # key to ignore in the thesaurus file my $thesaurus = {}; my $newWords = {}; # read in the thesaurus word list open CMDS, "<${thesaurusPath}${thesaurusFile}" or die "cannot open \"$thesaurusFile\" for reading"; my $line; while($line = <CMDS>){ chomp($line); # remove comments and leading and trailing space $line =~ s/\s*\#.*$//; $line =~ s/^\s+//; $line =~ s/\s+$//; next if not length($line); # uppercase only $line =~ tr/a-z/A-Z/; # break the list apart and stash it my @words = split(/\s/,$line); # key word to which the others resolve is $words[0], the first one for(@words) { $thesaurus->{$_} = $words[0]; } } close CMDS; # now lets see what results with rewriting print "> "; while(<>) { my @result = (); my @words = (); chomp; tr/a-z/A-Z/; # uppercase only last if /^\s*$/; # end if no input @words = split; for(@words){ if(not defined $thesaurus->{$_}){ # increment or create entry for new word if(defined $newWords->{$_}) { ++ $newWords->{$_}; } else { $newWords->{$_} = 1; } push(@result,"?$_?"); # flag it in output }else{ next if $thesaurus->{$_} eq $ignore; push(@result,$thesaurus->{$_}); } } print join(" ",@result),"\n"; print "> "; } print "These words were not recognized:\n"; for (keys %$newWords) { print "$_\t\t$newWords->{$_}\n"; } exit;

In reply to Re: NLP - natural language regex-collections? by perlcapt
in thread NLP - natural language regex-collections? by erix

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?

What's my password?
Create A New User
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (5)
As of 2021-05-13 15:33 GMT
Find Nodes?
    Voting Booth?
    Perl 7 will be out ...

    Results (136 votes). Check out past polls.