Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: arabic alphabet ... how to deal with?

by ForgotPasswordAgain (Deacon)
on Feb 12, 2009 at 15:49 UTC ( #743367=note: print w/ replies, xml ) Need Help??


in reply to arabic alphabet ... how to deal with?

Did you try printing out the stop words? You probably have to read it in utf8 mode:

open (STOPWORDS, '<:encoding(UTF-8)', $ARGV[1])


Comment on Re: arabic alphabet ... how to deal with?
Download Code
Re^2: arabic alphabet ... how to deal with?
by Anonymous Monk on Feb 12, 2009 at 15:54 UTC
    I did but still does not work ...
      What about the infile? That needs to be opened UTF-8 as well.
      -derby
        It is uft8 as well ... It does not change the results but also it gives some errors as well regarding the wide character ... even i change my code to this:
        #!/usr/bin/perl use Lingua::AR::Word::Encode; use Encode::Arabic; open (STOPWORDS, $ARGV[1]) || die "Error opening the stopwords file\n" +; $count = 0; while ($word = <STOPWORDS>) { $word=Lingua::AR::Word::encode($word); chop($word); $stopword[$count] = lc($word); $count++; } close(STOPWORDS); open (INFILE , $ARGV[0]) || die "Error opening the input file\n"; while ($line = <INFILE>) { $line=Lingua::AR::Word::encode($line); chop($line); @entry = split(/ /, $line); $i = 0; while ($entry[$i]) { $found = 0; $j = 0; while (($j<=$count) && ($found==0)) { if (lc($entry[$i]) eq $stopword[$j]) { $found = 1; } $j++; } if ($found == 0) { print "$entry[$i]\n "; } $i++; } } close(INFILE);
        still does not work , it does not remove my stop words :(

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://743367]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2014-09-19 23:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (151 votes), past polls