Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: Similarity measurement

by Khen1950fx (Canon)
on Apr 18, 2015 at 22:33 UTC ( #1123910=note: print w/replies, xml ) Need Help??


in reply to Similarity measurement

Give Text::Similarity::Overlaps a try. For example:
#!/usr/bin/perl -l use strict; no strict 'refs'; use warnings; use Text::Similarity::Overlaps; my( %opt ) = ( verbose => 1, Text::Similarity::NORMALIZE => 1, ); my $mod = Text::Similarity::Overlaps->new( \%opt ); die "$mod failed" unless defined $mod; my $file1 = "/usr/lib/perl5/5.10.0/i386-linux-thread-multi/Encode.pm"; my $file2 = "/usr/lib/perl5/5.10.0/i386-linux-thread-multi/Encode.pm"; open $file1, '<', \*STDOUT or die $!; binmode STDOUT, ":encoding(UTF-8)"; open $file2, '<', \*STDOUT or die $!; binmode STDOUT, ":encoding(UTF-8)"; my $score = $mod->getSimilarity( $file1, $file2 ); print "The similarity of $file1 and file2 is: $score"; close( $file1 ); close( $file2 );
It'll take a few minutes, but it comes back with a score. In this case, the result was: 0.999615754082613 for two files exactly the same.

For two completely different files:

#!/usr/bin/perl -l use strict; no strict 'refs'; no warnings::anywhere qw(uninitialized); use Text::Similarity::Overlaps; use warnings qw(uninitialized); my( %opt ) = ( verbose => 1, Text::Similarity::NORMALIZE => 1, ); my $mod = Text::Similarity::Overlaps->new( \%opt ); die "$mod failed" unless defined $mod; my $file1 = "/usr/lib/perl5/5.10.0/i386-linux-thread-multi/Encode.pm"; my $file2 = "/usr/local/lib/perl5/site_perl/5.10.0/POE.pm"; open $file1, '<', \*STDOUT or die $!; binmode STDOUT, ":encoding(UTF-8)"; open $file2, '<', \*STDOUT or die $!; binmode STDOUT, ":encoding(UTF-8)"; my $score = $mod->getSimilarity( $file1, $file2 ); print "The similarity of the two files is: $score"; close( $file1 ); close( $file2 );
The smilarity score for two completely different files came back at: 0.345969033635878

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1123910]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (5)
As of 2019-05-27 07:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you enjoy 3D movies?



    Results (155 votes). Check out past polls.

    Notices?
    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!