Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

The first thing you need to do is define how you, as a human being, would judge the similarity of the sets.

For example, you start with a set (A), and you make an exact copy (B). You will (presumably) judge these as very similar.

  • What if You remove 1 of the phrases. Are they still similar?

    If the original set contains 100 phrases, and you remove phrases 1 at a time from the duplicate, does the similarity drop linearly?

  • What if you reversed the words in all of the phrases in the second set. Is it still very similar or completely dissimilar?

    Is ordering of the phrase words important.

  • How about if you removed one word from each phrase in the second set?

    Do the phrases need to be exactly the same, to be counted similar.

  • How about if you looked up each word in a thesaurus and substituted the nearest alternative word. Similar? Dissimilar?

    Are looking for semantic similarity.

  • How about if you misspelled every word by one character -- an ommision, and insertion, or transposition. Similar? Dissimilar?

    Can typos occur? Is it possible for you to correct them?

  • How about if you reverse the ordering of the phrases in the second set. Similar? Dissimilar?

    Are the sets ordered or unordered.

  • If one set consists entirely of "large blue woolen jumper" and the other "Angora sweater, navy, XL". Similar? Dissimilar?

    Semantics again.

Once you've decided how you would make the judgement, then you stand some chance of being able to lay out a set of rules. And once you have that, you can start to look for a good way to implement them.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong

In reply to Re: Comparing sets of phrases stored in a database? by BrowserUk
in thread Comparing sets of phrases stored in a database? by BUU

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others scrutinizing the Monastery: (7)
    As of 2018-06-25 09:09 GMT
    Find Nodes?
      Voting Booth?
      Should cpanminus be part of the standard Perl release?

      Results (126 votes). Check out past polls.