Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^2: locales, encodings, collations, charsets... how can I match a given MySQL collation?

by xaprb (Scribe)
on Apr 02, 2007 at 03:08 UTC ( #607747=note: print w/ replies, xml ) Need Help??


in reply to Re: locales, encodings, collations, charsets... how can I match a given MySQL collation?
in thread locales, encodings, collations, charsets... how can I match a given MySQL collation?

Thanks for the link. I am sure I can use that module in a script I wrote at work, but not here. Here, I may be comparing two large tables and finding one or two rows that differ. I can't put it all in memory, especially not in seen-hashes. I need to be able to suck rows one at a time from MySQL, in sorted order, and have Perl know when a row is missing from one table because one row is "greater than" or "less than" the other.

I've seen the term XY but not seen a definition. What does it mean?


Comment on Re^2: locales, encodings, collations, charsets... how can I match a given MySQL collation?
Re^3: locales, encodings, collations, charsets... how can I match a given MySQL collation?
by GrandFather (Cardinal) on Apr 02, 2007 at 03:17 UTC

    Generally an XY question is one that pertains to solving a detail problem without providing the larger context. Very often in such a case the larger context suggests a better solution than solving the detail problem.

    I don't see a better solution straight off for your problem here however, although the larger context that you've provided may help those more database savvy than I to find a good solution for you.


    DWIM is Perl's answer to Gödel
Re^3: locales, encodings, collations, charsets... how can I match a given MySQL collation?
by roboticus (Canon) on Apr 02, 2007 at 13:43 UTC
    xaprb:

    Perhaps the right approach would be to use a mergesort-like method. Basically, execute a select on each table, specifying the order so MySQL will create the proper order for you. Then you can process the tables in parallel. To see an example, I previously described a similar thing in Re: How to deal with Huge data.

    ...roboticus

      Thanks! This is *exactly* what I'm doing currently. The only trouble is to get the le, ge, cmp etc operators to agree with MySQL's idea of whether a row is lt/gt/eq the other row.

      Imagine the left-hand table has "éclair" and "ecstatic", and the right-hand table has only "ecstatic". MySQL sorts é before other 'e' values, so the merge algorithm sees "éclair" on the left and "ecstatic" on the right after fetching the first row from each table. Perl thinks "ecstatic" should come first though -- at least that's how it works with just the default collations. So the merge algorithm concludes "ecstatic" doesn't exist in the left-hand table. After this, it runs out of rows in the right-hand table and decides "éclair" doesn't exist there, then fetches the next row from the left-hand table -- and decides "ecstatic" doesn't exist in the right-hand table.

      It's quite a train wreck unless I get Perl's sorting to exactly match MySQL's!

        xaprb:

        Okay ... then perhaps MySQL can help you in a different way. I don't know if this is suitable or not, but you might create a temporary table of the columns you're comparing, and put an index on it, and insert the key fields of both tables. Then you can select from all three columns using eq (which I (perhaps falsely) presume should work well in Perl).

        So the temp table would hold the ordering, and you simply select which of the other two tables holds the match for the record...the first, the second, or both.

        It may not be a good solution for you, if speed and/or database space is limited, but it's the only thing I can think of. (I know diddly-squat about fancy collation sequences....)

        ...roboticus

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://607747]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (11)
As of 2014-12-18 10:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (50 votes), past polls