Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: Help with database query

by sundialsvc4 (Abbot)
on Mar 15, 2013 at 09:08 UTC ( #1023645=note: print w/replies, xml ) Need Help??

in reply to Help with database query

Here’s an even more clever solution, I think:

Now, just read the rows.   If the first ID that comes back is, say, 50, then you know that IDs 1-49 are not in the table.   Read some more, and say you get 51, 52, 53, 60.   Okay, the next range of missing IDs is therefore 54-69.   Any time there is a gap in the returned sequence (when using ORDER BY), the keys in that space do not exist.

Furthermore, if the rows are indexed by this field, using the customary B-tree style index, then SQL does not have to physically sort the rows; it simply reads them in index order.   (Use the EXPLAIN verb, separately (not in your program), to confirm that this is what it will do.)

It’s an old (punched...) card trick.

If you find that the table is not so conveniently indexed and you are willing to scan it multiple times, you could read ranges of IDs, note them in a hash, and then loop through that range, one value at a time, finding the keys that are not in your hash.   Then empty the hash and repeat.   Grabbing 10,000 records or more in each chunk should not be a concern.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1023645]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (9)
As of 2016-10-26 10:37 GMT
Find Nodes?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?

    Results (340 votes). Check out past polls.