Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Help with database query

by sundialsvc4 (Monsignor)
on Mar 15, 2013 at 09:08 UTC ( #1023645=note: print w/ replies, xml ) Need Help??


in reply to Help with database query

Here’s an even more clever solution, I think:
SELECT DISTINCT ID FROM table ORDER BY ID;

Now, just read the rows.   If the first ID that comes back is, say, 50, then you know that IDs 1-49 are not in the table.   Read some more, and say you get 51, 52, 53, 60.   Okay, the next range of missing IDs is therefore 54-69.   Any time there is a gap in the returned sequence (when using ORDER BY), the keys in that space do not exist.

Furthermore, if the rows are indexed by this field, using the customary B-tree style index, then SQL does not have to physically sort the rows; it simply reads them in index order.   (Use the EXPLAIN verb, separately (not in your program), to confirm that this is what it will do.)

It’s an old (punched...) card trick.

If you find that the table is not so conveniently indexed and you are willing to scan it multiple times, you could read ranges of IDs, note them in a hash, and then loop through that range, one value at a time, finding the keys that are not in your hash.   Then empty the hash and repeat.   Grabbing 10,000 records or more in each chunk should not be a concern.


Comment on Re: Help with database query

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1023645]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (8)
As of 2014-07-24 23:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (167 votes), past polls