in reply to Auto-reaping of duplicates
I'm sure this must have been suggested before, but I wasn't watching then...
When a comment form is generated, included a "magic number" as a hidden field. The only condition being that the number is unique. Then track those numbers which are submitted.
Depending on performance implications, either note the generation of the number, and strike it when the comment is submitted (and generate an error if the number is not found due to previously being struck), or simply record all those numbers submitted (and generate an error if the number has been previously seen). In either mechanism, flush entries from the cache after a certain time to keep it down to size.