That's why I had $avg - I don't really like taking the average distance to do this (I'd much prefer to have a score system where I get everything and then filter out after) but as I can't even figure this out, I figure this is a good starting point. I could work with it if I got this working at least.
What you're trying to do is cluster analysis - naturally grouping data together in clusters (for some value of "naturally").
Most approaches I'm aware of require you to know the number of clusters ahead of time (which sort of defeats the purpose).
However, if you can come up with some heuristic, such as "any element of a cluster must be within 10% of the center point of the cluster's range", you might be able to quickly compute the results, and live with them. (Of course there are pathological cases where adding a new element changes the center point, causing other elements to be cast out.)
Quantum Mechanics: The dreams stuff is made of