One suggestion. It appears as though you may be rotating your points multiple times; and doing so by modifying the original coordinates by each rotation. The problem with this is that as the rotations accumulate, the rounding errors inevitably introduced by each successive rotation also accumulate, but those rounding errors will be different and accumulate differently for each point on the structure being rotated. The net affect is that after many small rotations, the registration between the points making up the structure will wander.
A concrete example might clarify here. Let's say you start with a nice unit vector cube distributed equidistant around the origin:
After rotating 360° around each axis in 1° steps -- 1080 floating point calculations, you will likely end up with values that look more like:
As you can see, the 'cube' has become irrevocably distorted. And that was starting with relatively large (compared to your atomic scale measurements) and floating point friendly (0.5 stored exactly as a FP number) values. With smaller values and values that cannot be exactly represented, the distortions are going to be even more noticeable.
An oft-used alternative is to retain the original coordinates and accumulate the rotations -- usually in the form of a transform matrix -- and only apply the rotations when fixing a position for output (eg. when displaying).
In this way, the you only have the distortions from a single calculations rounding errors at any given position, rather than the accumulated distortions of many calculations.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
In reply to Re: Rotation around an axis clockwise and anti-clockwise algorithm question?