I can't look at the reputations without voting, and I can't knowledgably vote.
You can knowledgably vote by reading the posts and deciding on their own merit which you choose to ++ or --.
However, I do think you raised a good point. How do I know what the most 'respected' answers to a question are? The trend is that good answers have higher reps than the noise, and bad answers.
Maybe an option to sort answers based on rep (in User Settings (which it is in now), and a button on root nodes) would be good. We wouldn't see the reps (unless you voted on it) but you would know what others thought were the best.