A couple of weeks back I talked to a few people about the idea of showing the total number of votes cast if the number of downvotes was, say, 5 or more. This way we don't get a huge increase in people complaining about single downvotes when their node reputation is positive (where they likely wouldn't notice the single downvote if the total number of votes wasn't displayed). But if your node is getting conflicting votes, then you can watch the tug of war. Also, if your node has a negative reputation, you will quickly be able to see that a few upvotes are coming in as well.
To deal with "vote inflation", the value of 5 would actually be computed as 1+int($NORM/2) where $NORM is taken from Voting/Experience System. Note that $NORM is recomputed each day (and tends to slowly creep up) so, after a while, you might no longer be able to see the total votes cast on some "old" node that you used to be able to see the total for.
(but my friends call me "Tye")