I've sometimes seen it happen that I've gotten 1 vote but gained 2 XP. I've noticed this at times that I've kept a close eye on noderep xml ticker. I used to think that what I was seeing was actually 3 votes—2 up and 1 down—which would show as delta 1 vote up but, with some help from rand, wound up being delta 2 XP.
More recently I've decided that the XP calculations are fuzzy. This is just another reason to ignore them. Would it be possible to have the XP Nodelet tell me how many votes I have but not my changes in XP? That would be nice.