A lot of people for some reason, do find XP to be important. This would prevent them from downvoting. I think that downvoting is a Good Thing. It helps weed out bad posts/flames/etc. On the other hand, encouraging upvoting simply inflates all the posts, which makes finding the best nodes more difficult.
If you really wanted XP to mean something, then you should ONLY get XP if other people vote on your nodes. However, this might discourage voting (thanks to Laziness).