I have been thinking about something similar, but perhaps not as drastic. It was inspired by a post somewhere here, but I don't remember where, so I can't give credit at the moment. Basically, for a certain "sufficiently low" reputation (or downvote-to-upvote ratio, with a minimum number of votes required), just reduce the font size*, perhaps in two steps. That would give readers a visual clue as to which nodes have a "sufficiently low" reputation without going so far as to hiding them or publicly displaying the vote count.
* Update: Through the various counterarguments in this thread I have been sufficiently convinced that reducing the font size is probably not the best way to go, so for now I'm going with my other idea of an indicator message at the top of the node.