Bad idea. Nodes are sometimes downvoted for good cause, and other times for bad cause, e.g. antipathy towards the poster.
Leave the system as is. If anyone believes the node should not be displayed they can can consider it. Then the community will decide.
However, I would support showing the vote count for especially high- or low-ranked posts to all readers. Maybe a formula based on overall rep, up- vs. down-votes percentage, and number of up- or down-votes compared to all votes cast for posts authored on that day ? Not easy to calculate, I guess, but it would be an improvement to reveal that a node is viewed very negatively. Just no automated reaping or hiding -- make someone take the action of considering.
Also, under NO circumstances should the font size be reduced. That just punishes readers whose eyesight is dimming.
The way forward always starts with a minimal test.