|There's more than one way to do things|
I don't think this is a flaw in the current voting mechanism. Take a look at Best Nodes. Sure, some of the stuff on the daily list is overrated -- simple answers to simple questions sometimes get 20 points.
If you look at the 'All Time' list, you'll notice that stuff there is thoughtful discussion, careful experimentation, and really really interesting obfuscation.
Sorting by score (option available from user settings) lets you see, roughly, where posts fit in relation to each other. Sometimes I vote up the lower ones on a page because they have good information in them that shouldn't be overlooked.
I'm not in the practice of voting high-ranked posts down. I do check out Worst Nodes daily, and bump up stuff that just isn't that bad.
If you're concerned about inflation (which is, I suspect, the bigger issue), perhaps implementing some sort of dual Writeup/XP requirement to gain a level would be in order. You would be allowed to reach level 4 or 5 by XP alone, but to progress beyond that, you'd need a certain level of writeups in addition to XP points. I would support that (and voluntarily give up my level 10 status to go back down to level 7 or so.)