I've made what one might call an "enemy" on here.

That appears to be a fairly common belief. I see it claimed rather regularly, and for nearly exactly the reasons that you have presented. You may well have made one or more "enemies". However, as usual, the evidence that you based this conclusion on doesn't actually support your conclusion. If you have enemies here, they aren't systematically downvoting your nodes right after you post them. (When I see complaints of potential abuse, I often investigate in part to make sure that I understand what problems there might be and if there are ways to improve things.)

Human brains are great at seeing patterns. So good that they can often see patterns where there are none, which leads many humans to see conspiracies where there are none. It is also human nature to blame others before blaming oneself. So it is pretty natural to see yet another downvote and eventually come to the conclusion that some other single person is responsible and thus that they have some personal grudge against you.

When (rarely but still too frequently) someone forms a grudge and starts systematically downvoting another monk, claims start approaching a real conspiracy, inventing an army of downvoting robots. (:

The downvotes against you have been cast by a wide variety of monks. I did see that the frequency of downvotes against you was higher a while back, but they were being cast by a wide variety monks both then and now. So it seems more likely that your nodes have changed over that time (or perhaps your extra-node presence has changed, as you suspect).

It is still true that (thankfully) most monks rarely downvote. But when someone downvotes a node, it is rare to contemplate the node and then come back and cast the downvote later. A downvote is more likely to be a snap decision or at least a fast reaction while upvotes often take longer. So nodes getting an initial downvote or few and then moving to a positive rep appears to rarely have much to do with who wrote the node.