On one machine, no, it isn't hard to change it. Making the same change on 15 or 20 hosts starts to get irritating. Not difficult, just irritating. Which mirror am I using now again? Oh, right... Then 'o conf urllist blah blah..." I tend to only change when my selected mirror gets slow, stale, or disappears.
It's while I was at it that I wanted to start mirroring CPAN (for speed as much as keeping the load off the mirror) to avoid downloading all the same modules over and over, but I didn't want to mirror the whole thing or even the mini-list. It takes up too much space it's it's even more load for the mirror, rather than less. What I really wanted was a cache.
The reason I only pick one mirror at a time is that they don't always seem to be synced up with the same modules. I originally set it up to be able to pick randomly from a list of favorites, but it causes unexpected problems -- so I started just picking one.