I think the best answer here is "design choice". Test::More is meant to be a core set of tests that handle the day-to-day basic Stuff You Gotta Do. So you can do the basics: this did what I expected, or something different; this is the same as that, as I expected, or it's different, as I expected. The "deeply" versions just amp up that basic test a little: like the thing I expected, or not.

Test::Exception, Test::LongString, Test::Tester, Test::WWW::Simple, and on and on, just add a little more specific testing power in specific areas.

Another possibility is to put Test::Exception in t/lib and distribute it with your code. That way you get the win of getting the test you want without requiring the user to install it.

Yet another is to check for whether Test::Exception is available on the machiine where the module's being installed, and skip the tests if it isn't. This is the way I generally prefer to handle such things; I use this for POD tests, which are useful to me, but maybe not so much to the person installing this module later.