Each .t does specific setup and teardown, and many use END {} blocks to do useful things like destroy temporary database objects that were inserted for purposes of testing, and which shouldn't be there by the time the next bunch of tests runs.
Doing this sort of thing for large test suites ended up with me writing Test::Class. It may or may not prove useful to you :-)
Combining these tests would be possible, but only at the expense of a lot of work. We're getting more mileage by attacking bloat first.
Whatever works. Refactoring a large test suite can certainly be a huge job (having had to do it a few times in the past).