And, bingo!

#! /usr/bin/perl

use feature qw (switch);
use Try::Tiny;

my $uv;
given (17) {
    when (3) { print "three\n"; };
    when (17) {
    try {
        print "In try block\n";
        die( "Substitute undefined variable $uv\n");
        print "Still in try block\n";
    } catch {
        print "We caught $_\n";
    };
    }
}
[download]

Output is:

$ ./t4.pl 
In try block
We caught 17

It was indeed an interaction between Try::Tiny and the switch feature. Might not happen in later versions of Perl, according to what I've read, but we're running 5.10, where it does.