You actually have two different variables, both named "tests". One is the hash, %tests, and the other is a scalar, $tests, which holds a reference to a hash. If you're not familiar with references, swing on over to perlreftut and perlref. After that, References quick reference is correctly named.
The first line ($tests->{$stype} ||= {};) checks to see if $tests->{$stype} is a true value. If not, it sets it to "{}", which is a new, empty, anonymous hash reference. The intent of this is (probably) to ensure that $tests->{$stype} is a hash ref (in case it's empty), but it will actually do nothing if there's some other (true) value there already. It's equivalent to:
if ( ! $tests->{$stype} ) {
$tests->{$stype} = {};
}
If you're interested in the "||=" operator, have a look at perlop.
The second line (my %tests = %{$tests->{$stype}};) dereferences the hash reference in $tests->{$stype} and copies it to %tests. This is probably the reason for the first line. If there's no hash reference in there, the second line will die.