It isn't the start-up time. It is probably an effect of the working set "settling in" or any number of other things that can affect benchmark numbers. In general, a 5% or less difference isn't something I would consider "real" as running it an hour later could certainly swing the answer that much. Here is the code I used:

use Benchmark qw(cmpthese);

my $str= "This is a test, " x 200;
my $single= "'".$str."'";
my $double= '"'.$str.'"';
cmpthese( -3, {
    a_double => sub { eval $double },
    b_single => sub { eval $single },
    c_double => sub { eval $double },
    d_single => sub { eval $single },
} );
[download]

           Rate a_double c_double b_single d_single
a_double 4773/s       --      -1%     -23%     -23%
c_double 4830/s       1%       --     -22%     -22%
b_single 6170/s      29%      28%       --      -0%
d_single 6172/s      29%      28%       0%       --
[download]