You can't do that in Perl. You cannot stuff an integer in a reference and have Perl retrieve what's in that memory address. And while you can add something to a reference, the result is a number - not a reference. You cannot dereference the result of the addition.
IMO, saying that "Perl references" and "C pointer" are kind of the same is only useful if you know enough Perl and C to know the differences.