They are not the same. Sure, they both "point" to something, but that's about it. In C, a pointer is just an integer - you can add to it: you can say, don't give me the thing it's pointing at, give me whatever is 5 places further. Regardless whether 5 places further actually has stored something that's meaningful.
You can't do that in Perl. You cannot stuff an integer in a reference and have Perl retrieve what's in that memory address. And while you can add something to a reference, the result is a number - not a reference. You cannot dereference the result of the addition.
IMO, saying that "Perl references" and "C pointer" are kind of the same is only useful if you know enough Perl and C to know the differences.