According to http://search.cpan.org/~rkitover/Net-SSH2-0.44/lib/Net/SSH2.pm#auth_publickey_%28_username,_public_key,_private_key_,_password__%29
It appears that passwords are supported for public keys.. "auth_publickey ( username, public key, private key , password )
Note that public key and private key are names of files containing the keys!
Authenticate using keys and an optional password."
Also, i'm not sure if this is a typo but your "passphrase => $passphrase," should be "passphrase => $passPhrase,"
If I am reading the documentation correctly, "passphrase =>" should be "password =>"
I hope this helps