You would probably be better off using something like Mail::Box because Net::SMTP is very low-level.
But in answer to your question, mails sent by SMTP have two places where the recipient is specified: the so called "envelope header" which is part of the SMTP protocol itself (yes, I'm aware what the P stands for), and the "message header" which is part of the RFC 822 format (and successors).
It is the message header which email clients display. They don't usually have access to the envelope header at all, unless the mail server makes part of it available to them (which is quite common; usually by reformatting parts of it and adding them to the message header). Net::SMTP is an SMTP implementation which doesn't concern itself with the formatting of the message itself at all.
Anyway, to cut a long story short...
$smtp->datasend("To: $to\nSubject: TEST MAIL\n\n@csv");