Since you are dealing with raw packets, you have to listen to everything and filter yourself for the packets you are interested in. You will probably want to consider the packets that are received on the port you used to send packets from, and discard all others.
This code will listen to every TCP packet and show you the IP addresses and port numbers involved.
#!/usr/bin/perl
use strict;
use Socket;
socket S, PF_INET, SOCK_RAW, 6;
my $result;
while (my $sender = recv(S, $result, 2000, 0)) {
print "got a ", length($result), " byte TCP packet...\n";
if (length($result) < 20) {
print " ...which is too small to be an IP packet\n";
} else {
printf " from %d.%d.%d.%d to %d.%d.%d.%d\n",
map { unpack("C", substr($result, $_, 1)) } (12..19);
if (length($result) < 40) {
print " ...and is too small to be a TCP packet\n";
} else {
printf " from port %d to port %d\n",
map { unpack("n", substr($result, $_, 2)) } (20, 22);
}
}
}