Perhaps you could show us a few lines from the file to give us a sample? So what you are saying is that the number you are looking for is more like an id number or phone number than a line number? So that when you say "the line number will be used to pull the line" you perhaps mean something more like the "phone number", to use your example?
If there are no repeats of the line number you want from anywhere else in the file, then perhaps the following:
my $number = 42;
#or $number = $ARGV or whatever line number you were looking for
my $file = <FILE>;
my ($line, $name);
# ($line) = $file =~ /$($number\s+.*)/m;
# would store the whole line
($number, $name) = $file =~ /$($number)\s+(.*)/m;
# use the second form since you just want number and name
# and yes, you are overwriting the value of $number
# with itself all over again. Use (undef, $name) = $file...
# if that makes you feel more comfortable.