Re: how to open a data file via http

[hangon]

The http protocol does not directly support what you're trying to do. It's designed to return an entire file specified in an http request, not parts of a file. The easiest solution would be to slurp a copy of the file into a variable. Here are a couple of ways using LWP::Simple.

use LWP::Simple;

# get file into variable
my $data = get('http://www.jaywil.com/libsearch/SAMPLE.TXT');

# print all at once
print $data;

# or break up and print line by line
 # update: to keep blank lines at eof see comment by ikegami below 
@lines = split(/\n/, $data);
for (@lines){
    print "$_\n";
}

If for some reason you actually need to get the file one line at a time over http, you would need some help on the server side. If someone hasn't already written a module for this, you could write a cgi script that keeps state information and handles a few operations such as *open*, *read_line* and *close*. Basically you would be designing your own mini protocol to run over http.

[ikegami]

You incorrectly defined a file as "a series of lines seperated by newlines", whereas it's really "a series of lines ending in newlines (except possibly the last one)".

split(/\n/, $data) doesn't work. It removes blank trailing lines.
split(/\n/, $data, -1) doesn't work either. It adds a blank line.
split(/^/m, $data, -1) works. Bonus: It doesn't remove the newline!

The last one can also be written as split(/^/m, $data) and special handling allows split(/^/, $data) to work too.

[pKai]

split(/\n/, $data) doesn't work. It removes blank trailing lines.

What is your definition of "doesn't work" here? The dropping of empty trailing fields in the case of the LIMIT parameter being 0 or not specified works as advertised in the split docs.

split(/\n/, $data, -1) doesn't work either. It adds a blank line.

Perl doesn't do this on my system:

>perl -MData::Dumper -e "$_=qq(1\n2\n3); print Dumper split /\n/,$_,-1
+"
$VAR1 = '1';
$VAR2 = '2';
$VAR3 = '3';

>perl -MData::Dumper -e "$_=qq(1\n2\n3\n); print Dumper split /\n/,$_,
+-1"
$VAR1 = '1';
$VAR2 = '2';
$VAR3 = '3';
$VAR4 = '';

When you read your data from file on the other hand, you have to be careful that your editor does not hide a trailing newline on the display. (E.g. Vim, where you can't tell the difference, other than by an initial [noeol] in the status line.)

[ikegami]

What is your definition of "doesn't work" here?

The goal was a replacement for <FILE>, so anything that doesn't return what <FILE> is buggy. In hangon's favour, I didn't consider autochomping a bug.

my $data = "\n"        # 1
         . "abc\n"     # 2
         . "def\n"     # 3
         . "\n"        # 4
         . "\n"        # 5
         . "ghi\n"     # 6
         . "\n"        # 7
         . "\n";       # 8

{
   print("<FILE>\n");
   my $lines = 0;
   open(my $fh, '<', \$data);
   while (<$fh>) {
      ++$lines;
      chomp;
      print("[$_]\n");
   }
   print($lines, "\n");
}

print("\n");

{
   print("split(/\n/, \$data)\n");
   my $lines = 0;
   foreach (split(/\n/, $data)) {
      ++$lines;
      print("[$_]\n");
   }
   print($lines, "\n");
}

print("\n");

{
   print("split(/\n/, \$data, -1)\n");
   my $lines = 0;
   foreach (split(/\n/, $data, -1)) {
      ++$lines;
      print("[$_]\n");
   }
   print($lines, "\n");
}

print("\n");

{
   print("split(/^/m, \$data)\n");
   my $lines = 0;
   foreach (split(/^/m, $data, -1)) {
      ++$lines;
      chomp;
      print("[$_]\n");
   }
   print($lines, "\n");
}

outputs

<FILE>
[]
[abc]
[def]
[]
[]
[ghi]
[]
[]
8       <-- This is what <FILE> gives.

split(/\n/, $data)
[]
[abc]
[def]
[]
[]
[ghi]
6       <-- Not 8. Failed to emulate <FILE>.

split(/\n/, $data, -1)
[]
[abc]
[def]
[]
[]
[ghi]
[]
[]
[]      <-- Extra line
9       <-- Not 8. Failed to emulate <FILE>.

split(/^/m, $data)
[]
[abc]
[def]
[]
[]
[ghi]
[]
[]
8       <-- Bingo!

wc -l says 8 as well, by the way.

Perl doesn't do this on my system:

uh, yes it does... $VAR4 = '';. Lines aren't *seperated* by newlines, they are *terminated* by them.