Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

File Sent via HTTP Content

by cmilfo (Hermit)
on May 03, 2002 at 20:37 UTC ( [id://163897]=perlquestion: print w/replies, xml ) Need Help??

cmilfo has asked for the wisdom of the Perl Monks concerning the following question:

Most esteemed Monks,

I've searched for a while on this one. I would like to retrieve data sent via the HTTP Content area. Normally, to upload a file to a CGI one would specify the form-data to contain a form variable of type file and then the file, LWP example of this:
my $ua = new LWP::UserAgent; my $res = $ua->request(POST 'http://myserver.com/Parse.pl', Content_Type => 'form-data', Content => [file => ['Test.ini']]);
CGI.pm example of reading the file:
my $query = CGI->new(); my $file = $query->param('file'); my $mimetype = $query->uploadInfo($file)->{'Content-Type'}; my $data; binmode $file if $mimetype !~ /text/; while (read ($file, $data, 1024)) { # do something with data }
CGI.pm can easily pull the file variable and file. The program uploading to my CGI is doing the equivalent of this (but in Java):
my $ua = new LWP::UserAgent; my $res = $ua->request(POST 'http://myserver.com/Parse.pl', Content => $file);
The file is stored after the Content-type: 'application/x-www-form-urlencoded' (in this case HTTP::Request is generating that for me, I could specify it). Anyway, I have not found any documentation explaining how to do this. I would like to use CGI.pm, but it seems like it parses the file (in Content-type) as if it were form-data. LWP doesn't seem to work here either. I assume this has been solved before, I just have not found the solution. Any help you can give will be greatly appreciated.

Thank you!

Casey

Replies are listed 'Best First'.
Re: File Sent via HTTP Content
by chromatic (Archbishop) on May 04, 2002 at 14:31 UTC
    File uploads must be sent in multipart/form-data encoding, per RFC 1867. See the 'POST' section in the documentation for HTTP::Request::Common for a nice way to upload a file. Once you've fixed the upload, CGI.pm will happily decode it.
      Thank you for the information. I read the RFC; for anyone interested (and not wanting to do a quick Google search), here is a link to RFC 1867. When I was asked if I could receive a POST within the content-type and non-form-data, I never even considered it shouldn't/couldn't be done that way. The Java library my coworker is using allows this to be done; he has scaffolding in place that returns random data until I finish the Parser. Well, I will just have to change the protocol we've agreed upon.

      Thank you!
      Casey
Re: File Sent via HTTP Content
by beebware (Pilgrim) on May 03, 2002 at 22:24 UTC
    I've had to do a very similar thing before, the browser plugin we had to use for a project didn't quite send the data in a proper GET/POST request, therefore CGI.pm couldn't handle the data. For "commercial" reasons, I've had to extract that specific part of the file:
    sub library_read_form { # Reads the incoming data from the web browser my ($buffer,$pair,$datasent,$name,$value,$mydata,$filename,$contentty +pe)=("","","","","","","",""); if (length($ENV{'CONTENT_LENGTH'})>0) { if ($ENV{'CONTENT_TYPE'}=~/^multipart\/form-data/i) { binmode(STDIN); # cope with Win platforms } read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'}); if (length($ENV{'QUERY_STRING'}>0)) { $buffer.="&".$ENV{'QUERY_STRIN +G'};} } else { $buffer=$ENV{'QUERY_STRING'}; } foreach $var (sort(keys(%ENV))) { $val = $ENV{$var};$val =~ s|\n|\\n|g;$val =~ s|"|\\"|g; } if (($buffer=~/^\<doc /) && ($buffer=~/\<\/doc\>\n?$/)) { ### section removed..... ### don't ask what this did or why, I'm not allowed to say :( } else { if (!($ENV{'CONTENT_TYPE'}=~/^multipart\/form-data/i)) { @pairs = split(/(&|;)/, $buffer); foreach $pair (@pairs) { ($name, $value) = split(/=/, $pair); $value =~ tr/+/ /; $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; $datasent.=";$name=$value"; if (length($form{$name})>0) { $value=$form{$name}."\0".$value; } $form{lc($name)} = $value; } } else { $ENV{'CONTENT_TYPE'}=~/boundary=([^"]+)/i; $boundary="--".$1;$section=0; @lines=split(/(\n|\r)/,$buffer); MULTIPART: foreach $line (@lines) { if ($section==3) { if ($line=~/content-type:\s?(.*)/i) { $contenttype=$1; } elsif (!($line=~/^(\n|\r|\n\r|\r\n)?$/)) { $section=4; } } elsif ($section==2) { if ($line=~/^content-type:\s?(.*)/i) { $contenttype=$1; } elsif ($line=~/^(\n|\r|\n\r|\r\n)?$/) { $section=3; # data possibly starts next line } } elsif ($section==1) { if ($line=~/^content-disposition:\s?form-data;\s?name="([^"]+)" +(.*)$/i) { $name=$1;if ($2=~/filename="([^"]+)"/) { $filename=$1; } $section=2; # prepare for data } } if ($line=~/^$boundary(--)?$/) { $section=1;# found boundary if ((length($name)>0) && (length($filename)<1)) { chomp($mydata);chop($mydata); # munch munch $datasent.=";$name=$mydata"; if (length($form{$name})>0) { $mydata=$form{$name}."\0".$mydata; + } $form{lc($name)}=$mydata; } elsif (length($name)>0) { $form_filenames{$name}=$filename;$form_contenttype{$name}=$conte +nttype; $tempfile=$tempdir.(time())."_".int(rand(300000)); $tempfile.=".tmp"; open (DATA,">".$tempfile) || die("Unable to open temp dir for st +orage\n"); binmode(DATA);print DATA $mydata;close DATA; $form_data{$name}=$tempfile; } $name="";$mydata="";$filename="";$contenttype=""; } if ($section==4) { $mydata.=$line; } } } } $datasent.=";"; }
    Data is normally stored in the $form hash (for example $form{'lastnode_id'}=160961), but if have an uploaded file (say from a <input type="file" name="tester"> tag, you'll have $form_data{'tester'} (which contains the name of the temporary file), $form_filenames{'tester'} (which contains the 'user provided' upload path), $form_contenttype{'tester'} (which contains the user provided MIME type).
    Where it says 'code removed' will probably be the bit you need to hack away at.I put in the appropriate regexp to 'identify' the plugin supplied file, but you'll need to tweak it to recognise your uploaded files and store it either in an appropriate file or variable yourself.
    Ok, ok, the code is neither neat, (probably not) secure or 'strict-safe' but it should give you an idea how to do what you are trying to achieve (and it's been happily running on a Windows2000 Server box with Apache for nearly 2 years now). Hope it helps!
      therefore CGI.pm couldn't handle the data
      That's like saying perl can't handle OO programming, yeah, perl(1|2|3|4) maybe.

      Which version of CGI have you had problems? (it's not really a problem w/cgi, it handles more non-rfc conforming crap than anything out there)

        Well, the entire system was written within a fortnight (database design,planning to the finish product) with me holed up in a B+B during the time, so it was a bit of a rush job. chromatic mentioned 'fixed the upload', well the plugin I had to work with cost the company I was working for $0.25million so getting it re-written in the time scale allowed wasn't a possibility.How would you use CGI.pm to handle the sort of crap I had to deal with (all the plugin sent to the server with the appropriate URL in the request, no other header data, and just the body just contained <doc>...<doc> - if CGI.pm could handle it, I'll re-write the system, but the upload was so 'broken' in regard to the RFC's I didn't see a way it could cope!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://163897]
Approved by Corion
Front-paged by sparkyichi
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (3)
As of 2025-03-20 02:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    When you first encountered Perl, which feature amazed you the most?










    Results (60 votes). Check out past polls.