Problem uploading file data using CGI

[dougconran]

I have an Android app that sends multipart data to a web server. The data consists of compressed binary data (which is actually a file but sent as data) and 2 text fields. Using CGI

$data = $q->param('param');

I can pick up the text data and filename easily enough but, whatever I do, I cannot get the binary data. The raw data coming into the server is

--mkGAOmWzlLaPNjkDQ57CNA9mUc6uoe1JMSRI
Content-Disposition: form-data; name="mtdata"; filename="filename"
Content-Type: application/octet-stream
A whole lot of binary data

--mkGAOmWzlLaPNjkDQ57CNA9mUc6uoe1JMSRI
Content-Disposition: form-data; name="email"

email address
--mkGAOmWzlLaPNjkDQ57CNA9mUc6uoe1JMSRI
Content-Disposition: form-data; name="mtfilename"

file name
--mkGAOmWzlLaPNjkDQ57CNA9mUc6uoe1JMSRI--

The CGI man page suggests

 my $data = $query->param('POSTDATA');

If POSTed data is not of type application/x-www-form-urlencoded or multipart/form-data but that doesn't work. I've also tried:-

         $lightweight_fh  = $q->upload('field_name');

         # undef may be returned if it's not a valid file handle
         if (defined $lightweight_fh) {
           # Upgrade the handle to one compatible with IO::Handle:
           my $io_handle = $lightweight_fh->handle;

           open (OUTFILE,'>>','/usr/local/web/users/feedback');
           while ($bytesread = $io_handle->read($buffer,1024)) {
             print OUTFILE $buffer;
           }
         }

but that's not working either. I cannot (at the moment) change the content type of the uploaded data - can anyone see what I'm doing wrong?

[tobyink]

"The CGI man page suggests my $data = $query->param('POSTDATA'); If POSTed data is not of type application/x-www-form-urlencoded or multipart/form-data"

But the data you showed is of type multipart/form-data. If you're submitting a form with a file upload field, it must always be multipart/form-data.

In your last example, you are trying to open a file called /usr/local/web/users/feedback in append mode and write the data to that file. Do you have permission to modify this file? You should check the return value from open() to make sure you successfully opened the file. This is what the documentation for open says:

"When opening a file, it's usually a bad idea to continue normal execution if the request failed, so open() is frequently used in connection with die(). ... you should always check the return value from opening a file."

Here's an example of using open() with die():

open (OUTFILE,'>>','/usr/local/web/users/feedback')
  or die("Cannot open 'feedback' file: $!");

[dougconran]

Thanks for the quick reply.

1. The data as it comes into the web server is

Content-Type: application/octet-stream

and that is what is causing the problem because it really needs to be multipart/form-data. CGI.pm is supposed to be able to read application/octet-stream (and, I'm sure, does) but I'm obviously doing something wrong.

2. I think that I've mislead you with regard to the code I'm using to try and extract the data. In both cases it was cut and pasted from the man pages because I'd changed the code in the program to many different things. Here is what I've actually used:-

############  First Attempt

$data = $Data->param('POSTDATA');

open(FILE,">/tmp/$Filename");
print FILE $data;
close(FILE);

##########  Alternative attempt

$handle = $Data->upload($Mtfilename);
if (defined $handle) {
        $io_handle = $handle->handle;
        open(FILE,">/tmp/$Filename");
        while ($bytesread = $io_handle->read($buffer,1024)) {
                print FILE $buffer;
        }
        close(FILE);
}

[tobyink]

"1. The data as it comes into the web server is Content-Type: application/octet-stream"

No, it doesn't. You're not looking at the real HTTP headers. A multipart/form-data HTTP message has an uber-header at the top, and then each form field has its own set of mini-headers.

The Content-Type: application/octet-stream header you posted is from the file field's mini-headers. The uber-headers will say Content-Type: multipart/form-data.

A full multipart/form-data HTTP request might look something like this:

POST /handler.cgi HTTP/1.1
Host: http://www.example.com/
Content-Type: multipart/form-data; boundary=XYZ

--XYZ
Content-Disposition: form-data; name="title"

Picture of Camel
--XYZ
Content-Disposition: form-data; name="upload"
Content-Type: image/jpeg

... some binary data ...
--XYZ--

The kind of form that might result in that request is:

<form action="http://www.example.com/handler.cgi"
      method="post"
      enctype="multipart/form-data">
  <fieldset>
    <legend>Image upload</legend>
    <label>Image <input name="title"></label><br>
    <label>File <input name="upload" type="file"></label>
    <input type="submit">
  </fieldset>
</form>

And my advice to check the return value of open() still stands.

[poj]

What is the name of the file input tag, is it 'field_name' or perhaps 'mtdata' ?

<input type="file" name="????" >

poj

[dougconran]

The filename is actually coming in in 2 places. The first is as part of the binary data

Content-Disposition: form-data; name="mtdata"; filename="filename"
Content-Type: application/octet-stream
... binary data ...

The second is as a text field

Content-Disposition: form-data; name="mtfilename"
filename

[Anonymous Monk]

The CGI man page suggests

The manpage of the CGI module version you have installed?

Upgrade, also try Bug #79102 for CGI.pm: PUTDATA/POSTDATA, uncorrupted by -utf8, and as tempfiles