http://www.perlmonks.org?node_id=32916

bhess has asked for the wisdom of the Perl Monks concerning the following question: (http and ftp clients)

Originally posted as a Categorized Question.

  • Comment on How can I upload a file using HTTP POST with Content-Type=multipart/form-data?

Replies are listed 'Best First'.
Re: How can I upload a file using HTTP POST with Content-Type=multipart/form-data?
by merlyn (Sage) on Sep 18, 2000 at 08:35 UTC
    I can't tell if you mean "write the server-side code to handle the uploaded file" or "write the client-side code to simulate what a browser does to upload a file".

    If the former, use CGI.pm:

    CREATING A FILE UPLOAD FIELD print $query->filefield(-name=>'uploaded_file', -default=>'starting value', -size=>50, -maxlength=>80); -or- print $query->filefield('uploaded_file','starting value',50 +,80); filefield() will return a file upload field for Netscape 2.0 browsers. In order to take full advantage of this you must use the new multipart encoding scheme for the form. You can do this either by calling start_form() with an encoding type of &CGI::MULTIPART, or by calling the new method start_multipart_form() instead of vanilla start_form(). Parameters 1. The first parameter is the required name for the field (-name). 2. The optional second parameter is the starting value for the field contents to be used as the default file name (-default). For security reasons, browsers don't pay any attention to this field, and so the starting value will always be blank. Worse, the field loses its "sticky" behav- ior and forgets its previous contents. The starting value field is called for in the HTML specification, however, and possibly some browser will eventually provide support for it. 3. The optional third parameter is the size of the field in characters (-size). 4. The optional fourth parameter is the maximum number of characters the field will accept (-maxlength). When the form is processed, you can retrieve the entered filename by calling param(): $filename = $query->param('uploaded_file'); Different browsers will return slightly different things for the name. Some browsers return the filename only. Others return the full path to the file, using the path conventions of the user's machine. Regardless, the name returned is always the name of the file on the user's machine, and is unrelated to the name of the temporary file that CGI.pm creates during upload spooling (see below). The filename returned is also a file handle. You can read the contents of the file using standard Perl file reading calls: # Read a text file and print it out while (<$filename>) { print; } # Copy a binary file to somewhere safe open (OUTFILE,">>/usr/local/web/users/feedback"); while ($bytesread=read($filename,$buffer,1024)) { print OUTFILE $buffer; }

    If the latter, use HTTP::Request::Common:

    The POST method also supports the `multi- part/form-data' content used for Form-based File Upload as specified in RFC 1867. You trigger this content format by specifying a content type of `'form-data'' as one of the request headers. If one of the values in the $form_ref is an array reference, then it is treated as a file part specification with the following interpretation: [ $file, $filename, Header => Value... ] The first value in the array ($file) is the name of a file to open. This file will be read and its content placed in the request. The routine will croak if the file can't be opened. Use an `undef' as $file value if you want to specify the content directly. The $filename is the filename to report in the request. If this value is undefined, then the basename of the $file will be used. You can specify an empty string as $filename if you don't want any filename in the request.
Re: How can I upload a file using HTTP POST with Content-Type=multipart/form-data?
by bhess (Initiate) on Sep 19, 2000 at 06:27 UTC
    Thanks for the feedback... I am looking for the client side HTTP::Request::Common works OK for small files, but I need a solution that can handle very large files (100 MB+). I can upload very large files using a web browser (Netscape and IE). LWP seems to "slurp" the file before writing it to the Content-Disposition section of the HTTP message, thus causing memory woes on the client - this is also very slow. I am looking for some client side Perl that opens a socket and generates the appropriate HTTP to upload any file - no matter the size - here is an example of a very small text file:
    POST /cgi-bin/upload.pl HTTP/1.1 Accept: application/vnd.ms-excel, application/msword, application/vnd. +ms-powerpoint, image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, a +pplication/pdf, */* Referer: http://deville/cgi-bin/upload.pl Accept-Language: en-us Content-Type: multipart/form-data; boundary=-------------------------- +-7d03135102b8 Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT) Host: deville Content-Length: 316 Connection: Keep-Alive -----------------------------7d03135102b8 Content-Disposition: form-data; name="file"; filename="D:\hess\dev\www +\notes\hello.txt" Content-Type: text/plain Hello there -----------------------------7d03135102b8 Content-Disposition: form-data; name="done" done -----------------------------7d03135102b8--
    This is an actual header that I sniffed coming from IE 5 on Win NT. I cannot seem to reproduce this header - even hardcoded - to upload t
      Read further down in the LWP::Request::Common:
      If you set the $DYNAMIC_FILE_UPLOAD variable (exportable) to some TRUE value, then you get back a request object with a subroutine closure as the con- tent attribute. This subroutine will read the content of any files on demand and return it in suitable chunks. This allow you to upload arbitrary big files without using lots of memory. You can even upload infinite files like /dev/audio if you wish. Another difference is that there will be no Content-Length header defined for the request if you use this fea- ture. Not all servers (or server applications) like this.
      Still no reason to write the code yourself. Remember... LWP has had many many man-years of coding put into it. Don't reinvent the wheel.

      -- Randal L. Schwartz, Perl hacker

        I have tried using this with no luck in the past and gave up. I wrote a simple client script using HTTP::Request::Common and had it working with small files and then set the DYNAMIC_FILE_UPLOAD variable at the beginning of the script. And this seemed to break it. Is there something else I need to do? Also, I am using Apache as my server. Is there some setting in Apache that I will need because of no Content-Length? Thanks