Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Preventing Oversized Uploads

by Kirsle (Pilgrim)
on Dec 04, 2009 at 17:49 UTC ( #811128=perlquestion: print w/replies, xml ) Need Help??

Kirsle has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

My web server was relatively unresponsive earlier today and I ssh'd in thinking that maybe Apache had crashed and I was going to restart it. Instead, I discovered that about 7 Apache-owned processes were all chugging away taking up >= 90% CPU.

The processes were all a CGI script I have on one of the sites that lets a user upload a TTF font file, and the server converts it into an EOT file using ttf2eot. This is the kind of thing that I explicitly went out of my way to avoid when I was first programming this CGI script.

The script sets $CGI::POST_MAX to a limit of 1 MB (as I've never seen a TTF file that is greater than 1 MB), and then when it calls the ttf2eot binary it wraps it in an eval with an alarm clock set to 10 seconds.

One theory I had as to why these processes might've been running away is that somebody tried to upload a very large file to it. So, for some reason the $CGI::POST_MAX wasn't working? So I started testing it by uploading oversized files myself and seeing what the script could do about it. I tried a couple other things such as setting an explicit check for $ENV{CONTENT_LENGTH}, setting a CGI upload hook to try to kill the script as soon as $bytes_read exceeded 1 MB, and set a global alarm timer of 15 seconds for the whole script.

  • In all three cases, I uploaded the installer rpm for VirtualBox which is 40MB -- well over my 1MB limit set. In all three cases, my browser waited and uploaded the whole file before showing any output from the CGI script.
  • With the 15 second alarm for the entire script set, as soon as the browser finished uploading it immediately gave me my error message saying the timer had been reached. Does this imply the CGI was running the entire time and died after 15 seconds and Apache just waited for the rest of my upload before telling me? The upload itself took about 2 minutes.
  • With the $ENV{CONTENT_LENGTH} check, again it waited until the entire file was uploaded before it showed me my error message saying the content length was too long.
  • Setting a CGI upload hook to die when too many bytes were read didn't seem to have any effect.

Isn't there a way to get the script to just abort itself as soon as it detects that the user is going to be troublesome? What if a user decides to upload a 4 GB DVD image or something? Why does Apache wait until the file is completely uploaded before letting the user know that the CGI script already isn't a fan of what they're currently trying to do?

Preferably the CGI script should've just aborted the upload after 1 MB and immediately shown them an error message. Even more preferably it should've tested the CONTENT_LENGTH before even entertaining the idea of an upload and telling them right away that what they're uploading is too large.

It's said everywhere that scripts should set $CGI::POST_MAX as a method to prevent DDoS attacks on the script which happen when people upload super large files. But what does $CGI::POST_MAX even do for you? From what I can tell it just causes param() to return undef on everything, so most likely they'll see the default form page again or something and not the results page. But it doesn't prevent the file from being received by the server, or stored in a temporary directory? Isn't this still a way to DDoS a server, by filling up its hard drive from the temp files?

Any insights about this?

Here is the full code of the TTF to EOT CGI:

#!/usr/bin/perl -w # ttf2eot - Convert TTF files to EOT files. use strict; use warnings; use CGI; use CGI::Carp qw(fatalsToBrowser); use Digest::MD5 qw(md5_hex); use lib "./lib"; # Don't let the process run away. #$SIG{ALRM} = sub { # die "<h1>Operation Timed Out</h1>\n\n" # . "The maximum time limit for this script to run has been rea +ched. This generally " # . "shouldn't happen unless you uploaded something really weir +d. Contact " # . "<a href=\"mailto:***\\">Kirsle</a> with details + if you feel this was " # . "in error."; #}; #alarm(15); if ($ENV{CONTENT_LENGTH} > 1024*1024) { die "Content length too long"; } # Go through and delete old files. &deleteOldFiles(); # Enforce upload size limits. $CGI::POST_MAX = 1024 * 1024; # 1 MB my $cgi = new CGI (\&upload_hook); my $action = $cgi->param('action') || 'index'; # Set an upload hook just so we can enforce POST_MAX during the upload +. sub upload_hook { my ($filename,$buffer,$bytes_read,$data) = @_; if ($bytes_read > $CGI::POST_MAX) { die "File size is too large! Abort!"; } } print "Content-Type: text/html\n\n"; print qq~<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www"> <html> <head> <title>ttf2eot on the web!</title> <style type="text/css"> body { background-color: #FFFFFF; font-family: Verdana,Arial,Helvetica,sans-serif; font-size: small; color: #000000 } a:link, a:visited { color: #0000FF; text-decoration: underline } a:hover, a:active { text-decoration: none } h1 { font-family: Verdana,Arial,Helvetica,sans-serif; font-size: xx-large; font-weight: bold; color: #000000; margin-top: 0px; margin-bottom: 20px; padding: 0px; } pre { border: 1px solid #990000; font-family: "Lucida Console","Courier New",Courier,monospace; font-size: small; color: #000000; padding: 5px; display: block } </style> </head> <body>~; if ($action eq 'upload') { my $filename = $cgi->param ('font'); my @parts = split(/(\/|\\)/, $filename); my $name = pop(@parts); # Filter the file's name, since it's going into the shell. $name =~ s/[^A-Za-z0-9\_\-\.]/_/g; # Determine the file's type. my ($type) = $name =~ /\.([^.]+?)$/; $type = lc($type); if ($type ne "ttf") { &printError("The uploaded file must be a TrueType Font (TTF)!" +); } # Download the file. my $handle = $cgi->upload ('font'); my $bin = ''; while (<$handle>) { $bin .= $_; } # Save it to a temporary file. mkdir ("./ttf2eot-temp") unless (-d "./ttf2eot-temp"); my $tmp = md5_hex($ENV{REMOTE_ADDR} . $ENV{HTTP_USER_AGENT} . time +()); while (-d "./ttf2eot-temp/$tmp") { $tmp = md5_hex (int(rand(99999))); } mkdir ("./ttf2eot-temp/$tmp"); open (WRITE, ">./ttf2eot-temp/$tmp/$name"); binmode WRITE; print WRITE $bin; close (WRITE); # Make the EOT filename. my $output = $name; $output =~ s/\.ttf$/.eot/ig; $output .= ".eot" unless $output =~ /\.eot$/i; # Run ttf2eot on our file. eval { local $SIG{ALRM} = sub { die; }; alarm(10); my $app = "/home/cuvou/public_html/wizards/bin/ttf2eot"; my $cmd = "$app < ./ttf2eot-temp/$tmp/$name > ./ttf2eot-temp/$ +tmp/$output"; my $result = `$cmd`; alarm(0); }; # Was there a problem? if ($? != 0 || (-s "./ttf2eot-temp/$tmp/$output" == 0) || $@) { &printError("The conversion has failed! Maybe your uploaded TT +F file isn't valid?",$tmp); } &printSuccess($tmp,$name,$output); } else { &printIndex(); } #--------------------------------------------------------------------- +---------# # Index Page + # #--------------------------------------------------------------------- +---------# sub printIndex { print qq{ <h1>TTF to EOT Font Converter</h1> Use this tool to convert a TrueType (TTF) font file into an OpenType ( +EOT) font file, for use with Internet Explorer's embedded font support. After us +ing this tool, you will be able to embed fonts on your web pages that can be se +en on Internet Explorer 4 and higher, and all current modern web browsers th +at support embedded fonts via CSS3 (Firefox 3.5 and higher are among such browser +s).<p> <form name="upload" action="ttf2eot.cgi" method="post" enctype="multip +art/form-data"> <input type="hidden" name="action" value="upload"> <fieldset> <legend>Upload a TrueType Font</legend> Use this form to upload a TrueType Font file to be converted. Browse a +nd select a ".ttf" file.<p> <input type="file" name="font" size="40"><p> <input type="submit" value="Convert TTF to EOT!"> </fieldset> </form> <p> <strong>Note:</strong> on every TTF font file I've personally tested t +his on, the conversion to EOT takes mere seconds (usually less than 1 or 2 sec +onds). If what you upload causes the script to run for too long, the conversi +on will be aborted. If you ever see this error when you upload a legitimate TT +F file (e.g. you're not just <em>trying</em> to break the script) there will +be an e-mail link so you can tell me about it. <h1>About This Program</h1> The <strong>TTF to EOT Converter</strong> is a web-based tool for crea +ting OpenType EOT font files from TrueType TTF font files. Once you have a +pair of TTF and EOT fonts, you can use both of them to embed the font on yo +ur web pages in a way that works with both Internet Explorer and CSS3 web bro +wsers such as Firefox 3.5 and Safari 4.<p> <h1>About Font Embedding</h1> The ability to embed fonts on web pages was originally implemented by +Microsoft in Internet Explorer 4.0 - the catch was that these font files needed +to be in a custom form of OpenType format, with an EOT file extension. The othe +r catch is that embedding EOT files <strong>only</strong> works in Internet Ex +plorer. Therefore, it didn't really catch on.<p> More recently, the new CSS 3 added a specification for embedding fonts on web pages in a more open, standardized way. Browsers that support t +he full CSS 3 specification can render web pages which embed a TrueType font f +ile.<p> New browsers such as Firefox 3.5 therefore support <em>TrueType Fonts< +/em> to be embedded on pages, whereas Internet Explorer supports <em>OpenType +Fonts</em>. Firefox 3.5 won't render OpenType, and Internet Explorer won't render +TrueType. To get around this problem, both types of fonts may be embedded on a p +age at the same time. <a href=" +ml">This is a demonstration page</a> that embeds my Rive font, for your referen +ce. <h1>Instructions:</h1> <ol> <li>Upload a valid TrueType Font file and click the button. If all goes well, you'll be able to download the EOT font file on the next page.</li> </ol> <h1>Author</h1> Casey Kirsle &copy; 2009<br> [<a href=""></a>]<p> This web tool is a front-end to the <a href=" +ttf2eot/"> ttf2eot</a> converter program, written by <a href=" +com/u/taviso/"> taviso</a>. Casey Kirsle merely wrote this front-end. </body> </html>}; exit(0); } #--------------------------------------------------------------------- +---------# # Error Page + # #--------------------------------------------------------------------- +---------# sub printError { my $str = shift; my $pj = shift || ''; # Delete this project? if (length $pj) { &deleteProject ($pj); } print qq~ <h1>Processing Error</h1> Your request could not proceed due to the following error:<p> <strong>$str</strong> </body> </html>~; exit(0); } #--------------------------------------------------------------------- +---------# # Success Page + # #--------------------------------------------------------------------- +---------# sub printSuccess { my ($tmp,$file,$eot) = @_; print qq{ <h1>Success</h1> Your TrueType font, <strong>$file</strong>, has been successfully conv +erted into an EOT font. Use the download link below to save your EOT font file.<p> <h1>Demonstration</h1> If you are running a modern CSS 3 compliant web browser, or Internet E +xplorer, you should see your custom font embedded in the paragraph below:<p> <style type="text/css"> \@font-face { font-family: EotDemonstration; src: url("ttf2eot-temp/$tmp/$eot"); /* EOT for IE */ } \@font-face { font-family: EotDemonstration; src: url("ttf2eot-temp/$tmp/$file"); /* TTF for CSS3 */ } span.demo { font-family: EotDemonstration; font-size: 14pt } </style> <span class="demo">ABCDEFGHIJKLMNOPQRSTUVWXYZ<br> abcdefghijklmnopqrstuvwxyz<br> 123456789.:,;(:*!?&apos;&quot;)<br> The quick brown fox jumps over the lazy dog.</span><p> <h1>Download Your Font</h1> To download your EOT font file, right-click the link below and choose "Save Link As..." or "Save Target As..." -- or whatever vocabulary you +r web browser uses.<p> <a href="ttf2eot-temp/$tmp/$eot">Download $eot</a> <h1>Instructions</h1> To embed this font on your web page, you will need to place the follow +ing CSS code in the &lt;head&gt; section of your page, or in an external C +SS file:<p> <pre>&lt;style type="text/css"&gt; \@font-face { font-family: MyCustomFont; src: url("$eot") /* EOT file for IE */ } \@font-face { font-family: MyCustomFont; src: url("$file") /* TTF file for CSS3 browsers */ } &lt;/style&gt;</pre> Note that in the code above, "<code>MyCustomFont</code>" can be any na +me you want. You will probably want to set this to be the real name of yo +ur font. This script was not able to determine the name of your font - so +rry!<p> To use your embedded font, you can simply refer to it by name like you + do with any other font. Some examples: <pre>body { font-family: MyCustomFont, Verdana, Arial, sans-serif; font-size: medium; color: black } span.header { font-family: MyCustomFont, Impact, "Arial Black", sans-serif; font-weight: bold; color: red }</pre> <b>Note:</b> your generated EOT file will be deleted from this web server after 24 hours. </body> </html>}; exit(0); } #--------------------------------------------------------------------- +---------# # Subroutines + # #--------------------------------------------------------------------- +---------# sub deleteOldFiles { # Expiration date. my $life = 60*60*24; # 24 Hours. # Projects to be deleted. my @flagged = (); # Dig through our temporary directory. if (-d "./ttf2eot-temp") { opendir (DIR, "./ttf2eot-temp"); foreach my $project (readdir(DIR)) { next if $project =~ /^\./; next unless -d "./ttf2eot-temp/$project"; # Descend into the project directory. opendir (PJ, "./ttf2eot-temp/$project"); foreach my $file (readdir(PJ)) { next if $file eq '.'; next if $file eq '..'; # Check the time stamp on these files. my ($mtime) = (stat("./ttf2eot-temp/$project/$file"))[ +9]; # Has it expired? if (time() - $mtime >= $life) { # Flag this project for deletion. push (@flagged,$project); last; } } closedir (PJ); } closedir (DIR); # Process the flagged directories. if (scalar(@flagged)) { foreach my $project (@flagged) { &deleteProject ($project); } } } } sub deleteProject { my $project = shift; opendir (DIR, "./ttf2eot-temp/$project"); foreach my $file (readdir(DIR)) { next if $file eq '.'; next if $file eq '..'; # Delete this file. unlink ("./ttf2eot-temp/$project/$file"); } closedir (DIR); # Remove the directory. rmdir ("./ttf2eot-temp/$project"); }

Replies are listed 'Best First'.
Re: Preventing Oversized Uploads
by zwon (Abbot) on Dec 04, 2009 at 19:49 UTC

    I think Apache starts CGI script only after it has received the whole request. So your script isn't started till Apache receive all data, and Apache doesn't have an idea about limit set by your script. Use LimitRequestBody directive in Apache config instead.

      That was my thought too at one point. But then I set that 15 second alarm clock for the whole script. In Google Chrome you can see the % uploaded while it sends the file to the server; immediately after it got to 100% I got my error message about the timeout expiring. So it looked like Apache started my CGI script earlier and it waited for a while (perhaps when the CGI object is created it blocks until Apache receives the whole request?)

      Setting an upload hook is how you can create a progress bar for file uploads; in this case the CGI script is running since the beginning of the request. Although in my experience if the upload hook crashes (intentionally or due to run-time error) it doesn't affect the rest of the script.

      Anyway, I looked into that Apache config instead. I put a limit of 5 MB for the directory that contains this (and other) CGI scripts - none of the other CGI scripts accept inputs greater than 5 MB (in fact none of them accept higher than 1 MB but I'm adding some wiggle room). Tried uploading VirtualBox again instead of a font file, and after 5 MB was uploaded Apache reset my connection. :)


Re: Preventing Oversized Uploads
by zentara (Archbishop) on Dec 05, 2009 at 11:54 UTC
    ... to build on zwon's lead, you might try testing the ENV..... oops i see you did

    Nodereaper please delete

    ...on second thought....if the

    if ($ENV{CONTENT_LENGTH} > 1024*1024) { die "Content length too long"; }
    dosn't catch it.... you may need to use some Server directive available in your Cpanel ( or other site server admin control panel ) .... google for "apache upload limit directive"

    you may be forced to set this, call your ISP on how to do it, if you can't find out from their menus..... there may also be a .htaccess entry that you can use on a per directory basis.... it's all up to the ISP setup

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://811128]
Approved by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (4)
As of 2021-09-18 14:47 GMT
Find Nodes?
    Voting Booth?

    No recent polls found