Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Temp file strategy

by Sprad (Hermit)
on Sep 15, 2005 at 15:48 UTC ( #492294=perlquestion: print w/replies, xml ) Need Help??

Sprad has asked for the wisdom of the Perl Monks concerning the following question:

I'm working on a web app that needs to dynamically generate temp files for users to download. How is this sort of thing typically handled? There are a couple concerns that I can think of:

* The files need unique names
* The files shouldn't stick around any longer than necessary

I have a session ID, so I can base a reasonably unique filename off of that. But the file expiration has me going in circles. Is there a better way than simply dumping them all in a temp directory and setting a cron job to sweep the old ones?

A fair fight is a sign of poor planning.

Replies are listed 'Best First'.
Re: Temp file strategy
by marto (Archbishop) on Sep 15, 2005 at 15:51 UTC

    You may want to start by looking at the File::Temp module.

    Hope this helps.

Re: Temp file strategy
by osunderdog (Deacon) on Sep 15, 2005 at 15:55 UTC

    File::Temp is probably the best place to start.

    use constant TEMP_FILENAME_TEMPLATE=>File::Spec::Unix::tmpdir() . "myt +mp_XXXXXXX'; my ($tempfh, $tempFilename) = File::Temp::tempfile( TEMP_FILENAME_ +TEMPLATE, UNLINK=>1); # Write the job text to the temp file. print $tempfh $resultJobText; # Close the file (flush) close($tempfh);

    Hazah! I'm Employed!

Re: Temp file strategy
by ikegami (Pope) on Sep 15, 2005 at 15:53 UTC

    1) There are modules that generate uniquely named temporary files. (Search CPAN for "temporary files".)

    2) Save the file name in the session. Have the session delete the file when it gets deleted.

Re: Temp file strategy
by blazar (Canon) on Sep 15, 2005 at 16:54 UTC
    A "cheap" solution that has not been mentioned yet, relies on an open feature that IMHO should be better known:
    open my $temp, '+>', undef or die $!;
      This is an interesting feature, but how can a user download such a file? If I understand perldoc -f open correctly, this syntax gives you a filehandle to an anonymous temporary file. Will you need some sort of streaming or some webserver trickery to let the user access this file from outside the perl script?
        Indeed. But it depends on the OS you're under: I'm not really sure, but I think that Windows doesn't support real anonymous files, so the temporary file that you can see should still be there somewhere.

        Here I'm under Linux and I get

        $ strace -e open perl -e 'open $f, "+>", undef' 2>&1 | tail -n 1 open("/tmp/PerlIO_vfNALU", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = +3
        but of course it is soon deleted:
        $ perl -le 'open $f, "+>", undef; print readlink "/proc/self/fd/" . fileno $f' /tmp/PerlIO_vJg2Uf (deleted)
        $ perl -le 'open $f, "+>", undef; print readlink "/proc/self/fd/" . fileno $f' /tmp/.nfs0004821b00000353
        if run on a machine where /tmp is mounted under nfs.)

        Whatever, all this is at best fragile and tricky. This interesting open feature is best suited for a file to write stuff into, to recover it later. For your application any of the other suggestions you got would be probably preferable.

        I just mentioned this temp file strategy option because IMHO it would deserve to be better known.

Re: Temp file strategy
by rob_au (Abbot) on Sep 16, 2005 at 13:15 UTC
    The tutorial node Using Temporary Files in Perl on this site may be of interest to you.


    perl -le "print unpack'N', pack'B32', '00000000000000000000001000000000'"

Re: Temp file strategy
by Tanktalus (Canon) on Sep 16, 2005 at 00:10 UTC

    Is it possible to not create the temp file until the user selects to download it, but to provide a link with all the CGI parms needed to generate the file? Then you can generate it directly to stdout (aka, the socket via apache).

    I don't think any of the temp file solutions written above will work across invocations of your CGI code since they will all remove the file as your script exits, too early for the user to click on a link to download.

      "Is it possible to not create the temp file until the user selects to download it...?"

      Sure, just output your preferred filetype w/ proper MIME type just like you would a web page. Nothing fancy at all, and I've been known to do this.

      "I don't think any of the temp file solutions written above will work across invocations of your CGI..."

      Reread the above example. Files are deleted after an arbitrary "timeout". We do this alot for CSV downloads of reports where I work. Usually we do 24hrs. Sure, if someone runs a report, but doesn't click on the CSV link until sometime tomorrow the file will be gone. I don't think that's unreasonable, but it's really up to you how long the file sticks around.

      Of course, these are hand-rolled solutions. I think I'm going to have to look into the File::Temp module folks keep talking about.

Re: Temp file strategy
by samizdat (Vicar) on Sep 15, 2005 at 17:56 UTC
    I use the following to keep webserver-generated links (and files) valid for three hours:
    #[snip]# # Finally, remove old spreadsheets from temporary directory opendir(TMPD, $tmpdir) or die "No tmp Directory! $! \n"; while (my $f = readdir(TMPD)) { if ($f =~ /\.xls$/) { if ((-A "$tmpdir/$f") > 0.125) # if file is older than 3 hours, de +lete it { system('rm', '-f', "$tmpdir/$f") && die "Can't remove $f: $!\n"; } } } closedir(TMPD); #[snip]#
    The files and the directory need to be writable by the webserver, but since earlier incarnations of the same page create them, that is not a problem.
Re: Temp file strategy
by rvosa (Curate) on Sep 17, 2005 at 11:51 UTC
    Here's what I do: name your files after the value of time. This is (almost?) certain to give you unique names, and it's easy to search through the temp directory and sort by age - sweeping the old ones. Watch out for the corner case where two requests are made within a second. May need to lock the files?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://492294]
Approved by marto
Front-paged by grinder
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (10)
As of 2020-09-22 11:45 GMT
Find Nodes?
    Voting Booth?
    If at first I donít succeed, I Ö

    Results (129 votes). Check out past polls.