Temp file strategy

[Sprad]

I'm working on a web app that needs to dynamically generate temp files for users to download. How is this sort of thing typically handled? There are a couple concerns that I can think of:

* The files need unique names
* The files shouldn't stick around any longer than necessary

I have a session ID, so I can base a reasonably unique filename off of that. But the file expiration has me going in circles. Is there a better way than simply dumping them all in a temp directory and setting a cron job to sweep the old ones?

[marto]

Hi,

You may want to start by looking at the File::Temp module.

Hope this helps.

Martin

[osunderdog]

File::Temp is probably the best place to start.

use constant TEMP_FILENAME_TEMPLATE=>File::Spec::Unix::tmpdir() . "myt
+mp_XXXXXXX';

    my ($tempfh, $tempFilename) = File::Temp::tempfile( TEMP_FILENAME_
+TEMPLATE,
                                                       UNLINK=>1);
    # Write the job text to the temp file.
    print $tempfh $resultJobText;

    # Close the file (flush)
    close($tempfh);

[ikegami]

1) There are modules that generate uniquely named temporary files. (Search CPAN for "temporary files".)

2) Save the file name in the session. Have the session delete the file when it gets deleted.

[blazar]

A "cheap" solution that has not been mentioned yet, relies on an open feature that IMHO should be better known:

open my $temp, '+>', undef or die $!;

[chb]

This is an interesting feature, but how can a user download such a file? If I understand perldoc -f open correctly, this syntax gives you a filehandle to an anonymous temporary file. Will you need some sort of streaming or some webserver trickery to let the user access this file from outside the perl script?

[blazar]

Indeed. But it depends on the OS you're under: I'm not really sure, but I think that Windows doesn't support real anonymous files, so the temporary file that you can see should still be there somewhere.

Here I'm under Linux and I get

$ strace -e open perl -e 'open $f, "+>", undef' 2>&1 | tail -n 1
open("/tmp/PerlIO_vfNALU", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 
+3

but of course it is soon deleted:

$ perl -le 'open $f, "+>", undef;
print readlink "/proc/self/fd/" . fileno $f'
/tmp/PerlIO_vJg2Uf (deleted)

(or

$ perl -le 'open $f, "+>", undef;
print readlink "/proc/self/fd/" . fileno $f'
/tmp/.nfs0004821b00000353

if run on a machine where /tmp is mounted under nfs.)

Whatever, all this is at best fragile and tricky. This interesting open feature is best suited for a file to write stuff into, to recover it later. For your application any of the other suggestions you got would be probably preferable.

I just mentioned this temp file strategy option because IMHO it would deserve to be better known.

[rob_au]

The tutorial node Using Temporary Files in Perl on this site may be of interest to you.

[Tanktalus]

Is it possible to not create the temp file until the user selects to download it, but to provide a link with all the CGI parms needed to generate the file? Then you can generate it directly to stdout (aka, the socket via apache).

I don't think any of the temp file solutions written above will work across invocations of your CGI code since they will all remove the file as your script exits, too early for the user to click on a link to download.

[signal9]

"Is it possible to not create the temp file until the user selects to download it...?"

Sure, just output your preferred filetype w/ proper MIME type just like you would a web page. Nothing fancy at all, and I've been known to do this.

"I don't think any of the temp file solutions written above will work across invocations of your CGI..."

Reread the above example. Files are deleted after an arbitrary "timeout". We do this alot for CSV downloads of reports where I work. Usually we do 24hrs. Sure, if someone runs a report, but doesn't click on the CSV link until sometime tomorrow the file will be gone. I don't think that's unreasonable, but it's really up to you how long the file sticks around.

Of course, these are hand-rolled solutions. I think I'm going to have to look into the File::Temp module folks keep talking about.

[samizdat]

I use the following to keep webserver-generated links (and files) valid for three hours:

#[snip]#
# Finally, remove old spreadsheets from temporary directory

opendir(TMPD, $tmpdir) or die "No tmp Directory! $! \n";

while (my $f = readdir(TMPD))
{
    if ($f =~ /\.xls$/)
    {
    if ((-A "$tmpdir/$f") > 0.125) # if file is older than 3 hours, de
+lete it
    { 
        system('rm', '-f', "$tmpdir/$f") 
        && die "Can't remove $f: $!\n"; 
    }
    }
}

closedir(TMPD);
#[snip]#

The files and the directory need to be writable by the webserver, but since earlier incarnations of the same page create them, that is not a problem.

[rvosa]

Here's what I do: name your files after the value of time. This is (almost?) certain to give you unique names, and it's easy to search through the temp directory and sort by age - sweeping the old ones. Watch out for the corner case where two requests are made within a second. May need to lock the files?