Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: Best Way to Search and Delete files on a large Windows Filesystem

by OzzyOsbourne (Chaplain)
on Mar 08, 2002 at 18:20 UTC ( #150394=note: print w/ replies, xml ) Need Help??


in reply to Best Way to Search and Delete files on a large Windows Filesystem

I have a system for this. The caller script opens a bunch of instances of the main script (1 per server). The main script hunts down all the media files on the servers and puts them into logs called nameofserver.log. Then I run the siftall script to go through all of those logs to sift into logs by file type. Then the deleter script will delete every file listed in a particular log.

The caller script is where the performance gains are made. It allows me to run 100+ servers simultaneously, rather than sequntially. Most use strict. I apologise for those that don't. They were written long ago and never rewritten.

Let me know if you need more info on this...

Main Script

#finds .mp3.avi.exe.mpg.mpeg.mpe.wav.zip.mov.rmj.wma files on a server #called from the run multi scripts #added swf 5/17/01 #added pst 7/27/01 # ADDED ANOTHER NESTED IF FOR //SERVER/USERS # 8.09.01 added .ogg (ogg-vorbis files) use Getopt::Std; use File::Find; getopt('s'); # ********************************* # Process arguments ([h]elp,[s]erver) # ********************************* if ($opt_s){ $server=$opt_s; }else{ print "Please Enter Server name:"; chomp($server=<STDIN>); } $dir1="//$server/e\$/users"; if (!(-e "$dir1")){#if directory doesn't exist try d$ $dir1="//$server/d\$/users"; if (!(-e "$dir1")){ $dir1="//$server/users"; if (!(-e "$dir1")){ die "Directory not does not exist on $server\n...Exiting S +cript.\n"; } } } $out="//workstation/share/serverlogs/$server\.tmp"; $out2="//workstation/share/serverlogs/media/$server\.txt"; open (OUTFILE, ">$out") or die "Cannot open $out for write :$!"; print "finding media files on $dir1\.\.\.\n"; find ({wanted => \&wanted, no_chdir=>1}, $dir1); #find(\&wanted, $dir1); sub wanted { if (!("$File::Find::dir"=~/}/)&&(/\.asf$|\.mp.{0,2}$|\.avi$|\.exe$ +|\.wav$|\.zip$|\.mov$|\.rm.?$|\.wm.?$|\.qt$|\.mid.?$|\.ra.?$|\.swf$|\ +.pst$|\.ogg$|\.gho$/i)){ print OUTFILE "$_\n"; print "$_\n"; } } close OUTFILE; open (OUTFILE, "$out") or die "Can't open"; open (OUTFILE2, ">$out2") or die "Can't open"; @input=<OUTFILE>; foreach (@input){ s/\//\\/g; print OUTFILE2 "$_"; } close OUTFILE; close OUTFILE2; unlink $out;

Calls the Main script

# Created on 9/6/00 @all=('SERVER1','SERVER2); use Win32::Process; sub ErrorReport{ print Win32::FormatMessage( Win32::GetLastError() ); } foreach $server (@all){ Win32::Process::Create($ProcessObj, "c:\\program files\\perl\\bin\\perl.exe", "perl.exe c:\\public\\perl5\\cleanup\\findstuff6.pl -s +$server", 0, NORMAL_PRIORITY_CLASS, ".")|| die ErrorReport(); #$ProcessObj->Suspend(); #$ProcessObj->Resume(); #$ProcessObj->Wait(INFINITE); }

Log Sifter

# sifts all the server logs based on media type # added func to print -ok at end of empty files, Federal added 5/31/01 # added mpga on 6.08.01 # added pst on 7.27.01 # added a backup of old logs a MMDDYY directory based on the last exe. +txt time stamp use strict; use File::Copy; my ($type, $server,$out,$in,@input,$total,$kbytes,$mbytes); my @servers=('SERVER1','Server2'); my $dir1='//workstation/share/serverlogs/media'; my @types=('swf','asf','avi','mp2','mp3','mpg','mpga','mpe','mpeg','wa +v','mov','qt','mid','midi','ra','ram','rmi','rmj','rmx','zip','exe',' +wma','pst','ogg','gho'); ###################### # Create a new directory with the date of the last exe.txt MMDDYY unde +r sifted ###################### my @statarray=stat('c:/share/serverlogs/media/sifted/exe.txt'); my @statarray2=localtime($statarray[9]); my $month=$statarray2[4]+1; my $year=$statarray2[5]-100; my $dirname=sprintf("$dir1/sifted/%.2d%.2d%.2d",$month,$statarray2[3], +($statarray2[5]-100)); ###################### # Backup the old logs to the new dir ###################### opendir(DIR, "$dir1/sifted") or die "can't opendir $dir1: $!"; my @files = grep { !/^\./ && -f "$dir1/sifted/$_" } readdir(DIR); closedir DIR; mkdir ("$dirname")or die "Couldn't mkdir! $!"; foreach (@files){ copy("$dir1/sifted/$_","$dirname")or die "Couldn't Copy! $!"; } unlink <$dir1/sifted/*.txt>; ###################### # Sort the logs ###################### foreach $type (@types){ $total=0; my $out="$dir1/sifted/$type\.txt"; my $out2="$dir1/sifted/$type-ok\.txt"; open OUT, ">$out" or die "Cannot open $out for write :$!"; foreach $server (@servers){ $in="$dir1/$server\.txt"; open IN,"$in" or next; @input=<IN>; chomp @input; foreach (@input){ if (/\.$type$/i){ $kbytes = (stat)[7]/1024; $total+=$kbytes; print OUT "$_\t$kbytes KB\n"; } } close IN; } $mbytes=$total/1024; print OUT "\n\nTotal: $mbytes MB\n"; close OUT; if ($mbytes eq 0){ rename $out, $out2; } print "Finished $type...\n"; }

Delete files by log

#deletes MP3 files noted in the sifted MP3 log # usage: deletemp3.pl mp3[enter] use strict; my $infile="//workstation/share/serverlogs/media/sifted/$ARGV[0].txt"; my $outfile="//workstation/share/serverlogs/media/sifted/$ARGV[0]-dele +ted.txt"; my %filehash; open IN, "$infile" or die "Cannot open $infile for write :$!"; my @input=<IN>; close IN; foreach (@input){ my ($file,$size)=split /\t/; $filehash{$file}=$size; } foreach (sort keys %filehash){ if(-e){ unlink "$_" or warn "\ncan't delete $_:$!\n"; print "$_ deleted\n"; }else{ print"file does not exist\n" } } rename $infile, $outfile;

-OzzyOsbourne


Comment on Re: Best Way to Search and Delete files on a large Windows Filesystem
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://150394]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (6)
As of 2015-07-05 04:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (60 votes), past polls