<?xml version="1.0" encoding="windows-1252"?>
<node id="1014426" title="Re^5: Finding and sorting files in massive file directory" created="2013-01-21 06:21:27" updated="2013-01-21 06:21:27">
<type id="11">
note</type>
<author id="341121">
dave_the_m</author>
<data>
<field name="doctext">
If there is a process adding new files to the directory while your "tar up" script is running, then you need to face the twin issues of deleting files which haven't been put in the tar file, and putting empty or half-written files into the tarball. If possible, you need to be able to stop the process from adding any new files while the script is running; but if you can't, then the following should be safe.
&lt;p&gt;
Use the script I gave you above to, for example, move all files starting with 'b' into a b/ subdirectory. Then wait a few minutes, or however long it could reasonably take for the process to finish writing the current file, then from the command line, simply:
&lt;code&gt;
$ tar -cfz .../some-path/b.tar.gz b/
$ tar -tfz .../some-path/b.tar.gz &gt; /tmp/foo

View /tmp/foo in a text editor to see if it looks reasonable, then

$ rm -rf b/
&lt;/code&gt;
If the rm fails due to too many files, then write another perl script similar to the one above, but using 'unlink' to remove each file one by one.

&lt;p&gt;
Dave.</field>
<field name="root_node">
1014320</field>
<field name="parent_node">
1014410</field>
</data>
</node>
