<?xml version="1.0" encoding="windows-1252"?>
<node id="1000896" title="Re^4: Processing ~1 Trillion records" created="2012-10-25 12:06:39" updated="2012-10-25 12:06:39">
<type id="11">
note</type>
<author id="647953">
sundialsvc4</author>
<data>
<field name="doctext">
&lt;p&gt;
Indeed, if you can extract a list of what the so-called &amp;ldquo;markers&amp;rdquo; and &amp;ldquo;lines&amp;rdquo; are (which of course you &lt;em&gt;can)&lt;/em&gt;, then the problem decomposes itself quite-nicely into partitions. &amp;nbsp; All that you really need are indexes on the columns in question. 
&lt;/p&gt;&lt;p&gt;
It might also be advantageous to define a &lt;tt&gt;view&lt;/tt&gt; that corresponds to this query, so that you can examine and then optimize-the-hades-out-of that view. &amp;nbsp; In some DBMSes, defining a view serves to introduce the query as one that it now knows to be frequently-used, and it certainly puts that &amp;ldquo;6 days long&amp;rdquo; requirement squarely into the DBA&amp;rsquo;s management field-of-view.
&lt;/p&gt;

</field>
<field name="root_node">
1000794</field>
<field name="parent_node">
1000845</field>
</data>
</node>
