several hundred thousand to a milion or more key value pairs
How big are the keys and values on average? And how big are the xml files on average?
I am working on a blade server with 24 physical CPU's and more than 500gb of RAM
Is the blade server set up as a single SMP system? How many cores/threads per cpu?
Given those resources and requirements...what are your thoughts?
I'd want to see the answers to the above questions before reaching any conclusions about how I would go about tackling the problem.
Sight (public or private) of a typical example of the XML input and the keys/value pairs derived from it would help.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
In reply to Re^3: dynamic number of threads based on CPU utilization