Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^2: PSQL and many queries

by BrowserUk (Patriarch)
on Mar 24, 2009 at 01:27 UTC ( [id://752767]=note: print w/replies, xml ) Need Help??


in reply to Re: PSQL and many queries
in thread PSQL and many queries

My first thoughts were to use the system memory to load data file lines into a hash and load the DB records into an array (or visa versa) and check against each other. My hesitation on that was the memory load if the lists are large and more than one user runs this script

There's no need to load all of both datasets into memory.

If you load the names from the DB into a hash (which DBI will do for you in one hit--select names from table; fetchall_hashref), then you can process your file line by line.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^3: PSQL and many queries
by citycrew (Acolyte) on Mar 24, 2009 at 02:15 UTC
    There's no need to load all of both datasets into memory. If you load the names from the DB into a hash (which DBI will do for you in one hit--select names from table; fetchall_hashref), then you can process your file line by line.

    Of course!.. thanks BrowserUk

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://752767]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (7)
As of 2024-04-18 03:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found