Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Re: PostgreSQL cursors with Perl.

by ctilmes (Vicar)
on Apr 17, 2008 at 13:46 UTC ( #681157=note: print w/replies, xml ) Need Help??

in reply to PostgreSQL cursors with Perl.

I used to keep it in plaintext files, but now I'd want to run some simple queries with it. Hence, I need a database.

What type of queries? What type of data structures/schema are you using?

FWIW, I've found BerkelyDB very nice for simple data structures, and very fast local queries. (I also find PostgreSQL very nice for more complicated data structures and queries...)

Replies are listed 'Best First'.
Re^2: PostgreSQL cursors with Perl.
by atemerev (Beadle) on Apr 17, 2008 at 14:02 UTC

    I have quotes data for around 15 currency pairs (like 'EUR/USD', 'USD/JPY' etc). For every currency pair there is a separate table (there are no inter-table joins) of a following structure:

    id => int primary key, timestamp => bigint, # epoch timestamp in milliseconds bid => decimal, ask => decimal, bid_amount => int, ask_amount => int, data => varchar # lots of additional data that # need not to be indexed nor normalized

    The queries look like:

    SELECT * FROM table WHERE timestamp > ? AND timestamp < ?


    SELECT MAX(bid), MIN(bid), MAX(ask), MIN(ask) FROM table WHERE timestamp > ? AND timestamp < ?
Re^2: PostgreSQL cursors with Perl.
by camlet (Novice) on Apr 17, 2008 at 20:41 UTC
    i don't know if sec will work for 15 gig files but sec works very efficiently with monitoring and parsing simultaneously a large amount of log files. hth.
      Thanks for link. Probably I will not use it for this particular task, but this thing definitely is worth looking at.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://681157]
[LanX]: lanx wonders ... how likely is it to talk >95% BS without intention?
[erix]: "gigantic amounts of data" is also not SQLite (imho)
talexb wonders why sqlite2 was deprecated in favour of sqlite3.
[erix]: looks like a fork, rather, no?
LanX /me /me
[erix]: /hehehe
[marto]: LanX yesterday I found out about Gish gallop tactic
[marto]: "Eugenie Scott, executive director of the National Center for Science Education, dubbed this approach the Gish gallop, describing it as "where the creationist is allowed to run on for 45 minutes or an hour, spewing forth torrents of error that the
[marto]: evolutionist hasn't a prayer of refuting in the format of a debate." She also criticized Gish for failing to answer objections raised by his opponents"

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (9)
As of 2017-07-28 15:40 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (431 votes). Check out past polls.