Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: PostgreSQL cursors with Perl.

by ctilmes (Priest)
on Apr 17, 2008 at 13:46 UTC ( #681157=note: print w/replies, xml ) Need Help??


in reply to PostgreSQL cursors with Perl.

I used to keep it in plaintext files, but now I'd want to run some simple queries with it. Hence, I need a database.

What type of queries? What type of data structures/schema are you using?

FWIW, I've found BerkelyDB very nice for simple data structures, and very fast local queries. (I also find PostgreSQL very nice for more complicated data structures and queries...)

Replies are listed 'Best First'.
Re^2: PostgreSQL cursors with Perl.
by atemerev (Beadle) on Apr 17, 2008 at 14:02 UTC

    I have quotes data for around 15 currency pairs (like 'EUR/USD', 'USD/JPY' etc). For every currency pair there is a separate table (there are no inter-table joins) of a following structure:

    id => int primary key, timestamp => bigint, # epoch timestamp in milliseconds bid => decimal, ask => decimal, bid_amount => int, ask_amount => int, data => varchar # lots of additional data that # need not to be indexed nor normalized

    The queries look like:

    SELECT * FROM table WHERE timestamp > ? AND timestamp < ?

    Or:

    SELECT MAX(bid), MIN(bid), MAX(ask), MIN(ask) FROM table WHERE timestamp > ? AND timestamp < ?
Re^2: PostgreSQL cursors with Perl.
by camlet (Novice) on Apr 17, 2008 at 20:41 UTC
    i don't know if sec will work for 15 gig files but sec works very efficiently with monitoring and parsing simultaneously a large amount of log files. hth. http://www.estpak.ee/~risto/sec/
      Thanks for link. Probably I will not use it for this particular task, but this thing definitely is worth looking at.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://681157]
help
Chatterbox?
[choroba]: I keep the snippets in files of their own, and use a Makefile to syntax highlight them and insert them into slides, while also running them and inserting the output if required
[Corion]: choroba: Ooooh - I didn't think of that! I write my presentations as POD and if it "roughly" looks like Perl code, I should also syntax-check that...
[haukex]: Yes sorry I don't run them all the time, my POD tests are only run as author tests (and are excluded when I'm using Devel::Cover)
[Corion]: choroba: Hmm - no, I keep the snippets inline, but as my framework also has support for capturing output etc., maybe I should do the same...
[Corion]: haukex: Yes, that approach is sane, and it heals the fragility of Pod parsers in a nice way while still syntax-checking stuff
[choroba]: Unfortunately, none of it is online
[haukex]: I figured that POD tests make sense, but only as author tests
[choroba]: I mean, the slides are, but not the makefile with scripts to create them
[Corion]: haukex: I've only now arrived at that revelation ;)
[Corion]: choroba: I use spod5, which also has that support, and also implements its own kinda-make stuff

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (10)
As of 2017-02-27 12:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?






    Results (385 votes). Check out past polls.