Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^3: Print to Less Screen

by armstd (Friar)
on Jun 05, 2011 at 14:28 UTC ( #908190=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Print to Less Screen
in thread Print to Less Screen

A BerkeleyDB tied hash temp file might be a good choice. Your hash key would be your sort criteria, the value would ideally be the regexp replaced fields, unless you really need the original data for something else.

Once you're done reading/storing the SQL results, it's a simple matter to read the hash back in sorted order, doing the printf formatting and less piping then.

You might also consider saving aside the BerkeleyDB temp files as an expiring cache if you get a lot of common queries that don't necessarily need absolutely up-to-date data, and/or user can specify when they do need the latest data. You're doing a lot of I/O here, it's not gonna be fast. Some users might be willing to trade data currency for response time.

Is there any specific reason you're not making the SQL query directly from perl? Assuming its possible, that would likely reduce the complexity a great deal, and increase the reliability. Avoiding data problems introduced by parsing raw text output from another program is always good.

--Dave


Comment on Re^3: Print to Less Screen
Re^4: Print to Less Screen
by bigbot (Beadle) on Jun 06, 2011 at 01:44 UTC

    Thanks for your response Dave. The reason I am not doing SQL queries directly from Perl is that I do not have permission to access the DB directly. The person who runs the DB has created an intermediary tool that allows me to pass in SQL queries and it then sends back all the results like this:

    Input:

    sql_query_tool "SELECT * FROM table..."

    Output:

    Field1||Field2||Field3||Field4 Field1||Field2||Field3||Field4 ...

    Also, I have sent my sort criteria to the DB using the query tool, and it works fine. The problem is that certain fields are blank in some cases, so it causes the sort to get screwed up in the following way:

    category type1 type2 type3 1. fruit apple orange banana 2. fruit apple tangerine 3. deserts pie cake icecream

    When I try to sort the fields in order of category -> type1 -> type2 -> type3 I have a problem. Sometimes the type fields can be blank. What I want is for these blank fields to be ignored by sort and the next field to be sorted on. Instead what currently happens is the blank field is sorted on which always makes it first. In the example above record 2 is missing the type2 field. When I do the sort it will put record 2 before record 1. Then when I print out the final data it will look like this:

    fruit apple tangerine fruit apple orange banana

    So when printed out tangerine comes before orange, because of that blank field being sorted on. This is why I came to the conclusion that I could not sort using the DB, and needed to parse the data first myself and then sort.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://908190]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (8)
As of 2014-10-02 08:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (52 votes), past polls