Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Querying Hive tables using DBD::ODBC

by astroboy (Chaplain)
on Oct 29, 2018 at 03:25 UTC ( #1224818=perlquestion: print w/replies, xml ) Need Help??

astroboy has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to query data from a Hive table on a MapR platform. My query is:

use Modern::Perl; use DBI; use Data::Dumper; my $dbh = DBI->connect('dbi:ODBC:DSN=myDSN', undef, undef, {RaiseError + => 1}); $dbh->{LongReadLen} = 1000000; $dbh->{LongTruncOk} = 1; my $sth = $dbh->prepare(q{select * from table}); $sth->execute; while (my $row = $sth->fetchall_arrayref) { say Dumper $row; }

What I get is the following line repeated hundreds of times to STDERR

DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17.
Then I get the one row in the table, followed by a never-ending stream of the following to STDOUT
$VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = []; $VAR1 = [];
And the following to STDERR
DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17. DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing ... repeat forever ....
I have to Control-C to stop this. Any explanation and way forward is appreciated

Replies are listed 'Best First'.
Re: Querying Hive tables using DBD::ODBC
by poj (Abbot) on Oct 29, 2018 at 07:34 UTC
    ->fetchall_arrayref fetches all records so you don't need to loop (unless you don't have enough memory to fetch and return all the rows in one go.)

    my $rows = $sth->fetchall_arrayref; say Dumper $rows;

    Try fetching each row using ->fetchrow_arrayref.

    #while (my $row = $sth->fetchall_arrayref) {
    while (my $row = $sth->fetchrow_arrayref) {
       say Dumper $row;
    }
    
    poj
      Duh! You wouldn't think I'd been using DBI since the late 90s! Thanks for the fresh pair of eyes - I couldn't see the wood for the trees
Re: Querying Hive tables using DBD::ODBC
by afoken (Canon) on Oct 29, 2018 at 07:27 UTC
    use Modern::Perl; use DBI; use Data::Dumper; my $dbh = DBI->connect('dbi:ODBC:DSN=myDSN', undef, undef, {RaiseError + => 1}); $dbh->{LongReadLen} = 1000000; $dbh->{LongTruncOk} = 1; my $sth = $dbh->prepare(q{select * from table}); $sth->execute; while (my $row = $sth->fetchall_arrayref) { say Dumper $row; }

    What I get is the following line repeated hundreds of times to STDERR

    DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17.

    The code you are showing us is not the code you are running. You posted only 15 lines, so there is no line 17.

    Show the code that is really running. If it is to long (more than about 30 lines), copy it, shorten it to the relevant parts, run it. Post the shortened code and the error messages and warnings it generates.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      Yip, I had the code in one window and the output in another, I remved three lines to make the code more succinct but forgot to run the output again

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1224818]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (7)
As of 2022-06-30 08:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My most frequent journeys are powered by:









    Results (97 votes). Check out past polls.

    Notices?