Getting information from an array of hashes

[chumley]

hello Monks,

I have a problem at work that involves getting data from diary fields in a Remedy database. I'm using ARSPerl, which gets the data as an array of hashes. The ARSPerl documentation suggests a ways to get the data out of the hashes, but it doesn't work. I tried about 4 other things and they didn't work. One of these was from section 4.7.3 of Programming Perl.

I won't bore you with the details of every attempt. The last thing I tried looks like this:

    my %hash = $diary_data[0];
    my @temp1 = keys %hash;
    print "%hash\n";
    foreach $temp2 (@temp1) {
        print "$temp2\n";
        print "$hash{$temp2}\n";
    }

This printed:

Reference found where even-sized list expected at ./send_referral.pl l
+ine 200.<BR>
Use of uninitialized value at ./send_referral.pl line 204.<BR>
ARRAY(0x3c05dc)

(Line 200 starts with "my %hash" in the snippet above.

At this point, I'm wondering if there is a bug somewhere in this package? I have the same problem running this script on Solaris and on NT. I'm wondering about the error about an "even-sized list expected".

Thanks for the help.

Chumley

[AgentM]

When you assign to hash like that, Perl is expecting to see an even-number-sized list of elements to put in the hash- key/ value pairs. If $diary_data[0] is a reference (it is) to the array where the key/value pairs are (this I don't know), then you'll want to use

@{$diary_data[0]};

This resolves the pointer/reference and "enters" the array into the hash as key/value pairs. Then your print statements will work fine. Also, be sure to use strict;, use warnings;, and use diagnostics; for better error-reporting, though I'm not sure if you've already throw those in.

Update: Ovid's node is probably right, now that I reread your statement "returns array of hashes".

Update2: AgentM wins! Fatality!


AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

[chumley]

If I put in a line that says

print "$diary_data[0]\n";

it prints:

ARRAY(0x3bfda0)

If I change the print slightly, like this:

print "@{$diary_data[0]}\n";

This is what I get:

HASH(0x3bfdc4)

Then if I try

print "%{$diary_data[0]}\n";

it gives me:

%{ARRAY(0x3bfdd0)}

I don't think I'm understanding this correctly. The first example looks to me like a scalar reference, pulling data out of an array; but since the array contains hashes, then how does this get pulled out? Is this "ARRAY(0x3bfda0)" line an internal reference to how Perl remembers something?

Anyway, I'm off to read perldsc for a while.

Chumley

[AgentM]

Try Data::Dumper. Thanks for posting this since this helps us out alot in disagnosing the problem. What you don't want to do is print references, since they're rarely helpful, but this case is an exception. We are shown that you probably have an array of arrays which you should be assigning to a hash, so my above suggestion for an assignment would be best.

You're right. What you see being printed is an internal representation of Perl's memory management, which isn't very helpful to you- so use Data::Dumper to get the right debugging info. I think we even have a Tutorial here about it. Good luck!


AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

[Ovid]

I believe what you want is:

my %hash = @{ $diary_data[0] };

The "Reference found where even-sized list expected" stems from you trying to assign a scalar reference to a hash, but a hash expects a list with an even number of elements. You can reproduce that error from the command line:

perl -w -e "$a[0]=[1,2];%hash=$a[0]"

The following eliminates the error:

perl -w -e "$a[0]=[1];%hash=@{$a[0]}"

Cheers,
Ovid

Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

Update: Fixed error in original reply. I had %{ $diary_data[0] } instead of @{ $diary_data[0] }.

Update 2: Just saw AgentM's update. He was actually right. This code:

my %hash = %{ $diary_data[0] };

Will force a "Can't coerce array into hash" error.

[hdp]

You should probably read perldoc perldsc so that you understand why your code is wrong.

In a nutshell, it's because anything inside a data structure is a scalar -- either a plain scalar or a reference (e.g. to an array or a hash). There are no arrays inside arrays, only arrayrefs inside arrays (or inside hashrefs inside arrays, or... etc.).

hdp.

[chumley]

Thanks for the reference. I've looked through it, and there's some good stuff there to digest. However, one of the examples in perlman:perldsc is almost identical to something I already tried, following an example in Programming Perl.

    for ($i = 0 ; $i <= $#diary_data ; $i++ ) {
        print "$i is { ";
        for $temp_keys ( keys %{ $diary_data[$i] } ) {
            print "$temp_keys=$diary_data[$i]{$temp_keys} ";
        }
        print "}\n";
    }

This still generates an error. I broke it down to do one step at a time, like this:

 for ($i = 0 ; $i <= $#diary_data ; $i++ ) {
        print "I: $i\n";
        for $temp_keys ( keys %{ $diary_data[$i] } ) {
            print "Key: $temp_keys\n";
            print "Data: $diary_data[$i]{$temp_keys}\n";
        }
    }

This gives

I: 0
Key: timestamp
Use of uninitialized value at ./send_referral.pl line 197.
Data: 
Key: value
Argument "NWORLASTE800 LOGIN FAILED\nNWORLASTE801 LOGIN FAIL\n\nWA..."
+ isn't numeric in helem at ./send_referral.pl line 197.
Bad index while coercing array into hash at ./send_referral.pl line 19
+7.
$

However, if I comment out the line that starts with print "Data...", it prints the correct keys for this hash:

I: 0
Key: timestamp
Key: value
Key: user

What confuses me is the fact that I'm following examples straight out of Perl references and getting these errors. I'm starting to wonder if I've found a bug in ARS.pm.

Chumley

[hdp]

"However, one of the examples in perldsc is almost identical to something I already tried, following an example in Programming Perl."

I'm not sure that's true. What it seems you're trying to do -- i.e. take an arrayref and turn it into a hash -- isn't really covered in perldsc. The reason I pointed you to it was because of my %hash = $diary_data[0] and the accompanying error about Reference found where even-sized list expected, not because of anything in the other suggestions people made or other things you'd tried; I'm sorry if there was confusion there.

What happens if you try %hash = %{$diary_data[0]->[0]}? (This is a bit of a shot in the dark based on some of your other posts.)

"What confuses me is the fact that I'm following examples straight out of Perl references and getting these errors. I'm starting to wonder if I've found a bug in ARS.pm."

You may be right, but it's probably just a nested data structure that you haven't quite understood yet. Data::Dumper will help a lot, if that's the case (print Dumper($diary_data[0])).

hdp.

[chumley]

Update:

I wanted to add some details of what I've already tried. First of all, the function that returns this array of hashes is not mine. It's part of the ARSPerl package. The actual function is called ars_GetEntry() which returns a hash of all the fields in a form. I looked through ARS.pm and it looks like this function is coded in C, not Perl, so it doesn't look like I'll be able to hack it. (Unless my boss is willing to pay for me to take a C class... :-) )

When calling the function, one of the arguments is the index of the particular record I want to retrieve. The keys to the hash are the field ID numbers from that form. This works fine for all the other fields I need to get - I have no problem retrieving and printing all the other information I need.

If one of the fields is a 'diary' field (Which I think is specific to Remedy), then that element of the hash is an array of hashes. Does that make sense? It took me a few times reading to understand it. IOW, quoting the documentation for this function:

ars_GetEntry(ctrl,schema,entry_id,...)

This function returns a hash (associative array) of field ids and values for a given schema and entry id. If you specify any field ids after the entry_id parameter, only the values for those fields will be returned, otherwise all field id value pairs are returned. All field values are converted into numeric or string values, except for the diary type. The diary field type is encoded as an array of hashes. Each hash has a timestamp, user and value field.

Here is one thing I have done that works, sort of:

@diary_entry = keys %{$diary_data[0]};
for $temp (@diary_entry) {
    print "$temp\n";
}

This returns:

$ ./send_referral.pl 0000079100
Diary ID: 536870914
timestamp
value
user

According to the docs, this is what I should see for the keys. Next, change the print statement, like so:

print "$temp: $diary_data[0]{$temp}\n";

This was interesting:

$ ./send_referral.pl 0000079100
Diary ID: 536870914
Use of uninitialized value at ./send_referral.pl line 201.
timestamp: 
Argument "NWORLASTE800 LOGIN FAILED\nNWORLASTE801 LOGIN FAIL\n\nWA..."
+ isn't numeric in helem at ./send_referral.pl line 201.
Bad index while coercing array into hash at ./send_referral.pl line 20
+1.
$

The gibberish that looks like "NWORLASTE800 LOGIN FAILED\nNWORLASTE801 LOGIN FAIL\n\nWA..." looks like part of what I should see. This part of what should be in the data, under %hash{value}, and could run to thousands of characters. (The actual limit for this field is 500KB)

Finally, here are two things I have tried and the result:

@diary_entry = values %{$diary_data[0]};
print "@diary_entry\n";

and

 # Another try:
    my %hash = %{$diary_data[0]};
    @diary_entry = keys %hash;
    #@diary_entry = keys %{$diary_data[0]};
    print "@diary_entry\n";
    for $temp (@diary_entry) {
        print "$temp\n";
    }

give me an error that says "Out of memory during ridiculously large request at ./send_referral.pl line 194." (Line 194 says "my %hash = %{$diary_data[0]")

So, to summarize, I have an array of hashes. I can get the keys to those hashes but I can't get any data out of them. Thank you Ovid, AgentM, and hdp for the suggestions. I'll be reading them carefully to make sure I understand them.

Chumley

[chumley]

Data::Dumper showed me the way. What I had was not an "array of hashes". It was, as far as I can tell, an array containing one element, and that element was an array, containing the hashes.

I finally figured it out when I tried this on a hunch:

    print "${${$diary_data[0]}[0]}{user}\n";
    print "${${$diary_data[0]}[0]}{timestamp}\n";
    print "${${$diary_data[0]}[0]}{value}\n";

which gave me some good output:

MAWEBB
988227105
NSA             NOTIFY NETWORK MAINTENANCE              4/24/01       
+  22:05 CST

From there, I was able to come up with:

$i=0;
while (${${$diary_data[0]}[$i]}{user}) {
    print "User:\t\t${${$diary_data[0]}[$i]}{user}\n";
    print "Timestamp:\t${${$diary_data[0]}[$i]}{timestamp}\n";
    print "Information:\t${${$diary_data[0]}[$i]}{value}\n\n";
    $i++;
}

This gives me exactly the information I'm looking for. All that's left is munging it into a decent format for e-mail.

There are still some things I need to do with cleaning up this code, and I need to go back over exactly why this works the way it does, but I think I'm on the right track here. Thanks again Monks for the help today! :-)

Chumley

[hdp]

Glad to see you've figured out what this is.

A little easier to write, and much easier to read, would be:

for $hashref (@{$diary_data[0]}) {
  print "User:\t\t$hashref->{user}\n";
  print "Timestamp:\t$hashref->{timestamp}\n";
  print "Information:\t$hashref->{value}\n\n";
}

If specifically testing for the user key is important, add a next unless $hashref->{user} or something.

hdp.