Fast database access using DBM::Deep help

[Doozer]

Hi guys, I am using DBM::Deep to store some fairly large and complex hashes in databases. I use them to store information for 324 Set Top Boxes. These are separated in to 9 Zones of 36 STBs. In the hash (.db file) each box is a key and the values are a hash of further key/value pairs, for example:

Z1-B1 => { 'Moxa IP Address => '192.168.1.2',
                    'Moxa Port => '4001',
                    'Dusky Port => '01' };

This layout is repeated for all 324 STBs. I have some more databases which store hashes where Groups are defined as the key, and a list of STBs within that group are stored as an array in the value, for example:

'Group 1' => [Z1-B1,Z2-B1,Z3-B1,Z4-B1]

We use this information to get data for each STB so that we can control it using IO::Socket::Inet. Sometimes we control groups of STBs at a time, for example sending power on to all STBs in the group "Zone 1". This group would contain 36 STBs. The script iterates through each box in the group, retrieves their control details from the hash database, and then issues commands to them using that data.

My problem is that when I choose to control a large number of boxes in one go e.g, the group Zone 1 (36 boxes), I sometimes get an error of "Can't use an undefined value as a symbol reference at my_script.pl line xxx"

This error sometimes doesn't happen at all, but when it does, it is not in the same place. The line in the code it refers to is where a connection is made through IO::Socket::Inet. Below is a snippet of my code:

my $duskyip = $controls{"$stb"}{'DuskyMoxaIP'};
my $duskymoxaport = $controls{"$stb"}{'DuskyMoxaPort'};
my $dusky = new IO::Socket::INET(PeerAddr => $duskyip, PeerPort =>$dus
+kymoxaport, Proto => 'tcp', Timeout => 1);
my $duskyport = $controls{"$stb"}{'DuskyPort'};
my $fullcom = "A+$duskyport$rawcmd$closebit";
print $dusky $fullcom; #### LINE THAT ERROR REFERS TO ####

My first thought is that it doesn't retrieve the details needed from the database quick enough to create the IO::Socket::Inet connection. I am currently accessing the database files as follows:

tie %controls, 'DBM::Deep', {file => '/var/www/cgi-bin/Database/E2E_STB_Control_Data.db',locking => 1, autoflush => 1, num_txns => 100};

The "num_txns" option was added in to see if it helped but it doesn't. Any ideas or suggestions will be greatly appreciated.

Thanks

[BrowserUk]

My first thought is that it doesn't retrieve the details needed from the database quick enough to create the IO::Socket::Inet connection.

Without further explanation, that statement doesn't appear to make much sense. There is no way to arrive at the line that (attempts to) open the socket, before the preceding statements that provide the information to make that connection have completed.

What does make sense is that the attempt to create the socket connection is failing for some reason, thus when you get to the erring line, $dusky is still undefined.

The appropriate course of action would be to add error checking to the socket creation:

my $dusky = IO::Socket::INET->new(
    PeerAddr => $duskyip, PeerPort =>$duskymoxaport, Proto => 'tcp', T
+imeout => 1
) or warn "Connection to $duskyip:$duskymoxaport failed because $!/$^E
+" and next XXX;

That would probably give you a good clue as to why the connection is failing (and is good practice anyway!).

A likely candidate for the failure is that your 1 second timeout sometimes isn't enough when you've just opened connections to 30-odd previous machines and the network/stack is very busy. You might just need to extend the timeout; or perhaps add a retry loop.

[Doozer]

Thanks a lot!!

If anything this has helped to uncover what is causing the error. I randomly get "Connection Refused" messages. This largely points to the Moxa devices so I am currently playing around with the settings to allow more than 1 connection at a time. It seems silly that I missed such a simple thing as error checking. At least it means there isn't an issue with the database file or the hash.

Much appreciated :)