Unexplained flock() errors

[jpfarmer]

I'm working on a script using DBI and DBD::CSV with Perl 5.6.0 on Solars. Every time it's run, I get flock() error messages that I can't explain, such as this one:

Execution ERROR:  Cannot obtain exclusive lock on /u/jpfarmer/apps/shi
+ftrpt/data/consultants: No record locks available at /u/jpfarmer/lib/
+site_perl/5.6.0/DBD/File.pm line 500.
 called from ./reporter at 30.

There is no way that another process is using the database (the program is only being run by me), and so I don't understand how the lock can be unavalible. I get similar error messages every time the Database is accessed. Also, if make a quick hack to to File.pm from DBD::CSV and force it to disable file locking, everything works fine.

I wrote a quick test script to try to apply a flock() to the database file, and it was successful, so the problem seems limited to my usage here. What could be going on?

sub build_consultant_data() {
    $dbh = DBI->connect("DBI:CSV:f_dir=$CSV_DATA")
      or die "Cannot connect: " . $DBI::errstr;    # Connect to the DB

    my $directory =
      $ua->get('http://www.somwhere.com/blah/blah/blah/')
      ;                                            # Get directory
    my $te =
      new HTML::TableExtract(
                            headers => [qw/Name Int Username Group/] )
      ;                                   # Set up for TableExtract
    $te->parse( $directory->content );    # Extract the pertinent data
    my $ts =
      $te->first_table_state_found()
      ;    # Create a table_state object to get at the rows

    $sth =
      $dbh->prepare(
               "INSERT INTO $CONSULTANT_TABLE VALUES (?, ?, ?, ?, ?)")
      or die "Cannot prepare: "
      . $dbh->errstr();    # Prepare to insert fields.
    my $i = 0;             # Initalize counter
    foreach my $row ( $ts->rows ) {    # Handle all the rows of data
        next unless @$row[0]; # Dump the row if the real name is empty
        my ( $ln, $fn ) = split ( /,/, @$row[0] ); # Extract last name
        $fn = ( split ( / /, $fn, 1 ) )[0];    # Remove middle name
        $sth->execute( $i++, $fn, $ln, @$row[2], @$row[3] );
    }

    $dbh->disconnect();    # Disconnect from the DB
}

[Abigail-II]

It's a system error. DBD::File is trying to flock the database file by doing a system call. This call fails, due to the shown reason.

There will be a maximum number of locks your system can deal with, and there might be a limit per user, process of file as well. You might want to consult Stevens' "Advanced programming in the UNIX environment". You also might want to consult your system administrator. Perhaps there's a process out of control, consuming all the available locks. Perhaps NFS is involved. Perhaps you need to modify a kernel parameter - which might require a kernel rebuild, which in turn might require a reboot.

There are too many unknowns to say anything concrete. It's out of Perls hand, you've entered the territory of the OS.

Abigail

[Cabrion]

Not sure I can help with the flock issue, but in general Perl 5.6.0 is a bit buggy and DBD::CSV is quite buggy. Try upgrading DBD::CSV and its dependents.

As I offered here 231494, if your just using DBD::CSV to do inserts you might try replacing that code by using Text::CSV_XS instead.