Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Net::SFTP::Foreign Transferred Bytes

by yulivee07 (Sexton)
on Apr 26, 2016 at 13:24 UTC ( #1161544=perlquestion: print w/replies, xml ) Need Help??

yulivee07 has asked for the wisdom of the Perl Monks concerning the following question:

Hi fellow Perlmonks! I am currently having fun with Net::SFTP::Foreign. I am modifying a deamon which monitors a directory and automatically uploads files via sftp to a server. If there are multiple files, the deamon forks a child process for every file and transfers them in parallel.
Now I want to do some better error handling. Wenn a transfer aborts, I want to print the number of bytes which were transferred. I can quite get this to STDOUT with the "more => -v" option.
my $sftp = Net::SFTP::Foreign->new(host =>$host, user => $user, key_pa +th => [$privkey, $pubkey] , more => '-v'); [... debug output ...] Transferred: sent 32700048, received 60544 bytes, in 1.6 seconds Bytes per second: sent 20997978.1, received 38877.7 [... more debug output ..]
My first approach was, to grep that output for the "Transferred" line. While this does work, that seems a bit overkill.
I also tried utilizing the callback function of sftp->put like this:
my $BYTES = 0; $sftp->put( "testfile", "testfile", cleanup => 1, callback => sub { my ( $sftp, $data, $offset, $size ) = @_; print $offset ." of ". $size ." bytes copied\n"; last_bytes($offset); + } ); print "\nTransferred $BYTES bytes\n"; + sub last_bytes { my ($bytes) = @_; $BYTES = $bytes; }

However this is giving me a hard time, as I would have to track a variable per child I fork.

In contrast, WWW::Curl has a method for getting the number of transferred bytes:

$curl->getinfo( CURLINFO_SIZE_UPLOAD );
I hope I am not missing something from the documentation for getting the transferred bytes from Net::SFTP::Forerign.... Any Ideas? I get the feeling I am nearly there with the callback-solution...

Kind regards,

Replies are listed 'Best First'.
Re: Net::SFTP::Foreign Transferred Bytes
by stevieb (Canon) on Apr 26, 2016 at 14:00 UTC

    If you fork() for every file, you can use a hash to store the information. This doesn't include all of the SFTP stuff, but hopefully its enough to give you an idea you can merge in with the callback sub feature of Net::SFTP::Foreign.

    use warnings; use strict; use Data::Dumper; use Parallel::ForkManager; my @files = qw(one.txt two.txt); my %info; my $pm = Parallel::ForkManager->new; FILE: for my $file (@files){ $pm->start and next FILE; $info{$file}{size} = -s $file; $pm->finish; } $pm->wait_all_children; print Dumper \%info; __END__ $VAR1 = { 'two.txt' => { 'size' => 21 }, 'one.txt' => { 'size' => 30 } };

    Then you can process the collected data after all the workers have finished.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1161544]
Approved by stevieb
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (5)
As of 2023-12-02 21:48 GMT
Find Nodes?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?

    Results (19 votes). Check out past polls.