Streaming download from S3 compatible storage

[kikuchiyo]

$subject came up at $work.

The following minimal example appears to work:

#!/usr/bin/perl

use Modern::Perl '2021';
use Net::Amazon::S3::Client;

# edit these
my $aws_access_key_id = '...';
my $aws_secret_access_key = '...';
my $host = '...';
my $bucket_name = '...';
my $secure = 1;

my $client = Net::Amazon::S3::Client->new (
    host                  => $host,
    aws_access_key_id     => $aws_access_key_id,
    aws_secret_access_key => $aws_secret_access_key,
    secure                => $secure,
    retry                 => 1,
);

my $bucket = $client->bucket( name => $bucket_name );

my $object = $bucket->object( key => $ARGV[0] );

$object->get_callback(sub {
    my $s = length($_[0]);
    print STDERR "Got chunk size $s\n";
    # do something with $_[0]
});

This is applicable if you want to serve a file from an S3 compatible storage via an async backend.

The get_callback method is not documented (it is mentioned in passing only in Net::Amazon::S3::Client::Object::Range), but in the end it works.