Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Streaming download from S3 compatible storage

by kikuchiyo (Hermit)
on Feb 08, 2024 at 15:51 UTC ( [id://11157613]=CUFP: print w/replies, xml ) Need Help??

$subject came up at $work.

The following minimal example appears to work:

#!/usr/bin/perl use Modern::Perl '2021'; use Net::Amazon::S3::Client; # edit these my $aws_access_key_id = '...'; my $aws_secret_access_key = '...'; my $host = '...'; my $bucket_name = '...'; my $secure = 1; my $client = Net::Amazon::S3::Client->new ( host => $host, aws_access_key_id => $aws_access_key_id, aws_secret_access_key => $aws_secret_access_key, secure => $secure, retry => 1, ); my $bucket = $client->bucket( name => $bucket_name ); my $object = $bucket->object( key => $ARGV[0] ); $object->get_callback(sub { my $s = length($_[0]); print STDERR "Got chunk size $s\n"; # do something with $_[0] });

This is applicable if you want to serve a file from an S3 compatible storage via an async backend.

The get_callback method is not documented (it is mentioned in passing only in Net::Amazon::S3::Client::Object::Range), but in the end it works.

Replies are listed 'Best First'.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: CUFP [id://11157613]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2024-06-16 22:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.