Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Dancer as a proxy

by gsiems (Deacon)
on May 09, 2013 at 19:43 UTC ( #1032829=perlquestion: print w/replies, xml ) Need Help??
gsiems has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to create a new route in dancer that will serve as a proxy of sorts to another service. This other service returns various kinds of files of varying size.

For small files, the following snippet works fine:

my $ua = LWP::UserAgent->new(timeout => 180,); $ua->env_proxy; my $rsp = $ua->head($new_route); send_error('Bad Gateway (head)', 502) unless (ref $rsp && $rsp->is_s +uccess); foreach my $key ('Content-Encoding', 'Content-Length', 'Content-Type +', 'Last-Modified') { my $value = $rsp->header($key) || ''; header($key => $value) if ($value); } $rsp = $ua->get($new_route); send_error('Bad Gateway (get)', 502) unless (ref $rsp && $rsp->is_su +ccess); return $rsp->content;

The problem that I'm having is that, for large files, I would like to be able to stream the response. Now LWP::UserAgent allows for grabbing the response in chunks using a callback and Dancer allows sending data in chunks using send_file with a callback but I (as yet) have no clue how to tie the two together in any workable fashion.

Any ideas how to make this work?

Replies are listed 'Best First'.
Re: Dancer as a proxy
by Corion (Pope) on May 09, 2013 at 19:54 UTC

    Take a look at PSGI and what it says about streaming responses. I don't know if Dancer properly supports streaming, but only because I haven't looked at it closely enough.

      streaming support depends on the webserver as much as the framework , and a grep http://search.cpan.org/grep?cpanid=YANICK&release=Dancer-1.3113&string=streaming&i=1&n=1&C=3 shows Dancer higher than 1.3072 supports streaming

      Streaming data using dancer, Re^3: Resume downloads, Re^2: file download with connection problems, http://search.cpan.org/perldoc/Dancer#send_file

      This was really hard to write as the documentation has holes on this, there are no tests on this in test suite, and Dancer hides the errors from these callbacks, so you have to eval { ...; 1 } or warn $@;

      get '/stream_example' => sub { return send_file( \'fake contents to be replaced in override', streaming => 1, callbacks => { override => sub { eval { my ( $respond, $response ) = @_; require LWP; my $ua = LWP::UserAgent->new; my $writer ; $ua->get( 'http://www.example.com', ':content_cb' => sub { my ( $data, $response, $protocol ) = @_; if(not $writer ){ my $h = $response->headers ; my $r = [ $response->code, [ map { $_ => $h->{$_}, } 'last-modified' , 'date' , 'content-length' , +'etag' , 'content-type' ], ]; $writer = $respond->( $r ); } $writer->write( $data ); }, ## end of :content_cb ); ## end of $ua->get } or warn "\n\n OH NO\n$@\n\n"; return; }, ## end of override }, ); ## end of send_file }; ## end of get '/stream_example'

        Thank you Anonymous Monk! I did need to change it a bit to make it work as I was having issues getting the headers information and things appear to working rather nicely now.

        For future reference, and in case others find it useful, the modified send_file is:

        return send_file( \'fake contents to be replaced in override', streaming => 1, callbacks => { override => sub { eval { my ( $respond, $response ) = @_; require LWP; my $ua = LWP::UserAgent->new; my $writer ; $ua->get( $new_route, ':content_cb' => sub { my ( $data, $response, $protocol ) = @_; if(not $writer ){ my $h = $response->headers ; my %m = map {$_ => $h->header($_)} ('Content-Length', 'Content-Type', 'Last-Modified' +); $writer = $respond->( [ $response->code, [ %m ] ] ); } $writer->write( $data ); }, ## end of :content_cb ); ## end of $ua->get } or Dancer::Logger::error("stream-file: bad eval $@"); return; }, ## end of override }, ); ## end of send_file

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1032829]
Approved by moritz
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (3)
As of 2018-05-23 19:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?