Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

POST'ing a large File with LWP::UserAgent

by overrider (Acolyte)
on May 03, 2009 at 05:19 UTC ( #761560=perlquestion: print w/replies, xml ) Need Help??
overrider has asked for the wisdom of the Perl Monks concerning the following question:

Hello there, I have a Program that shall upload Files. It actually works very well with smaller Movies, but when tried for example on a 300MB File it will just crash with an Error like
Out of memory during "large" request for 536875008 bytes, total sbrk() + is 7065600 bytes at /usr/local/lib/perl5/site_perl/5.8.9/HTTP/Reques +t/ line 160.
What id like to do now is to build a callback function, so that when I POST the file, it will basically read the File chunk by chunk. That way even if i upload a 1GB Movie, Memory used stays pretty much the same. I tried to implement it like this
my $filename = "largefile.avi"; open(FH,"<$filename"); my $upload_cb = sub { if ( sysread FH, my $buffer, 4096 ){ return $buffer; } else { close FH; return ''; } } # later in the Code, when uploading the File, calling it like this $vid_fields{field_uploadfile} = [ $upload_cb ] ; $vid_fields{field_private_share_entities} = ""; $resp = upload_step_two($action_url, \%vid_fields);
When running the Program, i get a
Can't open file CODE(0x821d6d4): No such file or directory at /usr/loc +al/lib/perl5/site_perl/5.8.9/LWP/ line 401
Can someone give me a simple Example of POSTing a large File when using LWP::Useragent? Thanks a lot

Replies are listed 'Best First'.
Re: POST'ing a large File with LWP::UserAgent
by Anonymous Monk on May 03, 2009 at 06:21 UTC
    That is not a complete program.
      #!/usr/bin/perl -- use strict; use warnings; use LWP 5.825; my $url = "http://foo/upload"; my $file = __FILE__;#"foo/bar"; my $ua = LWP::UserAgent->new; $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1; use HTTP::Request::Common; my $req = POST( $url, Content_Type => 'multipart/form-data', Content => [ file => [$file] ], ); # set up callback { my $gen = $req->content(); die unless ref($gen) eq "CODE"; my $i = 0; $req->content( sub { my $chunk = &$gen(); # get chunk of data warn $i++; return $chunk; # send it to $url } ); #use Data::Dumper;die print Data::Dumper->new([($ua,$req, $gen)])->Ind +ent(1)->Deparse(1)->Dump; } my $res = $ua->request($req); #do it print $res->status_line;

        Thanks for the feed back. Hi can you explain what the below code actually do

        # set up callback { my $gen = $req->content(); die unless ref($gen) eq "CODE"; my $i = 0; $req->content( sub { my $chunk = &$gen(); # get chunk of data warn $i++; return $chunk; # send it to $url } );

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://761560]
Approved by McDarren
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (11)
As of 2017-04-25 15:36 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (458 votes). Check out past polls.