<?xml version="1.0" encoding="windows-1252"?>
<node id="1004091" title="Stream file from one HTTP server to another with HTTP GET and PUT requests" created="2012-11-15 19:44:25" updated="2012-11-15 19:44:25">
<type id="115">
perlquestion</type>
<author id="961">
Anonymous Monk</author>
<data>
<field name="doctext">
&lt;p&gt;I do not have a lot of experience w/ perl IO or LWP, WWW::Curl and POE::Component::Client::HTTP but I am looking for a way to stream a file from one server to another.&lt;/p&gt;

&lt;p&gt;I can acomplish it with curl but I will be copying millions of files and I am trying to avoid the overhead of all the extra forks.  Plus I will be threading the whole thing using a Boss/Worker model.&lt;/p&gt;

&lt;p&gt;Basically I'm trying to duplicate the following all in perl.&lt;br&gt;&lt;br&gt;
&lt;code&gt;curl http://server1/path/file.dat | curl --upload-file - http://server2/upload/file.dat&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;The trick is the streaming part.  I can get LWP and WWW::Curl::Easy to GET/PUT files from disk, but I really need to stream them.  I will be moving some large files and can't afford to GET then PUT a file.  Plus the disk IO may reduce my overall throughput.  Holding the content in memory is also out.  I really do need to stream...  That's where I'm stuck.&lt;/p&gt;

&lt;p&gt;I can probably patch something together w/ WWW::Curl::Multi but as I was looking around there was some suggestion to use POE::Component::Client::HTTP instead of WWW::Curl::Multi for someone else's task.&lt;br/&gt;&lt;br/&gt;

Regards,&lt;br/&gt;
-Alan&lt;/p&gt;</field>
<field name="reputation">
3</field>
</data>
</node>
