Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Help with add_part in HTTP::Message for making Batch HTTP requests for Google API

by Anonymous Monk
on Oct 05, 2018 at 20:44 UTC ( #1223595=perlquestion: print w/replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

The Google Batch Indexing API allows batches of HTTP requests in a format specified at: Batch Indexing Example HTTP Request Batch Post request shows an example of using add_part to do this, however is lacking one level of data within the "part" added via add_part, basically it needs a set of headers w/o a (GET/POST request) followed by the content of the actual HTTP request for the contained request Here's a "part":
Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+3> POST /v3/urlNotifications:publish [3] Content-Type: application/json accept: application/json content-length: 58 { "url": "http://example.com/jobs/43", "type": "URL_DELETED" }
I can get add_part to add the 2nd part
POST /v3/urlNotifications:publish [3] Content-Type: application/json accept: application/json content-length: 58 { "url": "http://example.com/jobs/43", "type": "URL_DELETED" }
But not the headers above that via add_part in HTTP::Message
Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+3>
Here's example code that generates a request minus those extra set of headers
#!/usr/bin/env perl use strictures; use HTTP::Request::Common; use JSON; use URI; my $request1 = POST "/v3/urlNotifications:publish", Content_Type => 'application/json', Accept => 'application/json', Content_ID => "1", Content => encode_json({ url => 'http://example.com/1', type => "U +RL_UPDATED" } ); my $request2 = POST "/v3/urlNotifications:publish", Content_Type => 'application/json', Accept => 'application/json', Content_ID => "2", Content => encode_json({ url => "http://example.com/2", type => "U +RL_UPDATED" } ); my $endpoint = URI->new("https://indexing.googleapis.com/batch"); my $base_request = POST $endpoint, Authorization => "Bearer TOKEN", Content_Type => 'multipart/mixed; boundary=xYzZY123'; $base_request->add_part( $request1, $request2 ); print $base_request->as_string;
Outputs:
Authorization: Bearer TOKEN Content-Length: 0 Content-Type: multipart/mixed; boundary=xYzZY123 --xYzZY123 POST /v3/urlNotifications:publish Accept: application/json Content-Length: 51 Content-Type: application/json Content-ID: 1 {"url":"http://example.com/1","type":"URL_UPDATED"} --xYzZY123 POST /v3/urlNotifications:publish Accept: application/json Content-Length: 51 Content-Type: application/json Content-ID: 2 {"url":"http://example.com/2","type":"URL_UPDATED"} --xYzZY123-
But need a way to add the additional headers of to each part
Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+2>
Overriding HTTP::Message's _part_class at: Link to code looks potentially promising, but not quite sure how to best accomplish. I think if I could force those add_part's to become a new class I define, I could override that classes as_string method to include the additional headers needed. I could use some pointers on how to accomplish that --or-- another suggested approach. Another thing I tried was composing the part of 2 HTTP::Requests -- first w/o a METHOD & URL, and those headers with content composed of a 2nd HTTP::Request. That was close to working but I got an extra line and - since I didn't specify a METHOD or URL for the upper (partial request) Any tips to help me get past this final hurdle would be most appreciated! Sample batch indexing format below:
POST /batch HTTP/1.1 Host: indexing.googleapis.com Content-Length: content_length Content-Type: multipart/mixed; boundary="===============73308459742167 +40156==" Authorization: Bearer oauth2_token --===============7330845974216740156== Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+2> POST /v3/urlNotifications:publish [1] Content-Type: application/json accept: application/json content-length: 58 { "url": "http://example.com/jobs/42", "type": "URL_UPDATED" } --===============7330845974216740156== Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+1> POST /v3/urlNotifications:publish [2] Content-Type: application/json accept: application/json content-length: 75 { "url": "http://example.com/widgets/1", "type": "URL_UPDATED" } --===============7330845974216740156== Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+3> POST /v3/urlNotifications:publish [3] Content-Type: application/json accept: application/json content-length: 58 { "url": "http://example.com/jobs/43", "type": "URL_DELETED" } --===============7330845974216740156==

Replies are listed 'Best First'.
Re: Help with add_part in HTTP::Message for making Batch HTTP requests for Google API
by Anonymous Monk on Oct 06, 2018 at 00:41 UTC

    Here's example code that generates a request minus those extra set of headers

    Ugh. Its a request within a request within a request. Its not hard. You dont have to involve LWP until you have to involve it.

    This is almost but not quite it

    #!/usr/bin/perl -- use strict; use warnings; use JSON; use LWP; my $endpoint = URI->new("https://indexing.googleapis.com/batch"); use HTTP::Request::Common(); my $base_request = HTTP::Request::Common::POST( $endpoint, Authorization => "Bearer TOKEN", Content_Type => 'multipart/mixed', ); $base_request->add_part( BlaFoo({ url => 'http://example.com/1', type => "URL_UPDATED" }), BlaFoo({ url => "http://example.com/2", type => "URL_UPDATED" }) ); print $base_request->as_string; exit 0 ; sub BlaFoo { return SnaFoo( POST => "/v3/urlNotifications:publish", ## todo? Content_Type => 'application/json', Accept => 'application/json', Content => encode_json( shift ) ); } sub SnaFoo { my $req = HTTP::Message->new; my $content; $content = shift if @_ and ref $_[0]; my($k, $v); while (($k,$v) = splice(@_, 0, 2)) { if (lc($k) eq 'content') { $content = $v; } else { $req->push_header($k, $v); } } $req->header('Content-Length' => length($content)) unless ref($con +tent); $req->content($content); $req = HTTP::Message->new( undef, $req->as_string ); $req->header( 'Content-Type' => 'application/http' ); $req->header( 'Content-Transfer-Encoding' => 'binary' ); $req->header( 'Content-ID' => '<b29c5de2-0db4-490b-b421-6a51b598bd +22+2>' ); ## todo? return $req; } __END__

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1223595]
Approved by stevieb
Front-paged by vroom
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2018-10-20 10:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    When I need money for a bigger acquisition, I usually ...














    Results (117 votes). Check out past polls.

    Notices?
    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!