Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Can anyone help with protocol buffers?

by Cody Fendant (Friar)
on Jun 09, 2019 at 04:24 UTC ( #11101151=perlquestion: print w/replies, xml ) Need Help??

Cody Fendant has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to read information from a service which only provides it in Protocol Buffer format.

I'd never heard of it until now, and this is what I understand about it:

  1. It's binary
  2. you don't just read it like JSON, you have to know the format of it first
  3. you know the format because you get a proto file and parse that
  4. this can be done by https://metacpan.org/pod/Google::ProtocolBuffers
  5. Google::ProtocolBuffers returns a list of accessor objects, essentially modules created on the fly
  6. Then you grab data from your binary file by using one of the accessors

I've got as far as step 5. Now what?

I have the binary file, and the proto file which describes the format, but I can't figure out how to access the data.

The data is basically hundreds of these entities (bus information for Sydney Australia):

entity { id: "42558_202397_3000_41_1" vehicle { trip { trip_id: "892878" start_time: "16:05:00" start_date: "20190609" schedule_relationship: SCHEDULED route_id: "3000_41" } position { latitude: -33.03611 longitude: 151.66042 bearing: 332.0 speed: 2.2 } timestamp: 1560060858 congestion_level: UNKNOWN_CONGESTION_LEVEL vehicle { id: "42558_202397_3000_41_1" [transit_realtime.tfnsw_vehicle_descriptor] { air_conditioned: true wheelchair_accessible: 1 vehicle_model: "Volvo~B7RLE~Bustech~VST" special_vehicle_attributes: 0 } } occupancy_status: MANY_SEATS_AVAILABLE } }

And this is what the module tells me are the accessors:

[ 'TransitRealtime::Alert::Cause', 'TransitRealtime::Alert::Effect', 'TransitRealtime::FeedHeader::Incrementality', 'TransitRealtime::TripDescriptor::ScheduleRelationship', 'TransitRealtime::TripUpdate::StopTimeUpdate::ScheduleRelati +onship', 'TransitRealtime::VehiclePosition::CongestionLevel', 'TransitRealtime::VehiclePosition::OccupancyStatus', 'TransitRealtime::VehiclePosition::VehicleStopStatus', 'TransitRealtime::Alert', 'TransitRealtime::EntitySelector', 'TransitRealtime::FeedEntity', 'TransitRealtime::FeedHeader', 'TransitRealtime::FeedMessage', 'TransitRealtime::Position', 'TransitRealtime::TfnswVehicleDescriptor', 'TransitRealtime::TimeRange', 'TransitRealtime::TranslatedString', 'TransitRealtime::TranslatedString::Translation', 'TransitRealtime::TripDescriptor', 'TransitRealtime::TripUpdate', 'TransitRealtime::TripUpdate::StopTimeEvent', 'TransitRealtime::TripUpdate::StopTimeUpdate', 'TransitRealtime::VehicleDescriptor', 'TransitRealtime::VehiclePosition' ];

What I would like to do is parse it like I would parse JSON. Say I want to see where that bus (route_id: "3000_41") is from that data, what should I do? What's the equivalent of parsing through like

foreach $entity ( @entities ){ if($entity->route_id eq '3000_41'){ print "bus is at " . $position->{latitude} . ', ' . $position +->{longitude}; } }

Replies are listed 'Best First'.
Re: Can anyone help with protocol buffers?
by ForgotPasswordAgain (Curate) on Jun 09, 2019 at 11:44 UTC
Re: Can anyone help with protocol buffers?
by bliako (Priest) on Jun 09, 2019 at 05:32 UTC

    based on this from Google::ProtocolBuffers's synopsis:

    ## ## Decode data from serialized form ## my $person; { open my($fh), "<person.dat"; binmode $fh; local $/; $person = Person->decode(<$fh>); close $fh; } print $person->{name}, "\n"; print $person->name, "\n"; ## ditto

    you should somehow end up with an $entities object. Then use dd to dump it and see if you can find about its methods or stored hashkeys.

    use Data::Dump qw/dd/; my $feed = ...; # parse the data print dd($feed); # this is a long shot, it will be clearer when you find what $feed con +tains # it is based on https://github.com/OneSkyWalker/protobuf-java-example +s/blob/master/ListBusInfo.java # getEntityList() is from the java api, what's in Perl? Seave through +the dd output above. #foreach $entity ($feed->getEntityList){ # print dd($entity); # my $veh = $entity->vehicle; # my $position = $veh->position; # print "bus is at " . $position->{latitude} . ', ' . $position->{lo +ngitude}; #} # or $entity = $feed->EntitySelector("42558_202397_3000_41_1"); print dd($entity);

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11101151]
Approved by haukex
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (7)
As of 2019-06-27 02:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Is there a future for codeless software?



    Results (111 votes). Check out past polls.

    Notices?