Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: Parsing JSON out of an incremental stream

by Sec (Monk)
on Feb 15, 2019 at 17:52 UTC ( #1229974=note: print w/replies, xml ) Need Help??


in reply to Parsing JSON out of an incremental stream

I have no cobbled together my own solution. It's less ugly than I expected.

Most tricky part (for me) was to come up with a regexp that works well enough for me. (I don't correctly deal with hashes inside arrays)

our $store; $ua->add_handler( response_data => sub { my($response, $ua, $h, $data) = @_; $store.=$data; eval { if ($store =~ s/^\s*({\s*(?:"[^"]*"|(?>[^"{}]* +)|(?1))+\s*})\s*//){ my $result = $json->decode( $1 ); print Dumper $result; }; }; if ($@){ print STDERR "Error: $@\n"; die }; 1; } );

Replies are listed 'Best First'.
Re^2: Parsing JSON out of an incremental stream
by Your Mother (Bishop) on Feb 15, 2019 at 18:53 UTC

    I strongly encourage you to look at huck's link instead and use the incremental parsing in concert with some higher level knowledge of the objects/arrays to expect to control what to do with the data as it becomes complete.

Re^2: Parsing JSON out of an incremental stream
by Sec (Monk) on Feb 18, 2019 at 17:40 UTC
    For anyone following along.
    $ua->add_handler( response_data => sub { my($response, $ua, $h, $data) = @_; eval { my $result = $json->incr_parse( $data ); if (defined $result){ print Dumper $result; }; }; if ($@){ print STDERR "Error: $@\n"; die }; 1; } );
    is a pretty nice solution in my eyes.

    The eval/catch is a bit of an eyesore, but LWP seems to silently eat all errors in handlers, and just not call them anymore which makes debugging quite difficult.

    Also make sure you install JSON::XS - the "core" version of JSON::PP may decide to die on you.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1229974]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (8)
As of 2019-10-21 14:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?