http://www.perlmonks.org?node_id=1017339

perlCrazy has asked for the wisdom of the Perl Monks concerning the following question:

Hi, getting this error while decoding the josn object via post method. surprosingly it works fine when GET method is used. client code:
use Data::Dumper; use JSON; use URI::Escape; use HTTP::Request::Common; use HTTP::Response; use LWP::UserAgent; my $rest_url = "some url"; my $inData = { "name" => "test", "id" => "test", "test" => [1,2,3,4] }; my $json = JSON->new->utf8->allow_nonref; my $jsonData = $json->encode($inData); my $apiCmd = "/projectname/restfulapi?inputdata=$jsonData"; my $iua = LWP::UserAgent->new; #my $response = $iua->request(GET $rest_url.$apiCmd); my $response = $iua->request(POST $rest_url.$apiCmd); if ($response->is_success) { #print Dumper($response->content()); my $perlData = $json->decode($response->content()); print Dumper($perlData); } else{ print " inside falied block\n"; }
api code:
use strict; use Readonly; use CGI; use Data::Dumper; use JSON; #use cgi.pm to get json as input my $q = CGI->new; my $jsonText = $q->param('inputdata'); my $json = JSON->new->utf8->allow_nonref(1); my $outputDataRef = $json->decode($jsonText); print qq{Content-type: application/json; charset=utf8\n\n}; my $outputData = $json->encode($outputDataRef); print $outputData;
It fails at api side while decoding. error details: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)")
Any help will be greatly appreciated, Thanks.

Replies are listed 'Best First'.
Re: Getting error while decoding JSON object via POST method
by tobyink (Canon) on Feb 06, 2013 at 06:39 UTC

    The error is on these two lines.

    my $jsonText = $q->param('inputdata'); my $outputDataRef = $json->decode($inputData);

    If you'd used strict Perl would have told you about the undeclared variable $inputData.

    package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name
      Sorry for not providing the complete code. updated the code just now. problem is not with variable definition, please check the error:
      malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)"

        Have you printed out your JSON data string at the receiving end to see what you get there? An empty string gives the same error message:

        >perl -MJSON -e "print JSON->new->decode('')" malformed JSON string, neither array, object, number, string or atom, +at character offset 0 (before "(end of string)") at -e line 1.

        I think you are trying to pass the data around in a very bad manner:

        <c> "/projectname/restfulapi?inputdata=$jsonData"; <c>

        If you send POST data, please read HTTP::Request on how to send POST data. If you want to send the data in an URI, please encode it properly using URI::Encode.

        This error message occurs when you pass the empty string or an undefined variable to be decoded by JSON, JSON::PP or JSON::XS.

        So I'd suggest that even with the updated code, the problem is related to the initialization of $jsonText - it appears to be undef.

        package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name
Re: Getting error while decoding JSON object via POST method
by tobyink (Canon) on Feb 06, 2013 at 09:48 UTC

    Here is an example of a CGI script that accepts JSON via HTTP POST, reads two numbers "a" and "b" from the input, and adds them together returning "c", plus an example client that uses the script...

    #!/usr/bin/env perl use strict; use warnings; use CGI (); use JSON (); my $q = CGI->new; my $json = JSON->new->utf8; my $input = $json->decode( $q->param('POSTDATA') ); my $a = $input->{a}; my $b = $input->{b}; my $c = $a + $b; print $q->header("application/json"); print $json->encode({ c => $c });
    #!/usr/bin/env perl use strict; use warnings; use JSON (); use LWP::UserAgent (); use HTTP::Request::Common qw( POST ); my $data = { a => 40, b => 2 }; my $json = JSON->new->utf8; my $ua = LWP::UserAgent->new; my $req = POST( "http://buzzword.org.uk/2013/json-addition.cgi", Content_Type => 'application/json', Content => $json->encode($data), ); my $result = $json->decode( $ua->request($req)->decoded_content ); print $result->{c}, "\n";
    package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name
      thanks for response, still getting same error. i tried code : cleint :
      use strict; use warnings; use JSON (); use LWP::UserAgent (); use HTTP::Request::Common qw( POST ); use Data::Dumper; my $data = { a => 40, b => 2 }; my $json = JSON->new->utf8->allow_nonref; my $ua = LWP::UserAgent->new; my $req = POST( "$my_url", Content_Type => 'application/json', Content => $json->encode($data), ); my $response = $ua->request($req); if ($response->is_success) { #print Dumper($response->content()); my $perlData = $json->decode($response->content()); print Dumper($perlData); } else{ print " inside falied block\n"; }
      server side:
      #use cgi.pm to get json as input my $q = CGI->new; my $json = JSON->new->utf8; my $input = $json->decode( $q->param('POSTDATA') ); #my $input = $q->param('POSTDATA'); my $a = $input->{'a'}; my $b = $input->{'b'}; my $c = $a + $b; print $q->header("application/json"); #print Dumper($input); print $json->encode({ c => $c });
      output of run : ./test_client.pl $VAR1 = { 'error_msg' => 'Failed during execution: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") ', 'status_code' => 500 }; Thanks
        And the reason you don't DUMP the data is because?
      I'm getting this exact same error when trying to access the body of an application/json POST when the request is HTTP 1.1. If it's an HTTP 1.0 request, all is fine. HTTP 1.1 requests don't set the "Content-Length" header and use "Transfer-Encoding" with a value of "chunked". Does anyone know the proper way to grab the JSON payload of a POST request that will work with both HTTP 1.0 and HTTP 1.1?

        Implementing a transfer encoding on the server side is not easy. I would try to prevent sending data using the "chunked" transfer encoding. Depending on your client library, there are different ways to prevent that.

Re: Getting error while decoding JSON object via POST method
by Mr. Muskrat (Canon) on Feb 07, 2013 at 20:00 UTC

    my $perlData = $json->decode($response->content());

    You don't want the raw content returned by $response->content(). You want the decoded content as returned by $response->decoded_content(). tobyink had it correct in his reply but he didn't explicitly call it out and you didn't pick up on it.

      I know that this is an old thread, but I'm having exactly this problem and the answer presented isn't entirely satisfactory. What I mean is that the problem presented used the CGI module on the server side, but the solution presented (using $request->decoded_content instead of $request->content) applies to the client script, which uses LWP, but not to the server script. I don't see anything in the CGI module that will do the required decoding. In my case, I'm using Apache and an Apache::Request object, which also does not (from what I can find on the web) chunked data. I have no control over the format of the data arriving at my server. Most of the time my script works, presumably because it's usually not chunked. I am using Fiddler to debug my script. When I first inspect the response, the POST body appears to have stray characters before and after the actual JSON code. However, Fiddler tells me that I need to decode it, which I do by simply clicking a button, after which the JSON appears correct. This leads me to believe that I'm having exactly the problem described in this thread. If I had an HTTP::Request object, I would just call $req->decoded_content() but the only way that I know to get one is using LWP, which doesn't make sense on the server side.

        So, can you show us the code you are using, together with "good" and "bad" input data? Ideally, the code would be about 20 lines.

        http://search.cpan.org/dist/HTML-Parser/lib/HTML/Entities.pm

        decode_entities( $string, ... )

        This routine replaces HTML entities found in the $string with the corresponding Unicode character. Unrecognized entities are left alone.

        use HTML::Entities qw/decode_entities/; my $ok=decode_entities($orig);
        Im not sure this would work on "chunks", $orig has to be the entire string to decode at once for with chunks an entity could be split across them.