Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re^2: Can someone please write a *working* JSON module

by cnd (Acolyte)
on Oct 24, 2021 at 09:57 UTC ( [id://11137953] : note . print w/replies, xml ) Need Help??

in reply to Re: Can someone please write a *working* JSON module
in thread Can someone please write a *working* JSON module

This node falls below the community's threshold of quality. You may see it by logging in.

Replies are listed 'Best First'.
Re^3: Can someone please write a *working* JSON module
by NERDVANA (Deacon) on Oct 24, 2021 at 14:14 UTC
    eval makes JSON::XS even slower than JSON::PP
    seriously - it's 10 to 100 times slower to use eval() rather than a native decode_json() in a mod_perl production webserver on non-trivial data

    Your experience seems to differ from everyone else's experience. This is probably why all the JSON module authors decided it was fine to use croak. Please show some kind of benchmark if you want to convince anyone.

    I just ran a benchmark (command line, on a laptop, perl 5.34) and see that for extremely small json decodes of 1 object with one key, the eval is as high as 20% overhead on the fastest XS modules (but still 2.5+ million per second). It's a ~0% difference for JSON::PP (mine actually ran the eval faster than without, but that probably indicates the noise level in the benchmark as a whole). This assumes valid json - of course for invalid JSON, triggering exceptions would be slower than a parser that returns undef, but if you're worried about denial of service, I think you'll run out of bandwidth before running out of processor to catch the exceptions.

    For extremely large JSON texts, the eval overhead is lost in the noise. (essentially no difference)

    $ perl 
    long_json: 2468099
                (warning: too few iterations for a reliable count)
                        Rate pp_small pp_small_eval xs_small_eval cxs_small_eval xs_small cxs_small
    pp_small         77101/s       --           -2%          -96%           -96%     -97%      -97%
    pp_small_eval    78309/s       2%            --          -96%           -96%     -97%      -97%
    xs_small_eval  2083333/s    2602%         2560%            --            -4%     -15%      -17%
    cxs_small_eval 2173913/s    2720%         2676%            4%             --     -11%      -13%
    xs_small       2439024/s    3063%         3015%           17%            12%       --       -2%
    cxs_small      2500000/s    3142%         3092%           20%            15%       3%        --
                     Rate pp_long_eval pp_long cxs_long_eval cxs_long xs_long_eval xs_long
    pp_long_eval  0.596/s           --     -0%          -98%     -98%         -98%    -98%
    pp_long       0.597/s           0%      --          -98%     -98%         -98%    -98%
    cxs_long_eval  24.4/s        3993%   3983%            --      -1%         -18%    -18%
    cxs_long       24.5/s        4018%   4008%            1%       --         -18%    -18%
    xs_long_eval   29.9/s        4909%   4897%           22%      22%           --      0%
    xs_long        29.9/s        4909%   4897%           22%      22%           0%      --

    Here's my benchmark: feel free to adjust it in such a way that it shows your 10x - 100x slowdown, and post for us to look at.

    use v5.20; use warnings; use Benchmark qw( :hireswallclock cmpthese ); use JSON::PP (); use JSON::XS (); use Cpanel::JSON::XS (); my $json_pp= JSON::PP->new; my $json_xs= JSON::XS->new; my $json_cxs= Cpanel::JSON::XS->new; my $small_json_string= q|{"test":1}|; sub random_data { return { map { rand() => $_[0]? random_data($_[0]-1) : rand() } 1..4 +0 } } my $long_json_string= $json_xs->encode(random_data(2)); say "long_json: ".length $long_json_string; cmpthese(1000000, { pp_small_eval => sub { eval { !$json_pp->decode($small_json_string) + } }, xs_small_eval => sub { eval { !$json_xs->decode($small_json_string) + } }, cxs_small_eval => sub { eval { !$json_cxs->decode($small_json_string +) } }, pp_small => sub { !$json_pp->decode($small_json_string) }, xs_small => sub { !$json_xs->decode($small_json_string) }, cxs_small => sub { !$json_cxs->decode($small_json_string) }, }); cmpthese(40, { pp_long_eval => sub { eval { !$json_pp->decode($long_json_string) } + }, xs_long_eval => sub { eval { !$json_xs->decode($long_json_string) } + }, cxs_long_eval => sub { eval { !$json_cxs->decode($long_json_string) +} }, pp_long => sub { !$json_pp->decode($long_json_string) }, xs_long => sub { !$json_xs->decode($long_json_string) }, cxs_long => sub { !$json_cxs->decode($long_json_string) }, });

    Edit: Removed my comment about threaded/nonthreaded perl, since it was a strawman argument.

      Try 'strings' instead if sub{}s for less noisy benchmark
Re^3: Can someone please write a *working* JSON module
by Fletch (Bishop) on Oct 24, 2021 at 17:53 UTC

    I'm the one pointing out that you're making a wild assertion (paraphrasing, "BLOCK eval causes an order of magnitude(s) slowdown") which runs counter to consensus experience without giving any concrete evidence or providing any sample code which evidences the problem in question by myself providing a counter example, yes.

    Show us code which evidences this purported order of magnitude(s) slowdown and I bet you'll get a reply from someone either pointing out what you're doing wrong, or if it's actually something you'll have eyes look at it from someone quite likely to be able to fix the problem in perl itself. BLOCK eval does add a small overhead because of the extra steps involved, but not that much (it's a couple of extra opcodes in the tree that don't really do much of anything if no exception occurs).

    Or keep whinging about downvotes because you posted something detail- and code-free that (observationally) most people reading think is bunk. Tomayto, tomahto.

    Edit: Tested the same benchmarks with a 700+k character "NON-TRIVIAL" chunk of both valid and invalid JSON (the later made by substituting in random garbage somewhere in the former). Aside from the processing rate understandably going down with the larger amount to parse the results look similar showing that the presence or absence of eval has no measurable effect (hundreds/sec for the non-PP versions versus 30k/sec; PP gets on the order of 1/sec eval or no).

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

      > a wild assertion (paraphrasing, "BLOCK eval causes an order of magnitude(s) slowdown") which runs counter to consensus experience

      My guess:

      The OP was highlighting a modperl environment, which is notorious for global side effects IIRC.

      So if one of the other modperl clients is manipulating the error mechanism, (like setting $SIG{WARN} or CGI::Carp or something homebrewn), this could lead to some slowdown.(Disclaimer: I never tried this, especially never tried modperl)

      It's hard to tell why this is triggered by eval {BLOCK} , but this could be connected to ignoring that special variable which indicates the handler was fired inside an eval. (See $^S in perlvar )

      All of this is of course not related to JSON ... and switching to another technology like fastcgi or PSGI should fix it.

      Tho I wouldn't be surprised if the OP fiddled with error handlers by himself without taking care of $^S

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re^3: Can someone please write a *working* JSON module
by dave_the_m (Monsignor) on Oct 24, 2021 at 10:31 UTC
    Are you talking about adding eval $string or eval { block }?


Re^3: Can someone please write a *working* JSON module
by ikegami (Patriarch) on Oct 25, 2021 at 15:32 UTC

    I did benchmark it properly.

    An empty claim when you could show your benchmark.