Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Can someone please write a *working* JSON module

by cnd (Acolyte)
on Oct 24, 2021 at 06:16 UTC ( #11137940=perlquestion: print w/replies, xml ) Need Help??

cnd has asked for the wisdom of the Perl Monks concerning the following question:

JSON::XS is not thread-safe (crashes mod_perl in production) and seems to have a memory leak (I see oob on apache after some time...)
All the other JSON's are stupidly slow.
JSON:XS (and all the others) are riddled with "croak" statements - making them unusable in production without wrapping them in "eval()".
eval makes JSON::XS even slower than JSON::PP

Philosophically - I strongly disagree with the concept of ever using "croak" on broken input - that's a fast-track way to derail production code... but that's just my opinion (based on a mere 37 years of programming every day).

The spec looks *really* simple: https://datatracker.ietf.org/doc/html/rfc8259 - with the exception of the unicode mess (which is the prime candidate for many of the croak deaths I observe above, and possibly the memory-leak.)

Is there any really-good C programmer here, who wants to engrave their name in history by writing a correct and stable JSON module, that can be safely used in scripts without any eval() needed ?

Suggestion: code some flags so that we users can specify the behaviour (in case some masochist actually wants to croak themselves), and other useful features (e.g. python interoperability including NaN, -inf, and other non-standard stuff that despite-not-being-standard everyone still wants/uses..., stripping of javascript-unsafe or broken unicode chars, pretty-printing, canonical output, ...)
  • Comment on Can someone please write a *working* JSON module

Replies are listed 'Best First'.
Re: Can someone please write a *working* JSON module
by karlgoethebier (Abbot) on Oct 24, 2021 at 08:27 UTC

    Did you try Cpanel::JSON::XS?

    From ibidem:

    «Cpanel::JSON::XS has proper ithreads support, unlike JSON::XS. If you encounter any bugs with thread support please report them.»
    «Is there any really-good C programmer here?»

    I guess some of these guys hang around here from time to time. I heard also through the grapevine that Perl is written in C. See also.

    «The Crux of the Biscuit is the Apostrophe»

      Cpanel::JSON::XS also fails under mod_perl unfortunately :-(
      Good thinking though.
        We use it at work in production under mod_perl without problems. Please provide more details.

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

        And JSON::Tiny also fails?

        «The Crux of the Biscuit is the Apostrophe»

Re: Can someone please write a *working* JSON module
by Fletch (Chancellor) on Oct 24, 2021 at 08:53 UTC
    eval makes JSON::XS even slower than JSON::PP

    Erm . . . Citation needed.

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

    A reply falls below the community's threshold of quality. You may see it by logging in.
    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Can someone please write a *working* JSON module
by talexb (Canon) on Oct 24, 2021 at 21:25 UTC
      JSON::XS is not thread-safe (crashes mod_perl in production) and seems to have a memory leak (I see oob on apache after some time...)

    Would it be possible to take mod_perl out of the equation, and instead have a daemon to do the JSON translation? Your mod_perl code could call the daemon and get back the object. The daemon could spawn a new kid to do the actual translation, which should obviate any possible memory leakage problems. I wrote something like that in 2007-08, and it worked really well.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

      Heh. OP is complaining about block eval slowing stuff down ...

        Did I miss the part where they patiently and carefully laid our their evidence .. proving this claim?

        Alex / talexb / Toronto

        Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

Re: Can someone please write a *working* JSON module
by eyepopslikeamosquito (Bishop) on Oct 26, 2021 at 05:44 UTC

    > Philosophically - I strongly disagree with the concept of ever using "croak" on broken input - that's a fast-track way to derail production code... but that's just my opinion (based on a mere 37 years of programming every day).

    Since you have no code to show us, let's discuss your problem "philosophically".

    First, as pointed out by a bona fide programming legend, programming every day for 37 years does not mean you are any good at it. The key is deliberative practice, not just doing it over and over again:

    Researchers (Bloom (1985), Bryan & Harter (1899), Hayes (1989), Simmon & Chase (1973)) have shown it takes about ten years to develop expertise in any of a wide variety of areas, including chess playing, music composition, telegraph operation, painting, piano playing, swimming, tennis, and research in neuropsychology and topology. The key is deliberative practice: not just doing it again and again, but challenging yourself with a task that is just beyond your current ability, trying it, analyzing your performance while and after doing it, and correcting any mistakes. Then repeat. And repeat again. There appear to be no real shortcuts: even Mozart, who was a musical prodigy at age 4, took 13 more years before he began to produce world-class music. In another genre, the Beatles seemed to burst onto the scene with a string of #1 hits and an appearance on the Ed Sullivan show in 1964. But they had been playing small clubs in Liverpool and Hamburg since 1957, and while they had mass appeal early on, their first great critical success, Sgt. Peppers, was released in 1967.

    Second, as noted at Two Different Languages: Two Similar Books (PBP and CCS), Perl expert Damian Conway and C++ experts Andrei Alexandrescu and Herb Sutter all agree that throwing exceptions is indeed a sound way to report errors:

    • PBP Chapter 13, "Throw exceptions instead of returning special values or setting flags"
    • CCS Guideline 72, "Prefer to use exceptions to report errors"

    Conway further illustrates this specific Perl best practice with an illustrative example (number 8) which uses block eval and Carp's croak.

    Admittedly, Exception handling in general is a difficult and controversial topic. Golang, for example, was initially released with exception handling explicitly omitted because the designers felt it obfuscated control flow - though the exception-like panic/recover mechanism was later added.

    See also: Reflections on Skills of the Skillful

      Conway further illustrates this specific Perl best practice with an illustrative example (number 8) which uses block eval and Carp's croak.

      I must say I despise croak and I prefer the return (of complex:hash/object) values in order to signify errors or success. I disagree with Conway's subjective assertion that Constantly checking return values for failure clutters your code with validation statements, often greatly decreasing its readability. (number 8) :

      • especially when the way to check the exception is the eval. Often nested (666)!
      • from looking at Try::Tiny's source code, I can see it is just an eval wrapper.
      • Carp's manual states that croak does not return objects (as in a "proper" exception) and when it is given an object it calls die or warn which I think looses the stacktrace. A big price to pay for getting a bit more information out of the errors.
      • Consequently, using strings instead of an exception object, is one-dimensional: has error or not. Unless one resorts to string comparisons to check what error it was which is the safest way back to BASICs.
      • eval is relatively cheap if it's used once especially when the block is expensive (as NERDVANA has benchmarked) but what happens when code is full of evals for simple file-open-and-read operations? (Conway's number 8).
      • Another aspect of using the eval/croak exception handling, is how the perl debugger handles eval. If it's transparent, fine but if it produces an eyesore then there is another point to consider (I can't say: I have never used the debugger).

      The bottomline for me, is that until Perl manages proper exception handling I return error integers or hashes and avoid recommending "exceptions" (as Conway does) which are not the real thing. What's the real thing? For me, it must include objects and avoid evals.

      bw, bliako

        I won't argue in favor or against exceptions, but your points contain some inaccuracies.
        • die and croak can deliver objects to $@ quite fine. If you croak with an object, that object is passed to die and you can evaluate that object in $@ as you like. It loses the stacktrace, but then, you don't get a stacktrace by returning error integers/objects as well.
        • The debugger has no problems with eval. You wouldn't know, of course, if you never used it, but then why raise the issue?
        • Also, to clarify something you wrote in Re^4: Can someone please write a *working* JSON module: eval BLOCK does not "spawn a new interpreter".
        If you don't like the keyword eval, then you can have try/catch as of Perl 5.34. There are also several CPAN modules offering that, and of course they're all eval wrappers, because that's just the mechanism Perl always had. The point of the modules isn't about using different words, but avoiding the pitfalls with localized $@ (The docs of Try::Tiny give some details).

        It doesn't have to be a dichotomy. Exceptions can be used when you need to fail the operation and jump in an overall exception handler multiple functions above in the call stack. Success/error return values (preferably when they are of different types and the compiler can make sure you're not using the success type in the error branch, a.k.a. the Optional<Success, Failure> idiom) can be used when you can check for and handle the error right away.

        People who use exceptions won't argue in favour of code like the following:

        eval { step_1(); 1; } or handle_error_1(); eval { step_2(); 1; } or handle_error_2(); eval { step_3(); 1; } or handle_error_3(); eval { step_4(); 1; } or handle_error_4();
        That's just more typing for the same result as
        step_1() or handle_error_1(); step_2() or handle_error_2(); step_3() or handle_error_3(); step_4() or handle_error_4();

        On the other hand, if an error means the whole operation should be aborted and the individual steps signal that by raising an exception, we can write it as

        try { step_1(); step_2(); step_3(); step_4(); } catch { handle_overall_error(); };
        and still be sure that when the error happens, (1) any resources will be cleaned up safely by desctructors and (2) the following steps won't be taken.

        By the way, why do you dislike eval? I mean, I don't like eval EXPR either, but what's wrong with eval BLOCK?

        I get all my json tools from the software that I've replicated of bliako's. Part of "tribal knowledge" is to know that M. Lehmann has left the fold. I wish there were a way to have perl know which modules are un-maintained, but one man's bug is another's feature.

        I can't say: I have never used the debugger

        How is it possible that you have never used the debugger?

Re: Can someone please write a *working* JSON module
by karlgoethebier (Abbot) on Oct 26, 2021 at 14:56 UTC

    One more thing: talexb suggested to get your JSON from somewhere out there as far as i understood.

    This idea seems to be fairly elegant. As you don't need to throw away your Apache/mod_perl stuff etc.

    What about this little sketch?

    # app.psgi use strict; use warnings; use Plack::Request; use Plack::App::URLMap; use JSON::Tiny qw(encode_json); my $slash = sub { my $env = shift; my $json = encode_json {nose => 'cuke'}; my $status = 200; my $request = Plack::Request->new($env); my $headers = [ 'Content-Type' => 'application/json', 'Content-Length' => length $json, ]; my $body = [ $json ]; [ $status, $headers, $body ]; }; my $urlmap = Plack::App::URLMap->new; $urlmap->mount( "/" => $slash ); my $app = $urlmap->to_app; __END__

    Then: starman --listen /tmp/starman.sock.

    And then try: echo -e "GET / HTTP/1.0\r\n" | nc -U /tmp/starman.sock.

    This should yield:

    HTTP/1.0 200 OK Content-Type: application/json Content-Length: 15 Date: Tue, 26 Oct 2021 14:25:29 GMT Connection: close {"nose":"cuke"}

    See also Plack::Request and Plack::App::URLMap.

    The other links you have already.

    Probably Try::Tiny might be helpful for handling malformed JSON.

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

Re: Can someone please write a *working* JSON module (Send money)
by Anonymous Monk on Oct 24, 2021 at 08:17 UTC
    Send money, that should do it
    A reply falls below the community's threshold of quality. You may see it by logging in.
A reply falls below the community's threshold of quality. You may see it by logging in.
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11137940]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (2)
As of 2021-12-06 06:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    R or B?



    Results (31 votes). Check out past polls.

    Notices?