Re^2: Quick 'n dirty extraction of JSON from an HTML page

[davebaker]

Yes, it certainly does give me what I need. Thanks, John!

Some of the JavaScript seems to be using key/value specifications that aren't valid JSON because the keys aren't quoted strings, e.g.

var renderer = new US.Opportunity.OpportunityRenderViewModel({

            opportunity: opportunity,

            currentJobBoardId: "6162c253-9d81-da08-c252-d43d2fcb8345",

            isViewingInternal: false

        });

... so I changed the regular expression to be

m/\((\{".*?\})\)/gms

(throwing in a leading quotation mark, in order to find only JSON that has a quoted initial key).

I also played with the possibility that the HTML page would contain more than one block of JSON, and changed your code to be

my ( $json, $ref );
for ( $scrape =~ m/\((\{".*?\})\)/gms ) {
    $json = $1;
    $ref = decode_json $json;
    print Dumper $ref;
}

...so as to find and print for me each of multiple JSON blocks (not shown here). Love it!

[tobyink]

Consider using the original regexp, which doesn't require keys to be quoted, and parsing the JSON using Cpanel::JSON::XS and turning relaxed mode.

Javascript objects can of course still include values which cannot be encoded into JSON, for example:

var obj = {
  "some_key": Date.now(),
  "other_key": function () {
    console.log("Hello world");
  }
};

So if your Javascript objects contain things like this, you'll be out of luck. You might want to wrap your JSON decoding in try/catch or eval.