Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

How can I convert this raw data to a hash?

by SergioQ (Beadle)
on Dec 21, 2020 at 19:01 UTC ( #11125553=perlquestion: print w/replies, xml ) Need Help??

SergioQ has asked for the wisdom of the Perl Monks concerning the following question:

I'm ashamed to admit how badly this has me stumped, especially since I did it once before and can't find the file. I only learned enough of Perl to do what I needed to do last time. This wasn't part of it.

I have this data which I can store in $string or a file, preferably the former. But I just can't get a handle on converting it into one hash, where the 2 keys and values are obvious.

Also, would love to know what data like this is called, and what the procedure to loop through would be called

Yes, I can just write something using regex to create the hash, but I really would love to learn the proper way to do it.

As always, any help would be appreciated.

{ "genres": [ { "id": 28, "name": "Action" }, { "id": 12, "name": "Adventure" }, { "id": 16, "name": "Animation" }, { "id": 35, "name": "Comedy" }, { "id": 80, "name": "Crime" }, { "id": 99, "name": "Documentary" }, { "id": 18, "name": "Drama" }, { "id": 10751, "name": "Family" }, { "id": 14, "name": "Fantasy" }, { "id": 36, "name": "History" }, { "id": 27, "name": "Horror" }, { "id": 10402, "name": "Music" }, { "id": 9648, "name": "Mystery" }, { "id": 10749, "name": "Romance" }, { "id": 878, "name": "Science Fiction" }, { "id": 10770, "name": "TV Movie" }, { "id": 53, "name": "Thriller" }, { "id": 10752, "name": "War" }, { "id": 37, "name": "Western" } ] }

Replies are listed 'Best First'.
Re: How can I convert this raw data to a hash?
by davido (Cardinal) on Dec 21, 2020 at 19:22 UTC

    The data you have is JSON. And a module such as JSON has decode_json to convert it to a Perl data structure. Alternatively, if you don't want to install JSON, Perl ships with JSON::PP, which is a drop-in replacement (or fall-back).

    #!/usr/bin/env perl use strict; use warnings; use JSON qw(decode_json); my $json = do { local $/ = undef; <DATA>; }; my $data = decode_json($json); foreach my $element (@{$data->{'genres'}}) { print "id: $element->{'id'}, name: $element->{'name'}\n"; } __DATA__ { "genres": [ { "id": 28, "name": "Action" }, { "id": 12, "name": "Adventure" }, { "id": 16, "name": "Animation" }, { "id": 35, "name": "Comedy" }, { "id": 80, "name": "Crime" }, { "id": 99, "name": "Documentary" }, { "id": 18, "name": "Drama" }, { "id": 10751, "name": "Family" }, { "id": 14, "name": "Fantasy" }, { "id": 36, "name": "History" }, { "id": 27, "name": "Horror" }, { "id": 10402, "name": "Music" }, { "id": 9648, "name": "Mystery" }, { "id": 10749, "name": "Romance" }, { "id": 878, "name": "Science Fiction" }, { "id": 10770, "name": "TV Movie" }, { "id": 53, "name": "Thriller" }, { "id": 10752, "name": "War" }, { "id": 37, "name": "Western" } ] }

    This produces:

    id: 28, name: Action id: 12, name: Adventure id: 16, name: Animation id: 35, name: Comedy id: 80, name: Crime id: 99, name: Documentary id: 18, name: Drama id: 10751, name: Family id: 14, name: Fantasy id: 36, name: History id: 27, name: Horror id: 10402, name: Music id: 9648, name: Mystery id: 10749, name: Romance id: 878, name: Science Fiction id: 10770, name: TV Movie id: 53, name: Thriller id: 10752, name: War id: 37, name: Western

    A foreach loop is useful here, as described in perlintro or perlsyn. Dealing with complex data structures is covered in Intermediate Perl (OReilly), in perlreftut, perlref, perldsc, and perllol.


    Dave

Re: How can I convert this raw data to a hash?
by haukex (Bishop) on Dec 21, 2020 at 19:14 UTC

    That looks very much like JSON, so you'd need to use a JSON module like Cpanel::JSON::XS, or JSON::PP has been in the core since Perl 5.14.

    If you need help in further processing this data and looping over it, then please explain what you'd like to do (with code) and what your expected output is. See also perldsc, perlreftut, and perlref.

Re: How can I convert this raw data to a hash?
by kcott (Bishop) on Dec 21, 2020 at 19:16 UTC

    G'day SergioQ,

    Your "raw data" is JSON. Take a look at JSON. If you also have JSON::XS installed it will run faster.

    What you want to do should be fairly straightforward from the example code in the SYNOPSIS.

    — Ken

Re: How can I convert this raw data to a hash?
by jdporter (Canon) on Dec 21, 2020 at 19:29 UTC
    use JSON; use Data::Dumper; use strict; use warnings; my $json = JSON->new->allow_nonref; my $raw = do { undef $/; <DATA> }; my $ds = $json->decode( $raw ); if(0) { # this replaces the value of 'genres' with a hash more like what you w +ant: for my $key ( keys %$ds ) # e.g. 'genres' { my %h; for my $r ( @{ $ds->{$key} } ) { $h{$r->{'id'}} = $r->{'name'}; } $ds->{$key} = \%h; # replace it! } print Dumper $ds; } # or if you really want to 'invert' the hash, so that 'id's are at the + top: my %data; for my $key ( keys %$ds ) # e.g. 'genres' { my %h; for my $r ( @{ $ds->{$key} } ) { $data{$r->{'id'}}{$key} = $r->{'name'}; } } print Dumper \%data; __DATA__ { "genres": [ { "id": 28, "name": "Action" }, { "id": 12, "name": "Adventure" }, { "id": 16, "name": "Animation" }, { "id": 35, "name": "Comedy" }, { "id": 80, "name": "Crime" }, { "id": 99, "name": "Documentary" }, { "id": 18, "name": "Drama" }, { "id": 10751, "name": "Family" }, { "id": 14, "name": "Fantasy" }, { "id": 36, "name": "History" }, { "id": 27, "name": "Horror" }, { "id": 10402, "name": "Music" }, { "id": 9648, "name": "Mystery" }, { "id": 10749, "name": "Romance" }, { "id": 878, "name": "Science Fiction" }, { "id": 10770, "name": "TV Movie" }, { "id": 53, "name": "Thriller" }, { "id": 10752, "name": "War" }, { "id": 37, "name": "Western" } ] }
Re: How can I convert this raw data to a hash?
by NetWallah (Canon) on Dec 21, 2020 at 21:56 UTC
    If the simplicity of this JSON string is guaranteed, you can try the following very crude (Module free) JSON to perl converter,
    use strict; use warnings; (my $datastring=do{local $/=undef; <DATA>}) =~s/:/=>/g; my $data = eval $datastring; my %by_name = map { $_->{name} => $_->{id} } @{$data->{'genres'}}; print "The ID for Science Fiction is:", $by_name{"Science Fiction"} , +"\n"; __DATA__ <<<YOUR JSON Here...>
    Output:
    The ID for Science Fiction is:878
    NOT RECOMMENDED for production code.

                    "Imaginary friends are a sign of a mental disorder if they cause distress, including antisocial behavior. Religion frequently meets that description"

      NOT RECOMMENDED

      I feel like this should be at the top of the node.

      But if you're going down that route, then this classic node is worth pointing out: JSON parser as a single Perl Regex, which works on the data in the root node when you apply fanasy's fix from here. Update: To be clear, I'm not seriously suggesting this as a solution here :-)

      If JSON::PP is in core, why avoid it and suggest code with a caveat that it shouldn't be used due to badness?

      Update Correct JSON to JSON::PP - thanks Haarg

      Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond
        Some potential excuses:

        • Older perl, missing core JSON
        • Too scared/lazy/ignorant to use JSON
        • Because you can

                        "Imaginary friends are a sign of a mental disorder if they cause distress, including antisocial behavior. Religion frequently meets that description"

Re: How can I convert this raw data to a hash?
by perlfan (Vicar) on Dec 23, 2020 at 09:30 UTC
    If you replace : with => (or even a comma), then you'd have PON - Perl Object Notation, also known as actual Perl code. So having that you could then do something silly like prepend our $hash_ref = then string eval or write it to a file then require it:

    dat.pl

    our $hash_ref = { "genres" => [ { "id" => 28, "name" => "Action" }, { "id" => 12, "name" => "Adventure" }, { "id" => 16, "name" => "Animation" }, { "id" => 35, "name" => "Comedy" }, { "id" => 80, "name" => "Crime" }, { "id" => 99, "name" => "Documentary" }, { "id" => 18, "name" => "Drama" }, { "id" => 10751, "name" => "Family" }, { "id" => 14, "name" => "Fantasy" }, { "id" => 36, "name" => "History" }, { "id" => 27, "name" => "Horror" }, { "id" => 10402, "name" => "Music" }, { "id" => 9648, "name" => "Mystery" }, { "id" => 10749, "name" => "Romance" }, { "id" => 878, "name" => "Science Fiction" }, { "id" => 10770, "name" => "TV Movie" }, { "id" => 53, "name" => "Thriller" }, { "id" => 10752, "name" => "War" }, { "id" => 37, "name" => "Western" } ] }

    show_dat.pl

    use Data::Dumper; require 'dat.pl'; print Data::Dumper::Dumper($hash_ref);
        To be fair, accepting rando Storable input files has thia caveat. My only point that it was very close to defining a hash ref.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11125553]
Approved by haukex
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (3)
As of 2021-11-28 15:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?