Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Keeping Order with YAML::XS

by walkingthecow (Friar)
on Jul 29, 2013 at 03:31 UTC ( #1046761=perlquestion: print w/ replies, xml ) Need Help??
walkingthecow has asked for the wisdom of the Perl Monks concerning the following question:

I have a data structure that looks something like this:
$VAR1 = { 'path' => '/export/home/frank' 'options' => [ { 'value' => '1001', 'param' => 'i' }, { 'param' => 'f' }, { 'value' => 'eyes', 'param' => 'x' }, {}, { 'value' => 'toes', 'param' => 'b' }, { 'param' => 'p' }, ], 'arguments' => [ { 'value' => 'test' } ], };
Written to a YAML file using YAML::XS DumpFile I get something like this:
arguments: - value: test path: /export/home/frank options: - param: i value: '1001' - param f - param: x value: eyes - flag: b value: toes - param: p
Rather, I'd like for it to be something like this:
path: /export/home/frank options: - param: i value: '1001' - param f - param: x value: eyes - flag: b value: toes - param: p arguments: - value: test
The problem is, I'm not sure what the data structure would look like to create this in YAML. Anyone have any ideas how to do this? I want things to be ordered. First the path, then the options (in order), then the arguments (also in order). But, with the way hashes work, and with YAML's use of dashes for ordering, I cannot seem to accomplish this with DumpFile.

Comment on Keeping Order with YAML::XS
Select or Download Code
Re: Keeping Order with YAML::XS
by Loops (Hermit) on Jul 29, 2013 at 03:39 UTC

    Since you know what you're aiming at, just use the yaml as a template. Load it into Perl and then dump the resulting data structure:

    use YAML::XS qw(LoadFile); use Data::Dump qw(pp); pp LoadFile(\*DATA); __DATA__ path: /export/home/frank options: - param: i value: '1001' - param f - param: x value: eyes - flag: b value: toes - param: p arguments: - value: test
    Produces:
    { arguments => [{ value => "test" }], options => [ { param => "i", value => 1001 }, "param f", { param => "x", value => "eyes" }, { flag => "b", value => "toes" }, { param => "p" }, ], path => "/export/home/frank", }

    Which after looking back at your original post, is essentially the same as your input (which has a missing comma). So its not a matter of finding a data structure that produces your desired YAML. Maybe it's better to have asked why you want a specific format to your YAML? Isn't any format that properly encodes your data, okay?

      I can load a file without issues, and it creates what I am expecting. The issue comes when trying to dump it back. Here's an example:

      in.yaml
      path: /export/home/frank options: - param: i value: 1001 - param: f - param: x value: eyes arguments: - value: test
      Here's the Perl code:

      Perl Code
      #!/usr/bin/env perl use strict; use warnings; use YAML::XS qw(LoadFile DumpFile); use Data::Dumper; my $data = LoadFile('test.yaml'); print Dumper($data); DumpFile('out.yaml', $data);
      And here's the output yaml (out.yaml) that it creates:

      out.yaml
      --- arguments: - value: test options: - param: i value: 1001 - param: f - param: x value: eyes path: /export/home/frank
      Notice that in the input file the syntax was in such a way that it was an ordered sequence (from what I am understanding of YAML anyway). Yet, in the output it is no longer ordered at all. I am trying to figure out from a Perl data structure how to make an ordered YAML file like what is the input.
        Why do you say that the input is creating what you're expecting? The in.yaml you give above produces:
        { arguments => [{ value => "test" }], options => [ { param => "i", value => 1001 }, { param => "f" }, { param => "x", value => "eyes" }, ], path => "/export/home/frank", }

        IE. the top level items (arguments,options,path) are in a hash, which is an unordered set in Perl. You've lost your ordering on input, not on output. Unless you're talking about the options? In which case both formats are equivalent, at least to the YAML::XS loader which creates the same data structures in both cases.

        In what external context does the yaml produced by YAML::XS cause a problem? Because both formats generate the same data structures in Perl and therefore should be interchangeable without issue.

Re: Keeping Order with YAML::XS (impossible, try YAML::Tiny, because no yamltidy)
by Anonymous Monk on Jul 29, 2013 at 04:01 UTC

    FWWIAFAIR, YAML::Tiny produces that type of output, I thought maybe if married with Tie::IxHash it will preserve order? But looks like not

    use YAML::Tiny qw/ Dump /; use Tie::IxHash; tie my(%hash), 'Tie::IxHash'; %hash = ( 'path' => '/export/home/frank', 'options' => [ { 'value' => '1001', 'param' => 'i' }, { 'param' => 'f' }, { 'value' => 'eyes', 'param' => 'x' }, {}, { 'value' => 'toes', 'param' => 'b' }, { 'param' => 'p' }, ], 'arguments' => [ { 'value' => 'test' } ], ); print Dump(\%hash); __END__ --- arguments: - value: test options: - param: i value: 1001 - param: f - param: x value: eyes - {} - param: b value: toes - param: p path: '/export/home/frank'

    This is what I don't like about YAML, no yamltidy

Re: Keeping Order with YAML::XS
by tobyink (Abbot) on Jul 29, 2013 at 14:21 UTC

    If you're willing to switch from YAML to JSON, you could take a look at my JSON::MultiValueOrdered module which preserves the order of hash keys (and also preserves repeated keys).

    package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name
Re: Keeping Order with YAML::XS
by Loops (Hermit) on Jul 30, 2013 at 05:01 UTC
    Hi wtc,

    Have been debating whether to post this or not. Was playing with this code while we were discussing your issue when a couple of people suggested using Tie::IxHash, but then found its ordering wasn't recognized by YAML::XS. Here it is for whatever it may be worth.

    In an earlier post a quote from the YAML Spec shows the proper format for ordered mappings. Below is code to recursively convert a data structure containing IxHash's to one needed for output by YAML::XS. There is also a function to recursively convert a structure loaded from YAML::XS into one containing properly ordered IxHash's.

    The code is presented as is, i'm sure there is a more elegant and less bug ridden version trying to get out -- would be happy to receive criticism. Mostly I hope it helps with your problem if you're still looking for a solution

    use strict; use warnings; use Tie::IxHash; use Data::Dumper; use Data::Rmap qw(:types rmap_to); sub ixhash_to_yamlxs { my $item = \+shift; my $get_yaml_struct = sub { my $raw = shift; my $ix = tied(%$raw); return $raw unless defined $ix and $ix->isa('Tie::IxHash'); return [ map { { $_ => $raw->{$_} } } $ix->Keys ]; }; rmap_to { $_ = $get_yaml_struct->($_) } HASH, $item; return $item; } sub yamlxs_to_ixhash { my $item = shift; my $make_ixhash_from_array = sub { my $raw = shift; tie my(%hash), 'Tie::IxHash'; eval { no warnings; for (@$raw) { return $raw if scalar keys $_ ne 1; my ($key) = keys $_; $hash{$key} = $_->{$key}; } }; return $raw if $@; return \%hash; }; rmap_to { $_ = $make_ixhash_from_array->($_) } ARRAY, \$item; return $item; } tie my(%embed), 'Tie::IxHash', (j => 5, k => 6, l => 7); tie my(%hash), 'Tie::IxHash', (a => 1, c => { z => 26 }, b => 2, d => +[ 'j', 'k', 'l'], e => \%embed); my $out = ixhash_to_yamlxs \%hash; print Dumper $out; my $in = yamlxs_to_ixhash $out; print Dumper $in;
    # IxHash converted for output by YAML::XS # Notice the embeded IxHash was handled # and also an embeded native hash (key 'c') # that is not converted to ordered structure \[ { 'a' => 1 }, { 'c' => { 'z' => 26 } }, { 'b' => 2 }, { 'd' => [ 'j', 'k', 'l' ] }, { 'e' => [ { 'j' => 5 }, { 'k' => 6 }, { 'l' => 7 } ] } ];
    # ordered Tie::IxHash recovered from YAML::XS format \{ 'a' => 1, 'c' => { 'z' => 26 }, 'b' => 2, 'd' => [ 'j', 'k', 'l' ], 'e' => { 'j' => 5, 'k' => 6, 'l' => 7 } };

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1046761]
Approved by Loops
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2014-10-02 08:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (51 votes), past polls