Post query to Metacpan API

[Anonymous Monk]

How does one query the metacpan api from perl? The documentation is not clear and the examples are for Lynx, like it's 1999. Also what format is the data metacpan sends back? How is that stuff turned into a perl data structure? Thank you!

#!/usr/bin/perl
# THIS SCRIPT DOES *NOT* WORK
# Get the latest version numbers of your favorite modules
# https://github.com/metacpan/metacpan-api/blob/master/docs/API-docs.m
+d

use strict;
use warnings;
use diagnostics;

use Data::Dumper;
use HTTP::Tiny;

my $url = 'https://fastapi.metacpan.org/v1/release/_search';

# WHAT IS THIS?
my $query = q({
    "query" : { "terms" : { "distribution" : [
        "Mojolicious",
        "MetaCPAN-API",
        "DBIx-Class"
    ] } },
    "filter" : { "term" : { "status" : "latest" } },
    "fields" : [ "distribution", "version" ],
    "size"   : 3
});

my $http = HTTP::Tiny->new;
my $res  = $http->post_form($url,$http->www_form_urlencode($query));

print Dumper($res);

[stevieb]

Use MetaCPAN::Client. I admit that it can be daunting at first as there's a lot to learn, but it works reliably.

Here's an example usage where I dig up all of my latest releases I've published to the CPAN, and their respective current versions:

use warnings;
use strict;

use MetaCPAN::Client;

my $mc = MetaCPAN::Client->new;
 
my $query = {
    all => [
        { author => 'STEVEB' },
        { status => 'latest' },
    ],
};

my $limit = { 
    '_source' => [
        qw(distribution version)
    ] 
};

my $releases = $mc->release($query, $limit);

while (my $rel = $releases->next){
    my $dist = $rel->distribution;
    $dist =~ s/-/::/g;

    printf("%s is at version %s\n", $dist, $rel->version);
}

Output:

RPi::DAC::MCP4922 is at version 2.3605
RPi::ADC::MCP3008 is at version 2.3604
RPi::BMP180 is at version 2.3604
RPi::Serial is at version 2.3603
WiringPi::API is at version 2.3616
App::RPi::EnvUI is at version 0.30
File::Edit::Portable is at version 1.24
Github::Backup is at version 1.02
RPi::ADC::ADS is at version 1.02
Hook::Output::Tiny is at version 0.05
Logging::Simple is at version 1.04
Test::BrewBuild::Plugin::TestAgainst is at version 0.06
RPi::WiringPi is at version 2.3628
RPi::SPI is at version 2.3608
RPi::StepperMotor is at version 2.3604
RPi::LCD is at version 2.3603
Test::BrewBuild::Plugin::Author is at version 0.03
Async::Event::Interval is at version 1.00
Devel::Trace::Subs is at version 0.22
Bit::Manip is at version 1.04
Bit::Manip::PP is at version 1.07
WWW::ProxyChecker is at version 1.005
Module::CheckDep::Version is at version 0.09
RPi::RTC::DS3231 is at version 0.01
RPi::I2C is at version 2.3606
Mock::Sub is at version 1.09
RPi::HCSR04 is at version 2.3604
FreeRADIUS::Database is at version 0.06
RPi::Pin is at version 2.3606
RPi::DigiPot::MCP4XXXX is at version 2.3604
Devel::Examine::Subs is at version 1.69
RPi::GPIOExpander::MCP23017 is at version 1.00
Wrap::Sub is at version 0.06
Geo::Compass::Variation is at version 1.00
GPSD::Parse is at version 1.02
RPi::Const is at version 1.04
RPi::DHT11 is at version 1.04
RPi::WiringPi::Constant is at version 1.01
App::CopyrightImage is at version 1.01
CGI::Application::Plugin::PageBuilder is at version 1.01
Test::Module::CheckDep::Version is at version 0.03
WWW::FreeProxyListsCom is at version 1.006
Plugin::Simple is at version 1.01
Test::BrewBuild is at version 2.20

The $query is what we're wanting to fetch (the 'latest' releases by me ('STEVEB') (because we're using the release() method), and the $limit limits the amount of data returned (we're only interested in the distribution name and its respective version number).

[Anonymous Monk]

That's cool but too complicated I just need to get version numbers back from module names like:

#!/usr/bin/perl
# THIS SCRIPT DOES WORK
# (if the module exists and has a normal version number)
# Get the latest version numbers of your favorite modules
# Usage: $0 Module::Name Other::Module Etc

use strict;
use warnings;
use HTTP::Tiny;

my @arg = @ARGV ? @ARGV : ('MetaCPAN::Client');

for (@arg) {
  print "$_\t",
  HTTP::Tiny->new->get("https://metacpan.org/pod/$_")->{content} =~
  /<span itemprop="softwareVersion">([^<]+)<\/span>/s, "\n"
}

[haukex]

That's cool but too complicated

The original question was "How does one query the metacpan api from perl?", to which the IMO best answer is indeed MetaCPAN::Client. Why reject it so quickly? I'd recommend having a look at the documentation. You'll see it's actually pretty easy to use.

Here's what fetching the version of a distro looks like without the module:

use warnings;
use strict;
use HTTP::Tiny;
use URI;
use Cpanel::JSON::XS qw/decode_json/;

my $http = HTTP::Tiny->new;

my @modules = @ARGV ? @ARGV : ('MetaCPAN::Client');

for my $mod (@modules) {
        my $uri = URI->new('http://fastapi.metacpan.org/v1/module');
        $uri->path_segments( $uri->path_segments, $mod );
        my $resp = $http->get($uri);
        die "$uri: $resp->{status} $resp->{reason}\n" unless $resp->{s
+uccess};
        my $api = decode_json($resp->{content});
        print "$mod: $api->{version}\n";
}

Update: In comparison, stevieb showed just how easy using the module makes it.

[stevieb]

I don't believe that this is very difficult:

use warnings;
use strict;

use MetaCPAN::Client;

my $mc = MetaCPAN::Client->new;

my @modules = qw(
    Test::BrewBuild
    RPi::WiringPi
    Devel::Examine::Subs
);

for (@modules){
    printf "%s: %s\n", $_, $mc->module($_)->version;
}

Output:

Test::BrewBuild: 2.20
RPi::WiringPi: 2.3628
Devel::Examine::Subs: 1.69

Here's the code broken up a bit to possibly better describe what's actually happening. The output is the same:

use warnings;
use strict;

use MetaCPAN::Client;

my $mc = MetaCPAN::Client->new;

my @modules = qw(
    Test::BrewBuild
    RPi::WiringPi
    Devel::Examine::Subs
);

for my $module_name (@modules){
    my $module_object = $mc->module($module_name);
    my $module_version = $module_object->version;

    print "$module_name: $module_version\n";
}

As I said, it may seem daunting at first, but the code will be more reliable going forward. Just takes a bit of time to read documentation, and do some basic testing :)

[Anonymous Monk]

Thank you both (haukex and stevieb) for showing different ways to do it. This is a good example of the benefits of using a module with a proper api. Without the module it needs a url, the decoder, and hash access, all of which could change; while the module neatly hides all the brittle parts behind a simple interface.

#!/usr/bin/perl
# Get the latest version numbers of your favorite modules
# (If they exist and have a normal version number...)
# Usage: $0 Module::Name Other::Module Etc

use strict; use warnings;
use HTTP::Tiny;
use JSON::PP 'decode_json';
use MetaCPAN::Client;

my @ARGS = @ARGV ? @ARGV : ('MetaCPAN::Client','MCE','Plack');

my $http = HTTP::Tiny->new;

for (@ARGS) {
  print "$_\t", decode_json($http->get(
  "http://fastapi.metacpan.org/v1/module/$_"
  )->{content})->{version},"\n"
}

my $cpan = MetaCPAN::Client->new;

for (@ARGS){
    printf "%s: %s\n", $_, $cpan->module($_)->version;
}

[stevieb]

I already answered this, but must have been too quick and possibly didn't hit the Submit button (very long day at work).

You're very welcome.

Explicit kudos to haukex who, as many Monks do, go unappreciated. I am glad that we were able to give you solutions that got you on your way.

-stevieb

[Haarg]

The direct problem with your original code is that you are trying to post the data incorrectly. The query should be posted directly as the body content of a POST, not as a query parameter. This can be done with the post method using the content option.

my $http = HTTP::Tiny->new;
my $res  = $http->post($url, { content => $query });