Finding what modules use and what uses the modules

[Lady_Aleena]

As some of you know, I'm doing an audit of my code. Part of that audit is finding what modules my modules use and what modules I have written are used by other modules I have written. If I ever get things nice enough in some of my modules, I might one day put a few up on CPAN. However, I need to know what modules my modules use so I can include them in the install if the user does not already have. Like, if I use Lingua::EN::Inflect in a module (or group of modules), it would need to be installed if the user does not have it already. At least I think so.

I came up with the following code to find the information but there might be a better way.

Update: I put the updated code I used here: Re: Finding what modules use and what uses the modules. Your options looked interesting but did not differentiate things the way I wanted.

#!/usr/bin/perl
use strict;
use warnings;

use File::Find;

my @files;
sub wanted {
  my $file = $_ =~/.pm$/ ? $File::Find::name : undef;
  push @files, $file if $file;
}

my @directories = ('/home/me/Documents/');
find(\&wanted,  @directories);

my $module_directory = $directories[0].'www/files/lib/';

my $modules;
for my $file (@files) {
  open my $fh, '<', $file or die "$file: $!";
  my $file_convert = $file;
     $file_convert =~ s/$module_directory(.+)\.pm/$1/;
     $file_convert =~ s/\//::/g;

  while (<$fh>) {
    chomp;
    if ($_ =~ s/^use ((:|\w)+)(.+)/$1/) {
      push @{$modules->{$file_convert}{'uses'}}, $1;    # what modules
+ the module uses
      push @{$modules->{$1}{'used by'}}, $file_convert; # what modules
+ use this module
    }
  }
}

my $in = shift;
use Data::Dumper;
print Dumper($modules->{$in});

Here is example output.

me@office:~$ perl Documents/scripts/find_module_use.pl 'RolePlaying::R
+andom'
$VAR1 = {
          'used by' => [
                         'RolePlaying::CharacterMutation',
                         'RolePlaying::Random::Range',
                         'RolePlaying::Random::Time',
                         'RolePlaying::Random::SpecialAttack',
                         'RolePlaying::Random::Title',
                         'RolePlaying::Random::WildPsionics',
                         'RolePlaying::Random::Descriptor',
                         'RolePlaying::Random::Spell',
                         'RolePlaying::Random::SavingThrow',
                         'RolePlaying::Random::Color',
                         'RolePlaying::Random::Monster',
                         'RolePlaying::Random::GemMetalJewelry',
                         'RolePlaying::Random::Thing',
                         'RolePlaying::Random::Weapon',
                         'RolePlaying::Random::Size',
                         'RolePlaying::Random::Food',
                         'RolePlaying::Random::Misc',
                         'RolePlaying::Random::Event',
                         'RolePlaying::Random::Water',
                         'RolePlaying::Random::Class',
                         'RolePlaying::Random::MagicItem',
                         'RolePlaying::Random::Color::VisiBone',
                         'RolePlaying::Random::Body::Function',
                         'RolePlaying::Random::Body::Modification'
                       ],
          'uses' => [
                      'strict',
                      'warnings',
                      'Exporter',
                      'List::Util'
                    ]
        };

If you know of a better way that will give me deeper or more sophisticated results, I am interested.

[Discipulus]

Hello Lady_Aleena,

in addition to scandeps you cam be interested to inspect what the following two lines do:

          perl -d:Modlist=options script.pl

          perl -MDevel::Modlist=options script.pl # equivalent

If you feel very insane you can be interested into one of my insane tricks: check modules used by a script and their version but please read carefully the thread to be aware of all caveats.

L*

[Tux]

I'd suggegst Devel::TraceUse:

$ perl -d:TraceUse=hidecore -MCSV -e'dcsv (in => [[1,2]])'
1,2
1
Modules used from -e:
   1.  CSV 1.29, -e line 0 [main]
   2.    Text::CSV_XS 1.29, CSV.pm line 1 [main]
  17.      Config_heavy.pl, Config.pm line 80
  18.        Config_git.pl, Config_heavy.pl line 1366 [Config]
  16.    Data::Peek 0.46, CSV.pm line 2 [main]
  19.      Perl::Tidy 20170521, Data/Peek.pm line 24 (eval 4)
  26.        Encode::ConfigLocal, Encode.pm line 61 (FAILED)
  46.        HTML::Entities 3.69, Perl/Tidy.pm line 4613 (eval 12) [Pe
+rl::Tidy::HtmlWriter]
  47.          HTML::Parser 3.72, HTML/Entities.pm line 152
  49.    JSON 2.93, CSV.pm line 3 [main]
  50.      JSON::XS 3.03, JSON.pm line 266 (eval 13)
  51.        Types::Serialiser 1.0, JSON/XS.pm line 115
Possible proxies:
   2 File/Copy.pm line 18, sub Perl::Tidy::BEGIN

[LanX]

The module has a a good "See Also" part discussing alternatives!

http://search.cpan.org/~book/Devel-TraceUse-2.095/lib/Devel/TraceUse.pm#SEE_ALSO

[stevieb]

update: d'oh! Another reminder not to post before first coffee of the day; my example below will not work in OPs case, as the modules are not yet on the CPAN :) Works well for those that are though.../update

Here's a modified version of a script I wrote that allows me to see if I have any of my distributions on the CPAN where I need to do required module version bumps in my Makefile.PLs for the required modules where I'm the author (original is here). It uses MetaCPAN::Client to do the hard work. Given a module name, it prints out its dependencies, as well as the reverse dependencies (which other modules use it. In CPAN speak, that's "upriver").

In this case, the module is hardcoded for the example, but pretty trivial to make it an argument.

use warnings;
use strict;

use MetaCPAN::Client;

my $c = MetaCPAN::Client->new;

my $dist = 'Mock::Sub';

check_deps($dist);
rev_deps($dist);

sub check_deps {
    my $dist = shift;

    $dist =~ s/::/-/g;

    my $release = $c->release($dist);
    my $deps = $release->{data}{dependency};

    print "Dependencies:\n\n";
    print "$_->{module}\n" for @$deps;
}
sub rev_deps {
    my $dist = shift;

    $dist =~ s/-/::/g;

    my @revdeps;
    my $rs = $c->rev_deps($dist);
    
    while (my $release = $rs->next){
        push @revdeps, $release->distribution;
    }
    
    print "\nReverse Dependencies:\n\n";

    for (@revdeps){
        s/-/::/g;
        print "$_\n";
    }
}

Output:

Dependencies:

perl
Carp
Scalar::Util
Test::More
ExtUtils::MakeMaker

Reverse Dependencies:

Devel::Examine::Subs
File::Edit::Portable
RPi::DigiPot::MCP4XXXX
Test::BrewBuild
App::RPi::EnvUI
Devel::Trace::Subs
PSGI::Hector

[hippo]

That's a handy script to keep around for sure. It might be worth pointing out to those not in the know that MetaCPAN::Client acts only on the meta data. That is, it will only report dependencies of a module which the meta data of that module declares to be dependencies. Such a list is not always correct, up-to-date, complete or even (in some cases) present at all.

For example, B::Tree uses GraphViz but doesn't declare this in its metadata, so using MetaCPAN::Client you would think it had no dependencies at all. B::Tree is far from being alone in this. The good people at CPANTS provide some statistics for this particular type of error on the part of module authors/maintainers. You'll see some of the modules on that list are recent releases and some by authors you might expect to be on top of such detail so it's a very easy human error to make. Module maintainers can always use something like Test::Prereq to help keep the dependency lists up to date.

For an end user looking to find dependencies, the lesson is to take the metadata with a pinch of salt.

[Anonymous Monk]

scandeps

[swl]

I assume scandeps.pl from Module::ScanDeps is meant?

https://metacpan.org/pod/Module::ScanDeps

https://metacpan.org/pod/distribution/Module-ScanDeps/script/scandeps.pl

[Lady_Aleena]

I am putting the updated code up so you can all see how I got what I wanted.

#!/usr/bin/perl
use strict;
use warnings;

use File::Find;
use Data::Dumper;

use Util::Line qw(line);

my @files;
sub wanted {
  my $file = $_ =~/.pm$/ ? $File::Find::name : undef;
  push @files, $file if $file;
}

my @directories = ('/home/me/Documents/');
find(\&wanted,  @directories);

my $module_directory = $directories[0].'www/files/lib/';
my @my_modules = map { my $file = $_; $file =~ s/$module_directory(.+)
+\.pm/$1/; $file =~ s/\//::/g; $file; } @files;

my $modules;
for my $file (@files) {
  open my $fh, '<', $file or die "$file: $!";
  my $file_convert = $file;
     $file_convert =~ s/$module_directory(.+)\.pm/$1/;
     $file_convert =~ s/\//::/g;

  while (my $line = <$fh>) {
    chomp($line);
    if ($line =~ s/^use ((:|\w)+)(.+)/$1/) {
      push @{$modules->{$file_convert}{'uses'}{'mine'}},     $1 if  gr
+ep(/$line/, @my_modules);    # what modules the module uses
      push @{$modules->{$file_convert}{'uses'}{'not mine'}}, $1 if !gr
+ep(/$line/, @my_modules);    # what modules the module uses

      push @{$modules->{$1}{'used by'}}, $file_convert; # what modules
+ use this module
    }
  }
}

my $in = shift;
my @ins = grep(/.*$in.*/, @my_modules);
for (sort @ins) {
  my $mod = $modules->{$_};
  line(0, $_);

  if ($mod->{'uses'}) {
    my $use = $mod->{'uses'};
    line(1, "$_ uses");

    if ($use->{'mine'}) {
      line(2, "My modules");
      line(3, $_) for sort @{$use->{'mine'}};
    }

    if ($use->{'not mine'}) {
      line(2, "Other modules");
      line(3, $_) for @{$use->{'not mine'}};
    }
  }

  if ($mod->{'used by'}) {
    line(1, "$_ used by");
    line(2, $_) for sort @{$mod->{'used by'}};
  }
  print "\n";
}

Here is example output.

Base::Data
  Base::Data uses
    My modules    # it uses one of my modules
      Base::Path
    Other modules # it uses several modules that are not mine
      strict
      warnings
      Exporter
      File::Basename
      File::Spec
      List::Util
  Base::Data used by # it is used by all of these modules
    Base::LineMagic
    Base::Menu
    Base::Page
    HTML::Forms
    Movie
    People
    RolePlaying::AssetCost
    RolePlaying::Character::AbilityScores
    RolePlaying::Character::Class
    RolePlaying::Character::GameTable::Psionics
    RolePlaying::Character::GameTable::SpellProgression
    RolePlaying::Monster
    RolePlaying::Random::Weapon
    RolePlaying::Random::WildPsionics
    RolePlaying::SpellList
    RolePlaying::Spellbook
    RolePlaying::WordFind
    Util::ExternalLinks

[Anonymous Monk]

If you run some kind of *nix and you just want to know the answer, it's probably quicker to use the existing tools rather than write a new one.

user@machine:~/src$ find . -name "*.p[lm]" -exec grep -h "^use\|requir
+e" {} + | sort |uniq
use strict;
use Template;
use v5.10;
use warnings;