Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Loading modules remotely

by bliako (Prior)
on Jul 20, 2018 at 09:29 UTC ( #1218872=perlquestion: print w/replies, xml ) Need Help??

bliako has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I would like to know if there is already a Perl module which provides a network server for serving Perl modules from a remote location and a directive to be embeded in a perl script so as to load said modules from the server. Preferably using encryption.

For example,

use Module::X from ssh='.ssh/id_rsa';

thanks, bliako

Replies are listed 'Best First'.
Re: Loading modules remotely
by Corion (Pope) on Jul 20, 2018 at 12:32 UTC

    Also see lazy (and its references) for an example implementation.

    These will not do ssh transport and do expect modules to conform to the CPAN standard, but maybe that isn't all bad.

Re: Loading modules remotely
by QM (Parson) on Jul 20, 2018 at 10:02 UTC
    It seems like this has been invented before, but I can't recall where I saw my favorite solution.

    This node might help.

    There are other solutions which suggest pulling a module over from the remote host. And others how to use a module without installing it (under some conditions).

    Quantum Mechanics: The dreams stuff is made of

      that node has a lot of references thanks

Re: Loading modules remotely
by marto (Cardinal) on Jul 20, 2018 at 10:07 UTC

    I've mentioned this a few times (for example), as the URL is not currently accessible check the way back machine archive. The Frankfurt slides detail 'Application repositories' which sounds like what you want to achieve.

Re: Loading modules remotely
by LanX (Cardinal) on Jul 20, 2018 at 09:59 UTC
    How would this work where an installation is needed?

    Especially with XS?

    I can see how to implement this for the simplest case*, but this tastes like an XY Problem...

    ... I'm reluctant to search CPAN for you before the question is clearer.

    Most probably you just need PAR::Packer

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

    *) by hooking into %INC

      yes XS poses a problem, though it can be solved with PAR from a binary-compatible server.

      Hooking into %INC is by far the simplest for my purposes (no XS, can be loaded into memory without intermediary files)

Re: Loading modules remotely
by tobyink (Canon) on Jul 20, 2018 at 12:58 UTC

    Personally, I'd just use stuff we know already works.

    Modules are just files. There are already solutions for accessing files on a remote system. Using things like SAMBA and sshfs, it is possible to mount remote directories as if they were part of the local file system.

    Then just add the right path to @INC.

      Yep, at the beginning it sounded like a lot. But after all the comments, it just amounts to running the script, load LWP module, transfer the module archives, possibly encrypted, from a remote server (public or private), extract/unencrypt the content and load within same script without even writing anything to local disk, which is what I wanted to do. I hope it will work as it is just a sketch as of this evening.

      I wouldn't call it an X-Y problem but may be it has another name like the C-B-A problem.

      thanks, bliako

        This loads module source from a database.

        package dbLoader; use Carp; use Data::Dumper; use DBI; use English qw(-no_match_vars); use Fcntl qw(:flock); use Scalar::Util qw(blessed); use strict; use warnings; my ($BUFFER,$LOGNAME,$VFH); # Default prefix (preference) order my $PREFIX_ORDER_AREF=['Deployed']; sub import { my $self=shift; # set up $PREFIX_ORDER_AREF $PREFIX_ORDER_AREF=[@_] if (@_); return; }; # import: my $KNOWN_PACKAGES_HREF; my ($DBH,$STH); { # INTERNALS: # Yes, you can use caller in a @INC coderef! sub _dbLoader { # warn Data::Dumper->Dump([\[caller(0)]],[qw(*call +er)]),' '; print $VFH "Seeking '$_[1]'.\n"; return # unless there's a prefix order sequence unless (@{$PREFIX_ORDER_AREF}); print $VFH Data::Dumper->Dump([\@_],[qw(*_)]); my (undef,$path_S)=@_; s{[/\\]}{::}g, s{\.pm$}{} for (my $package_s=$path_S); print $VFH Data::Dumper->Dump([\$path_S,\$package_s],[qw(*path +_S *package_s)]); return # unless the package is in the this library && we have +a prefix match unless (exists $KNOWN_PACKAGES_HREF->{$package_s} && grep { + exists $KNOWN_PACKAGES_HREF->{$package_s}{$_} } @{$PREFIX_ORDER_AREF +}); #my $body_sref; my $body_s; eval { print $VFH Data::Dumper->Dump([\$PREFIX_ORDER_AREF],[qw(*P +REFIX_ORDER_AREF)]); for my $prefix_s (@{$PREFIX_ORDER_AREF}) { if (exists $KNOWN_PACKAGES_HREF->{$package_s}{$prefix_ +s}) { # Found one ... get its body and stash a reference + to that body print $VFH Data::Dump([\$prefix_s],[qw(*prefix_s)] +); $STH->execute($package_s,$prefix_s); my $value_aref=$STH->fetchrow_arrayref(); $body_s=$value_aref->[0]; #print $VFH Data::Dumper->Dump([\$body_s],[qw(*bod +y_s)]); # Mark it as this rather than as CODE() $INC{$path_S}="DBI:Pg:$prefix_s:$path_S"; # And our work is done last; }; }; }; if (my $error=$@) { Carp::confess $@; }; ## Each of the following will work! return \$body_s; ## ## or #open my $fh,'<',\$body_s # or die "Couldn't open '\$body_ref' for reading. $!"; #return $fh; ## ## or #open my $fh,'<',\$body_s # or Carp::confess "Couldn't open string for reading! $!+"; #return ( # sub { # if ($_=<$fh>) { # return 1; # } # else { # return 0; # }; # } # Anonymous sub: # ); ## }; # _dbLoader: } # INTERNALS: BEGIN { # warn Data::Dumper->Dump([\[caller()]],[qw(*caller)]),' '; #NB: $INC{"$"} will be defined only when the package is +use'd. ($LOGNAME=__PACKAGE__)=~ s{::}{/}g; $LOGNAME=qq{$INC{"$"}.log}; # warn Data::Dumper->Dump([\$LOGNAME],[qw(*LOGNAME)]),' '; }; UNITCHECK { # warn Data::Dumper->Dump([\[caller()]],[qw(*caller)]),' ' +; local $Data::Dumper::Terse=1; local $Data::Dumper::Indent=1; # Pre-allocate vec($BUFFER,64*1_024,8)=0; $BUFFER = ""; open $VFH,'>',\$BUFFER or die "Couldn't open in-memory-file for writing. $!"; my $configuration_href; # Load the configuration $ENV{HOME}//="$ENV{HOMEDRIVE}$ENV{HOMEPATH}"; my $filename; # The preference is an eval and a Data::Dumper .dmp file # otherwise use Config::General and save the configuration to a Da +ta::Dumper .dmp for subsequent uses if (-f ($filename="$ENV{HOME}/config.dmp")) { # We can use eval. my $string=do {open my $fh,'<',$filename or die "Couldn't open + '$filename' for reading. $!"; local $/; <$fh>; }; print $VFH Data::Dumper->Dump([\$string],[qw(*string)]); $configuration_href=eval $string or die "Deserialization failed: $@"; } elsif (-f ($filename="$ENV{HOME}/config.ini")) { # Needs Config::G +eneral. require Config::General; $configuration_href={Config::General->new($filename)->getall}; my $fh; open $fh,'>',$filename="$ENV{HOME}/config.dmp" and do { require Data::Dumper; print {$fh} Data::Dumper->Dump([$configuration_href]); close $fh; } or warn "Couldn't open '$filename' for writing. $!"; } else { # WTF die "There is neither a config.dmp nor an config.ini file."; }; print $VFH Data::Dumper->Dump([\$configuration_href],[qw(*configur +ation_href)]); my $db='pg'; print $VFH Data::Dumper->Dump([\$db],[qw(*db)]); eval { $DBH=DBI->connect( @{$configuration_href->{db}{$db}}{qw(dsn username pass +word)} ,{ PrintError=>1, RaiseError=>1 } ); # Create a (global) hashref of packages/prefixes $STH=$DBH->prepare(<<"__SQL__"); SELECT package,prefix FROM packages; __SQL__ $STH->execute(); my $field_aref=$STH->{NAME_lc}; while (my $value_aref=$STH->fetchrow_arrayref()) { my %_h; @_h{@$field_aref}=@$value_aref; $KNOWN_PACKAGES_HREF->{$_h{package}}{$_h{prefix}}=undef; }; $STH->finish(); print $VFH Data::Dumper->Dump([\$KNOWN_PACKAGES_HREF],[qw(*KNO +WN_PACKAGES_HREF)]); # Statement handle for fetching source(s) $STH=$DBH->prepare(<<"__SQL__"); SELECT body FROM packages WHERE package = ? and prefix = ? +; __SQL__ print $VFH "SELECT prepared.\n"; # prepend the loader unshift @INC,\&_dbLoader; print $VFH "Prepended \@INC.\n"; }; die $@ if ($@); }; # UNITCHECK: #CHECK { # warn Data::Dumper->Dump([\[caller()]],[qw(*caller)]),' '; # }; # CHECK: #INIT { # warn Data::Dumper->Dump([\[caller()]],[qw(*caller)]),' '; # }; # INIT: END { # warn Data::Dumper->Dump([\[caller()]],[qw(*caller)]),' '; # Modules loaded by _dbLoader printf $VFH ("%40s\t%s\n",$_,$INC{$_}) for (sort grep { $INC{$_}=~ m{DBI:Pg:} } keys %INC); return # if there's nothing to do. unless (tell $VFH); # Open our log open(my $fh,'>>',$LOGNAME) or die "Could not open '$LOGNAME' for appending. $!"; # Get lock flock($fh,LOCK_EX) or die "Could not lock '$LOGNAME'. $!"; # and write $BUFFER to the log file print $fh "\n<< $PROCESS_ID\n", substr($BUFFER,0,tell $VFH), "\n$PROCESS_ID >>\n"; # and close close $fh or die "Could not write '$LOGNAME' - $!"; }; END; 1; __END__
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1218872]
Approved by hippo
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (9)
As of 2021-04-14 15:05 GMT
Find Nodes?
    Voting Booth?

    No recent polls found