Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Using Cartons to automate module installs

by Aldebaran (Chaplain)
on Jan 30, 2020 at 20:17 UTC ( #11112133=perlquestion: print w/replies, xml ) Need Help??

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I've presented lately with these awful "trying to do anything on windows" threads, and my results were so dismal for installing WWW::Mechanize::Chrome on windows 10, strawberry perl, I redoubled my efforts to get an ubuntu platform to develop on instead. Well, by golly, I figured out a way to make my laptop dual-boot. It took me over a hundred failures, but then I failed to fail.

Now I have a fresh debian install, and I wanted to address some ideas that have come up in other threads lately. (I've been lurking, which requires little effort. As an intermediate with more to learn than I already know, I also need to do write-ups, else stop evolving with perl.)

I will put the details of how to get a dual boot on this particular $169 laptop between readmore tags, so that that we can just move on.

I believe this manufacturer partners with windows so that it will have windows on it when you buy it. This is what it looks like from windows' perspective:

C:\Users\tblaz\Documents\evelyn>systeminfo Host Name: redacted OS Name: Microsoft Windows 10 Home OS Version: 10.0.17763 N/A Build 17763 OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Workstation OS Build Type: Multiprocessor Free Registered Owner: redacted Registered Organization: Product ID: redacted Original Install Date: 8/6/2019, 1:15:23 AM System Boot Time: 1/5/2020, 7:50:07 PM System Manufacturer: LENOVO System Model: 81MV System Type: x64-based PC Processor(s): 1 Processor(s) Installed. [01]: Intel64 Family 6 Model 142 Stepping 1 +1 GenuineIntel ~2304 Mhz BIOS Version: LENOVO ASCN29WW, 3/26/2019 Windows Directory: C:\Windows System Directory: C:\Windows\system32 Boot Device: \Device\HarddiskVolume1 System Locale: en-us;English (United States) Input Locale: en-us;English (United States) Time Zone: (UTC-08:00) Pacific Time (US & Canada) Total Physical Memory: 3,976 MB Available Physical Memory: 989 MB Virtual Memory: Max Size: 9,208 MB Virtual Memory: Available: 3,709 MB Virtual Memory: In Use: 5,499 MB Page File Location(s): C:\pagefile.sys Domain: WORKGROUP Logon Server: redacted Hotfix(s): 11 Hotfix(s) Installed. ... Network Card(s): 2 NIC(s) Installed. [01]: Realtek 8821CE Wireless LAN 802.11ac +PCI-E NIC Connection Name: Wi-Fi DHCP Enabled: Yes DHCP Server: 10.0.0.1 IP address(es) ... [02]: Bluetooth Device (Personal Area Netwo +rk) Connection Name: Bluetooth Network Co +nnection Status: Media disconnected Hyper-V Requirements: VM Monitor Mode Extensions: Yes Virtualization Enabled In Firmware: Yes Second Level Address Translation: Yes Data Execution Prevention Available: Yes C:\Users\tblaz\Documents\evelyn>

The central drama is trying to get the bios to recognize the usb stick, and our friends in Redmond are always trying to make that more difficult, which I can understand from a security standpoint. As a consumer, I like to be free to install an operating system that I can develop on. It hardly feels like the same machine. Details on how to adjust window's settings here.

First, I would like to turn to haukex's response in Re: List of Perl library modules needed run scripts. As I do the basics of a perl install and installing my html template, I find myself installing the same modules over and over again. I would like to apply the virtue of laziness to this. For example, every time I boil up a perl install, I would like to install these:

CPAN Log::Log4perl Perl::Tidy YAML
Q1) How do I create and roll out a Carton of these? (I have another machine with a new ubuntu install to test on.) Is a Carton the right tool?

As I look at this output during a basic install, I wonder if I want to increase the above list?

# === Test Suggests === # # Module Want Have # ------------------------ ----- -------- # CPAN::Meta::Check 0.011 missing # CPAN::Meta::Requirements any 2.140 # PadWalker any missing # Test::Tester 0.108 1.302073 # ... # # === Other Modules === # # Module Have # --------------- ---------- ... # Module::Runtime missing

I used a couple new tools that I found on this source to show me what I have:

$ ./1.mod.pl CPAN Capture::Tiny Config::Tiny Email::Find HTML::FromText Log::Log4perl Module::ScanDeps Net::SFTP::Foreign Path::Tiny Perl Perl::Tidy Spiffy Test::Base Test::Deep Test::FailWarnings Test::Fatal Test::MockRandom Test::More::UTF8 Test::Pod Test::Requires Test::WarningsExtUtils::Installed Test::YAML Text::Diff Text::Template Unicode::UTF8 YAML $ cat 1.mod.pl #!/usr/bin/perl -w use ExtUtils::Installed; my $inst = ExtUtils::Installed->new(); my @modules = $inst->modules(); foreach $module (@modules){ print $module . "\n"; } __END__ $

Q2) How do I know whether I have all the core modules? How do I not leave it to chance?

I also want a Carton, or equivalent bundling, for my html templating system. I used the Module::ScanDeps functionality on its primary pm's:

$ ./scandeps.pl html7.pm 'Config::Tiny' => '2.24', 'Email::Find::addrspec' => '0.09', 'HTML::Entities' => '3.69', 'HTML::FromText' => '2.07', 'HTML::Parser' => '3.72', 'Net::SFTP::Foreign' => '1.90', 'Net::SFTP::Foreign::Attributes' => '1.68_05', 'Net::SFTP::Foreign::Buffer' => '1.68_05', 'Net::SFTP::Foreign::Common' => '1.76_02', 'Net::SFTP::Foreign::Constants' => '1.63_05', 'Net::SFTP::Foreign::Helpers' => '1.74_06', 'Net::SFTP::Foreign::Local' => '1.57', 'Path::Tiny' => '0.110', 'Text::Template' => '1.58', 'Unicode::UTF8' => '0.62', 'trans1' => 'undef', 'utils1' => 'undef', $ ./scandeps.pl trans1.pm 'Config::Tiny' => '2.24', 'Net::SFTP::Foreign' => '1.90', 'Net::SFTP::Foreign::Attributes' => '1.68_05', 'Net::SFTP::Foreign::Buffer' => '1.68_05', 'Net::SFTP::Foreign::Common' => '1.76_02', 'Net::SFTP::Foreign::Constants' => '1.63_05', 'Net::SFTP::Foreign::Helpers' => '1.74_06', 'Net::SFTP::Foreign::Local' => '1.57', 'Path::Tiny' => '0.110', 'Unicode::UTF8' => '0.62', $

Q3) How do I take this output and instruct another machine to install this?

Finally, I have been developing new bash aliases with help from handy bash aliases. Some of these are perl related, some not. I haven't used them all yet, and I've added '| more' to verbose ones:

alias ..='cd ..' alias ...='cd ../../../' alias ....='cd ../../../../' alias .....='cd ../../../../' alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo termin +al || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//; +s/[;&|]\s*alert$//'\'')"' alias cx='chmod +x ' alias dir='dir --color=auto' alias egrep='egrep --color=auto' alias fgrep='fgrep --color=auto' alias grep='grep --color=auto' alias halt='sudo /sbin/halt' alias l='ls -CF' alias l.='ls -d .* --color=auto' alias la='ls -A' alias ll='ls -alF' alias ls='ls --color=auto' alias meminfo='free -m -l -t' alias music='mplayer --shuffle *' alias nplaymp3='for i in /nas/multimedia/mp3/*.mp3; do mplayer "$i"; d +one' alias nplayogg='for i in /nas/multimedia/ogg/*.ogg; do mplayer "$i"; d +one' alias nplaywave='for i in /nas/multimedia/wave/*.wav; do mplayer "$i"; + done' alias playavi='mplayer *.avi' alias playmp3='for i in *.mp3; do mplayer "$i"; done' alias playmp4='for i in *.mp4; do mplayer "$i"; done' alias playogg='for i in *.ogg; do mplayer "$i"; done' alias playwave='for i in *.wav; do mplayer "$i"; done' alias poweroff='sudo /sbin/poweroff' alias pscpu='ps auxf | sort -nr -k 3 | more' alias pscpu10='ps auxf | sort -nr -k 3 | head -10' alias psmem='ps auxf | sort -nr -k 4 | more' alias psmem10='ps auxf | sort -nr -k 4 | head -10 | more' alias pt='perltidy -i=2 -b -utf8 ' alias reboot='sudo /sbin/reboot' alias sc='sudo cpan' alias shutdown='sudo /sbin/shutdown' alias update='sudo apt-get update && sudo apt-get upgrade' alias vlc='vlc *.avi'

alias sc='sudo cpan' is already chugging away as keystroke-saver. Q4) What perl aliases do you use? What aliases in general?

I'm always humbled by the quality of responses I get if I can manage to post a succinct question. Thank you for your response.

Replies are listed 'Best First'.
Re: Using Cartons to automate module installs
by haukex (Chancellor) on Jan 30, 2020 at 20:45 UTC
    Q1) How do I create and roll out a Carton of these? (I have another machine with a new ubuntu install to test on.) Is a Carton the right tool?

    If all you care about is getting the latest version of the modules, then you don't need Carton, only App::cpanminus (cpanm), because once you have a cpanfile, the command cpanm --installdeps . will install everything listed in that file. Carton is useful if you want to install specific versions of modules, to exactly replicate a module setup on a different machine. See Carton::Doc::FAQ.

    Q2) How do I know whether I have all the core modules? How do I not leave it to chance?

    Debian should install its perl package by default, which includes its perl-modules package. If for some reason it doesn't get installed, you can always sudo apt-get install perl. If you're building a fresh perl with e.g. perlbrew, you should be getting all core modules by default as well.

    Q3) How do I take this output and instruct another machine to install this?

    That particular output is suited for a Makefile.PL format. You could of course write a quick&dirty oneliner to turn it into cpanfile format:

    $ scandeps.pl -R script.pl | perl -ne \ 'printf qq{requires "%s", "%s";\n}, eval'
    Q4) What perl aliases do you use? What aliases in general?

    The only one I personally have is alias prl='perl -wMstrict -MData::Dump', the rest I do by hand. I also have a ~/.perltidyrc and ~/.perlcriticrc.

    Update: Shortened oneliner.

Re: Using Cartons to automate module installs
by choroba (Archbishop) on Jan 31, 2020 at 10:13 UTC
    > Q2) How do I know whether I have all the core modules? How do I not leave it to chance?

    Why do you care? Name all the dependencies in your Makefile.PLs, or cpanfiles, including the core dependencies.

    > Q4) What perl aliases do you use? What aliases in general?

    I don't use aliases much, as they are less flexible than functions and scripts. You can find some of my tools I've carried around for years in this GitHub repo. Some of them are written in Perl (chjoin, dups, git-graphs), some are Perl related (bead, perl-etags.pl).

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
      Q2) How do I know whether I have all the core modules? How do I not leave it to chance?
      Why do you care?

      I suspect it's because the first paragraph of the Carton docs is:

      Carton only works with perl installation with the complete set of core modules. If you use perl installed by a vendor package with modules stripped from core, Carton is not expected to work correctly.
        I suspect it's because the first paragraph of the Carton docs is:

        You were exactly right, and while the absense of core modules might be problematic for a Carton, it is immaterial to a cpanfile, which, after a ton of reading about it, seems to suit my needs fine. I have been trying to implement them in varying ways and find success in a bash script that relies on scandeps.pl .

        It seems to me that it invites disaster to compose a cpanfile by hand, never mind the tedium. It has to be named 'cpanfile' for the command sudo cpanm --installdeps . to work. There has been a fair amount of evolution in these matters over the years, so newer sources of information are more reliable. I found these html pages helpful and up to date: specifying-dependencies-for-your-cpan-distribution and introduction-to-distribution-metadata. The bash script wraps the invocation of scandeps.pl, creating a directory to house the files to be scanned and the resulting cpanfile. I also get a "paper trail" for what happens on STDOUT with | tee -a "$out" .

        Here is sample output:

        $ ./1.cpan_dir.sh mkdir: cannot create directory ‘logs’: File exists basename dollar sign zero is 1.cpan_dir.sh path is /logs Munged time is 02-03-2020_17-25-09 out fn is /home/hogan/Documents/hogan//logs/02-03-2020_17-25-09.log /home/hogan/Documents/hogan /home/hogan/Documents/hogan/cpan_file_dir2 /home/hogan/Documents/hogan/cpan_file_dir2/cpanfile /home/hogan/Documents/hogan/cpan_file_dir2/scandeps.pl --> Working on . Configuring /home/hogan/Documents/hogan/cpan_file_dir2 ... OK <== Installed dependencies for .. Finishing. behold your output: requires "ExtUtils::MM_AIX", "7.44"; requires "ExtUtils::MM_Any", "7.44"; requires "ExtUtils::MM_BeOS", "7.44"; requires "ExtUtils::MM_Cygwin", "7.44"; requires "ExtUtils::MM_DOS", "7.44"; requires "ExtUtils::MM_Darwin", "7.44"; requires "ExtUtils::MM_MacOS", "7.44"; requires "ExtUtils::MM_NW5", "7.44"; requires "ExtUtils::MM_OS2", "7.44"; requires "ExtUtils::MM_QNX", "7.44"; requires "ExtUtils::MM_UWIN", "7.44"; requires "ExtUtils::MM_Unix", "7.44"; requires "ExtUtils::MM_VMS", "7.44"; requires "ExtUtils::MM_VOS", "7.44"; requires "ExtUtils::MM_Win32", "7.44"; requires "ExtUtils::MM_Win95", "7.44"; requires "ExtUtils::MakeMaker", "7.44"; requires "File::Temp", "0.2309"; requires "Module::ScanDeps", "1.27"; duration=20 Mon Feb 3 17:25:29 PST 2020 $

        Source:

        #!/bin/bash # # # keep a log named by time stamp export PATH=:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/ +bin set -u app=${0##*/} mkdir "logs" pathto=/logs timename=$(date +"%m-%d-%Y_%H-%M-%S") out=$HOME/Documents/hogan/$pathto/$timename.log declare -ir SUCCESS=0 declare -ir E_FATAL=1 if [ 1 -eq 1 ] ; then echo "basename dollar sign zero is" $app echo "path is" $pathto echo "Munged time is" $timename echo "out fn is $out" echo "Time is $timename " > "$out" fi # end if [ 1 -eq 0 ] pwd | tee -a "$out" folder='cpan_file_dir2' mkdir $folder cp /home/hogan/.cpan/build/Module-ScanDeps-1.27-0/blib/script/scandeps +.pl $PWD/$folder/ cd $folder pwd | tee -a "$out" scandeps.pl -R *.pl | perl -ne 'printf qq{requires "%s", "%s";\n}, e +val' >>cpanfile ls -d $PWD/* | tee -a "$out" sudo cpanm --installdeps . | tee -a "$out" echo "behold your output:" | tee -a "$out" cat cpanfile | tee -a "$out" gedit $out & read -n 1 echo "duration=$SECONDS" | tee -a "$out" date | tee -a "$out" exit $SUCCESS

        So, yay, that I'm getting results, but I don't quite understand the line of code I'm using to get them. Can you "talk through" what happens on this line, in particular, how the %s's get populated?

        scandeps.pl -R *.pl | perl -ne   'printf qq{requires "%s", "%s";\n}, eval'

        Thank you for your comments.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11112133]
Approved by haukex
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (8)
As of 2020-03-30 11:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    To "Disagree to disagree" means to:









    Results (175 votes). Check out past polls.

    Notices?