Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

ExtUtils::MakeMaker and dot files

by SleepyJay (Beadle)
on Jul 18, 2012 at 00:11 UTC ( #982352=perlquestion: print w/ replies, xml ) Need Help??
SleepyJay has asked for the wisdom of the Perl Monks concerning the following question:

I tend to edit files on a Linux server with a Mac. When I do this, the Mac will add these .AppleDouble files all over the place. I won't go into (even if I could) the history and/or purpose of these files, but suffice to say they are of zero use to my Perl modules.

There are at least two problems I run into with these files when I do a make. One, sometimes the make or the Perl compiler simply fails because of these files. Two, the existence of .AppleDouble files means they often get put into the "i386-linux-thread-multi" area. If they get installed twice (once in multi and one in the "correct" place), Perl is forced to guess which one of the modules to actually use. Usually this means I spend a few minutes making several changes to a file that doesn't appear to execute differently, until I realize the double problem, curse loudly, and clear out the offending module.

We have tried to combat this problem by running a script to remove the .AppleDouble files just before the make--but they keep coming back like the common cockroach. We have also tried server-configuration solutions, but the versions of the server thingies are all the wrong versions and so cause other problems. So, that's no good either.

Does anyone know of a way to make MakeMaker exclude these files from arriving at the Perl lib directories?

While the internet suggests I'm not alone in this problem, I cannot find a way to solve it.



Comment on ExtUtils::MakeMaker and dot files
Replies are listed 'Best First'.
Re: ExtUtils::MakeMaker and dot files
by davido (Archbishop) on Jul 18, 2012 at 07:15 UTC

    This really does seem like it should be a non-issue. The default MANIFEST.SKIP is usually smart enough, but if you find that you have to add some skips to it, look at ExtUtils::Manifest, where you can read everything you ever wanted to know about writing a MANIFEST and a MANIFEST.SKIP file.

    For what its worth, I never use the make manifest command, but do find make distcheck to be useful. The former will just pull in all files in the build path that aren't skipped by either an explicit skip-file, or by the default one. I don't care for that. But the make distcheck command will just spit out a bunch of messages letting me know what files probably should be added to the manifest. In this case, keeping an up-to-date skip file just serves to silence the warnings.


      Ha. I certainly agree: it shouldn't be an issue. But it is. :)

      Maybe I wasn't clear and it's totally my fault. When I said ".AppleDouble" files, I actually meant "shadow files that live inside of .AppleDouble directories". So, if I have a file "", I would likely also have a related file "./.AppleDouble/". Sorry about that.

      My MANIFEST and MANIFEST.SKIP files are properly set up (and do not include .AppleDouble directories). My script is something like: make manifest; make clean; removeAppleDouble; perl Makefile.PL; make; make install On the make step, all of files like .AppleDouble/ get put into blib. That's exactly what I'm trying to prevent!

      (I don't always run that set of commands, usually all I need is removeAppleDouble; make install. But the problem is still there.)

      Essentially, what I want is to stop make from looking in certain directories.

      I am no expert on make, and I've been just taking the commands on faith without fully understanding the intricacies, so I may be missing something.

        Then your MANIFEST.SKIP needs to have something like \.AppleDouble/ in it. (Untested.)

        MANIFEST.SKIP understands regular expressions, so any expression that matches the starting part of the path ought to do the trick. Once you've included your regex to exclude .AppleDouble and its children, your make manifest should be fine. This works for .git, _Include, blib, and many other common exclusions.


        My script is something like: make manifest; make clean; removeAppleDouble; perl Makefile.PL; make; make install

        :) Well, there's your problem!

         make manifest without a MANIFEST.SKIP gets you into trouble

        $ dmake manifest C:\perl\5.14.1\bin\MSWin32-x86-multi-thread\perl.exe "-MExtUtils::Mani +fest=mkmanifest" -e mkmanifest Added to MANIFEST: .AppleDouble/


        $ dmake help |grep "make manifest" -A3 make manifest rewrites the MANIFEST file, adding all remaining files found ( +See ExtUtils::Manifest::mkmanifest() for details)

        With an addition to MANIFEST.SKIP all goes as planned

        $ dmake manifest C:\perl\5.14.1\bin\MSWin32-x86-multi-thread\perl.exe "-MExtUtils::Mani +fest=mkmanifest" -e mkmanifest $ dmake skipcheck C:\perl\5.14.1\bin\MSWin32-x86-multi-thread\perl.exe "-MExtUtils::Mani +fest=skipcheck" -e skipcheck Skipping .AppleDouble/ Skipping Makefile Skipping MANIFEST.bak Skipping MYMETA.json Skipping MYMETA.yml
Re: ExtUtils::MakeMaker and dot files
by Anonymous Monk on Jul 18, 2012 at 00:23 UTC

    Um, don't put them in the distribution? Use  make dist which will consult MANIFEST/#MANIFEST.SKIP, which by default skips dot_underscore files?

Re: ExtUtils::MakeMaker and dot files
by Anonymous Monk on Jul 21, 2012 at 00:47 UTC

      HUZZAH! You have done it!

      I had to add another line to skip, but other than that, this was perfect. Thanks! Gold star for you.

      We really did try to read the documentation (and even the code), but we hadn't yet figured it out. I knew the Perl Monks would get it eventually. :)

      Just to round out the conversation, the SSH client I was trying to use instead of AFP couldn't refresh in the remote to local direction (not even manually) and was causing other issues. So we went back to AFP. But this time, I know the exact reason this was a problem. Whenever I would run make, my IDE would scan for changes, write the .AppleDouble resource fork, and sometimes the make would grab that .AppleDouble. That's where it was causing problems.


      BTW, it's perhaps not much of an excuse, but I just searched for libscan in the perldoc for ExtUtils::MakeMaker and found only one hit under PMLIBDIRS. I don't think this was very obvious of a solution. (Thanks again, this is huge.).

        I thought this was closed, but it turned out to only be a small part of the problem. I think we have fixed it completely.

        If anyone cares, here it is...

        When I navigate around the AFP connected folders in Finder, it writes .AppleDouble folders. But, when I am running make, it's in terminal window, so technically if I remove the metadata, make shouldn't find any .AppleDouble files and it should work as planned.

        However, this is where the editor I use, Komodo, comes it. There is some background process (we've seen it running) of Komodo's that essentially scans all of the documents in your source folders. This includes the Perl make folder--blib. This process tickles AFP for the blib folder, so a .AppleDouble folder (and some files) are written there. If there is a blib/arch directory and it isn't "empty" (it may contain some files that are ignored), then Perl will install to that alternate location and screw everything up. So, Komodo's scans are allowing that folder to not be empty, and thus Komodo is indirectly causing this problem.

        The current fix is to add a line way down in the bowels of Perl that ignores .AppleDouble directories when testing for an "empty" blib/arch folder. That is, of course, a crappy fix because we have to install it separately for each version and machine for which we want to use. (There might be a way to override the function without changing the source.)

        The quick summary is thus: We still need to clean metadata each time before the make. Also, we still need to override the libscan function to avoid these files during the creation of the Makefile. And, we need a stupid one line fix to ignore .AppleDoubles way down deep in Perl's source code.

        BTW, I was working around this by running make clean and perl Makefile.PL before make and make install. That was working nearly flawlessly because the check to see if blib/arch is "empty" doesn't run if blib is missing--which of course it will be after make clean is run. But of course by remaking my entire Perl source each time, I was totally invalidating the purpose of make (build only what has changed).

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://982352]
Approved by kcott
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (5)
As of 2015-11-28 15:11 GMT
Find Nodes?
    Voting Booth?

    What would be the most significant thing to happen if a rope (or wire) tied the Earth and the Moon together?

    Results (743 votes), past polls