Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Presto-chango on files [symlinks]

by Tanktalus (Canon)
on Mar 12, 2006 at 15:16 UTC ( #536098=CUFP: print w/replies, xml ) Need Help??

Here is a simple script that can turn symlinks into real files/directories. What I use this for is when I want two identical directory structures except for a few files. For example, what I'm using it for right as I write this is to have two versions of a Java application running on my system, but with different configurations. Any changes I make to the common data will be seen in both configurations, but the configurations themselves will be unique. In my case, it's because it's on a Linux box, and I share it to my Windows laptop via Samba, and thus configuration parameters that talk about directories are constantly getting messed up. With this, I have a myapp directory, and then I just did this:
$ ln -s myapp $ presto -d $ cd $ presto *.ini
Now all my configuration is separate, but the program and data are common. Everything else in the directory is a symlink. I could save a bit of space on the symlinks by building in a relative directory structure instead of a full path (which also makes everything moveable as a whole structure), but I figured the headache wouldn't be worth it. ;-)

Options are:

presto directories. This way you can just say "presto *" and you'll just get all the files, but "presto -d *" will presto the directories, too.
presto uses hardlinks instead of symlinks when creating subdirs. I assume hardlinks take up less diskspace when maintaining the commonality that if changed, they both get changed. You can still presto hardlinks with this program should you desire to have them separate later. Note that I don't think you can make hardlinks for directories, so those will remain symlinks.
I save this as "presto" in my personal "bin" directory. Working great so far.
#! /usr/bin/perl use strict; use warnings; use Getopt::Std; use File::Copy; use File::stat; use File::Spec; use IO::Dir; use Data::Dumper; my %opt; getopts('dh', \%opt); @ARGV = grep { $_ ne '.' and $_ ne '..' } map { s.[/\\]$..g; $_ } @ARG +V; foreach my $l (@ARGV) { unless (((lstat $l)->nlink() > 1 && -f _) || -l _) { print "$l isn't a link - skipping\n"; next; } if (-d $l) { unless (exists $opt{d}) { print "$l is a directory and -d wasn't specified - skippin +g\n"; next; } my $S = File::Spec->catdir(File::Spec->rel2abs(readlink($l))); unlink $l; mkdir $l; if (fork()) { chdir($l); my $Sdir = IO::Dir->new($S); while (defined (my $f = $Sdir->read)) { my $Sfile = File::Spec->catfile($S, $f); if (exists $opt{h} and -f $Sfile) { # if the hardlink fails, fall back to symlink link $Sfile, $f or symlink $Sfile, $f; } else { symlink $Sfile, $f; } } } } else { my $stat = stat($l); rename $l, "$l.old"; copy("$l.old", $l); unlink "$l.old"; utime $stat->atime, $stat->mtime, $l; chmod $stat->mode | 0200, $l; } } 1 while wait != -1;

Replies are listed 'Best First'.
Re: Presto-chango on files [symlinks]
by graff (Chancellor) on Mar 13, 2006 at 01:00 UTC
    If I understand your model correctly, you're doing something like this:
    • Create a set of files for an app in one directory, tailored to a given OS
    • Create a symink to that first directory, where you'll put a version of the app for some other OS
    • Run "presto" on the symlink, so that files known to be "common" are populated in the latter directory as links, and files expected to be OS-specific are populated as copies of the originals
    • Edit the copied set of files in some way to make them appropriate to the latter OS.

    If that's it in a nutshell, some folks might feel more comfortable with the "common" vs. "OS-specific" distinction made more explicit -- e.g. with a directory structure like:

    my_app/ common/ linux/ mswin/ macosx/

    and an "installation" process that simply takes all the files from the "common" directory plus all the files for a given target OS, when setting up an operational copy of "my_app" on a given machine. This gives you more reliability as well as flexibility -- OS-specific files don't need a specific file name pattern that you have to memorize, and there could even be a different number of files to install, depending on the OS.

    As for using hard links rather than symlinks, they are intrinsically cool, but they also may pose a hazard in your situation. Hard linking, which applies only to data files (not directories), is a matter of creating a second (third...) directory entry (in the same directory or a different one) that references the same inode on a given disk volume. Deleting a hard link means deleting a reference to the inode, and like perl data, when the last reference to an inode is deleted, its disk space is freed for re-use. The tricky thing is that a hard link is really just another directory entry pointing to an actual data file -- it's impossible to know that hard links are being used if you aren't specifically watching for them and paying really close attention.

    In your case, the hazard comes with possibly forgetting the distinction between an OS-specific file and a common file. If you decide to edit a hard-linked file in "", you are actually editing the same disk blocks referenced by that file name in the other "myapp" directory as well. If the edit makes the file OS-specific, the other version of the app breaks, and if you didn't keep a truly separate copy of the original file, you have a problem that may take a while to fix.

      Hey, if it were my application that I owned/developed, I would just fix the root problem in the configuration. ;-) There are some reasonably simple changes to the core configuration that would just make everything a relative path to the program, instead of hardcoding /home/foo/java/my_app and z:\java\my_app as part of the configuration, which would solve my entire problem. However, while the application is open source, I don't know that I really want to fork the project over it ;-) Any other solution that I can think of requires much more maintenance from me, and this was relatively simple and straight forward. Until the owners of the application see things my way (unlikely - they mostly use Windows or Mac OSX, and probably don't have computers sharing like this), this is good enough.

      Further, the data that I work on when I'm on Linux needs to be the same data that I work on when I'm on Windows. Constantly "installing" it (or copying or anything) back and forth to synchronise is maintenance. Symlinks means that the data is physically the same, which is what I want, while prestoing the configuration means that the configuration is still separate. There's still a lot of overlap in configuration that is OS-independant, but, again, I don't own the master code.

      Finally, hardlinks merely takes the problem of symlinks all being the same file, and makes it difficult to figure out in the other direction. That is, with symlinks you can't know that anyone is symlinked to you. But with hardlinks, you can, though it's not obvious whether you're looking at the original or at the linked directory entries. They make some easy things hard (seeing that it's a link from the new directory) and some impossible things merely possible (seeing that it's linked to from the original path, although still not where those other links are).

      Anyway, this CUFP was really only intended as something that others may find as an expedient "good enough" solution to a problem, not necessarily the best ideal long-term solution to the problem.

      Another place to use this is if you're trying to modify a read-only source, such as an NFS partition or CDROM, without copying the entire code locally. Just create a symlink to the read-only source, presto -d all the way to the directory that contains the file you want to change, and then either presto the file to allow you to modify it, or delete the symlink and replace it with the new file. It can come in handy when the support guys ask you to copy their DVD set to your hard drive to modify a single installation file. Rather than chewing up 4GB of disk space, you chew up only the size of the changed files plus a few (hundred?) inodes.

      # cd ~/tmp # ln -s /mnt/cdrom . # presto -d cdrom # cd cdrom # presto # vi # ./

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://536098]
Approved by wfsp
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (4)
As of 2017-08-16 19:42 GMT
Find Nodes?
    Voting Booth?
    Who is your favorite scientist and why?

    Results (272 votes). Check out past polls.