Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

[OT] Best Use of Subversion Branches in Perl Development

by jkeenan1 (Deacon)
on Feb 02, 2007 at 02:31 UTC ( [id://597867]=perlquestion: print w/replies, xml ) Need Help??

jkeenan1 has asked for the wisdom of the Perl Monks concerning the following question:

At $Job, I am admonished, "Always develop in a branch. Never develop in trunk." So when I recently earned commit privileges to the Parrot subversion repository, one of the first things I did was to create a branch in which to work on my refactoring of some of Parrot's build tools written in Perl 5.

The problem is: I'm spending too much time maintaining the branch. I seek the wisdom of the monks in figuring out how to use Subversion branches most expeditiously.

Let me describe the problem more precisely. At any given time, I'm taking a script executed by make and located in the tools/build/ directory, extracting its functionality into a new module placed in the lib/Parrot/ directory, and writing a test suite placed in the t/tools/ directory. While testing the new module's subroutines, I also have need to read source code such as ops files found in the src/ops/ directory.

That may sound complex, but it really amounts to only a half of one percent of all the files and directories in the Parrot source tree. So of all the files in my buildtools branch, 0.5% are there so that they don't hurt anything in trunk, while 95.5% might just as well be identical to the files in trunk -- which means that I would like to keep them in synch with trunk as much as possible. (After all, a change in one of those files might affect the outcome of the tests.)

After reviewing the Subversion book, the only way I've found to accomplish this is this:

  1. In my branch, call svn update to determine the version (let's call the revision n).
  2. Call svn merge -r 16803:n  https://svn.perl.org/parrot/trunk, where 16803 is the revision at which my branch was created.
  3. Ideally, at this point I call svn commit and everything updates nicely. But in practice, I get all kinds of conflicts which have little immediate significance but which must be resolved for a commit to succeed. Examples: changes in file properties; conflicts in the MANIFEST. In other words, things that waste my time and don't make development in my branch any easier.
In other words, I want to keep my branch reasonably up-to-date so that I can be assured that my own files will eventually commit to trunk correctly -- but I don't want to spend 15 minutes a night resolving spurious conflicts in the branch. How can I do this most effectively?

Thank you very much.

Jim Keenan

Replies are listed 'Best First'.
Re: [OT] Best Use of Subversion Branches in Perl Development
by perrin (Chancellor) on Feb 02, 2007 at 05:55 UTC

    First, remember that "trunk" is just another branch in Subversion, and it has no significance whatsoever, so admonitions to be on a branch rather than on "trunk" are meaningless. What they're trying to say is, "Don't do your development work in the stable branch." I think that projects nearly always need one stable branch and one development branch, unless there is no stable release yet.

    If the rest of the Parrot developers work in the trunk, there is no reason for you to work on a separate branch unless you are trying to make a large change which will be incompatible with other people's work if you commit it before it's all complete. It doesn't sound like that's the case. I think you could work in trunk.

    If you do want to keep working in your branch, I think you should change your merge methodology a bit. For one thing, you don't want to merge from the point where your branch was created; you want to merge from the point where you last merged. You also should use tags, rather than trying to keep track of revision numbers or look them up in logs.

    Here's a sample workflow:

    • Tag the trunk that you are about to merge into your branch:
      svn cp https://svn.perl.org/parrot/trunk https://svn.perl.org/parrot/t +ags/tools-merge-16
    • Merge everything from your last tag until this tag onto your branch:
      svn merge https://svn.perl.org/parrot/tags/tools-merge-15 https://svn. +perl.org/parrot/tags/tools-merge-16

    There may still be some conflicts, but if you ever get conflicts on files that you personally have not modified, you should be worried. Conflicts are only supposed to happen when you and someone else touch the same lines or properties.

      First, remember that "trunk" is just another branch in Subversion, and it has no significance whatsoever, so admonitions to be on a branch rather than on "trunk" are meaningless. What they're trying to say is, "Don't do your development work in the stable branch."

      So, so true ... the most general way i can think of to express the sentiment is: "commit to a branch that is no more stable then the changes you are commiting"

      • if the "head" is an extremely stable line of code, only commit to it once you are confident your changes are fairly stable and won't need any more tweaks
      • if the "head" is where lots of iterative development is going on, then commit to it early and often
      Thanks, everyone, for your rapid (as always) responses. Perrin, I had not previously had a reason to learn about tags. So I'll study up on them and see about implementing your suggestion.

      Thanks again.

      Jim Keenan
        Tags, branches, and the trunk are all the same to subversion; they are simply copies. To tag in subversion is to perform the same operation as creating a branch: svn copy. The difference between a tag and a branch is a semantic one. Branches are used for ongoing development. Tags are considered static snapshots.

        What is being suggested is that you create periodic snapshots of the trunk -- that way you don't have to remember the revision number of the previous time you merged in changes from the trunk; you simply remember the tag names.

      I've only been using svn @ $work and never really got into it, so thanks for the extensive explanation. I've read all the thread in detail, and followed the links provided. Many interesting ideas here! Hopefully, I'm learning something.

      Regarding the specific issue being discussed here, since versioning control systems keep track of what's getting modified anyway, it seems strange that there's not a "selective" cp command that will copy one branch over another without overwriting files that have been modified in the destination one. If there were, then the OP would issue it before running the tests, resolve problems, and then merge.

        That wouldn't really be enough. You have to get the changes to the files you have modified as well. The svn merge command is pretty good at this though.
    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: [OT] Best Use of Subversion Branches in Perl Development
by rinceWind (Monsignor) on Feb 02, 2007 at 09:54 UTC

    You might want to look at SVK, which, although you are not needing to use it offline, does take most of the pain out of maintaining your work in a branch; just use svk pull and svk push.

    This can also be completely transparent to the rest of your developers, who are working directly with svn.

    --

    Oh Lord, won’t you burn me a Knoppix CD ?
    My friends all rate Windows, I must disagree.
    Your powers of persuasion will set them all free,
    So oh Lord, won’t you burn me a Knoppix CD ?
    (Missquoting Janis Joplin)

      I have to second this, as SVK takes care of a lot of merge tracking for you. It does requires some different thinking as it replicates a repository to your computer and then manages the synchronization from there. See the SVK Wiki for more details and some guides.

      In SVK, people seem to usually make a local branch on their computer rather than a branch in the repository and work there. It might work like this:

      I think you can also do this entirely on a remote branch, too, like this (though there are other ways):

      -xdg

      Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

      This can also be completely transparent to the rest of your developers, who are working directly with svn.

      Indeed. Another vote for SVK here. It's become my default subversion "client" for some time and makes the private sandbox way of working very simple. Nice.

Re: [OT] Best Use of Subversion Branches in Perl Development
by merlyn (Sage) on Feb 02, 2007 at 02:54 UTC
    If you were using git, this would be far simpler. You could keep merging the latest changes into your local repo, and when you were ready to publish, you could rebase all your changes on the most recent published head, so all your changes would be published as one commit. Too easy.

    Too bad more Perl projects aren't using git, but apparently the lack of native windows support is a showstopper for some (although it apparently works fine under cygwin).

Re: [OT] Best Use of Subversion Branches in Perl Development
by runrig (Abbot) on Feb 02, 2007 at 17:17 UTC
    You may want to look at svnmerge (which ships with Subversion). It makes it work similar to the way git does (that merlyn mentions above). There's a good tutorial on it.

      You know what, I checked it out and it doesn't even come close. It's merging is extremely deficient compared with even svk and merges were complicated and painful. However, it does support tracking cherry picking which is more than svk.

      $h=$ENV{HOME};my@q=split/\n\n/,`cat $h/.quotes`;$s="$h/." ."signature";$t=`cat $s`;print$t,"\n",$q[rand($#q)],"\n";
Still waiting for Merge Tracking
by danb (Friar) on Feb 02, 2007 at 17:07 UTC
    To make the branch-develop-merge workflow easier, Subversion needs Merge Tracking. If you can't wait for them to finish the feature, I suggest you try one of the altnernatives.

    --Dan

Re: [OT] Best Use of Subversion Branches in Perl Development
by blackmateria (Chaplain) on Feb 04, 2007 at 04:29 UTC

    0.5% are there so that they don't hurt anything in trunk, while 95.5% might just as well be identical to the files in trunk—which means that I would like to keep them in synch with trunk as much as possible.

    I don't know if anyone has mentioned this yet, but the svn switch command lets you switch individual files (i.e., it is not limited to just directories). Depending on your circumstances, you could just switch the 0.5% of files in your working copy to the branch, and leave the rest on trunk.

    (The downside is, I think you can only switch files that already exist on trunk, so if you have added files, you'll at least have to switch their parent directories.)

    Also, I think the rule "always work on a branch" actually means "don't break the trunk." You have to balance your loss of productivity from the repeated need to hand-merge trunk changes, against the risk that you will make it impossible for the other Parrot developers to continue working by committing broken code. Usually I only work on a branch when I am planning on breaking something; otherwise, I just test really really carefully before checking in my work directly to trunk.

Re: [OT] Best Use of Subversion Branches in Perl Development
by Moron (Curate) on Feb 05, 2007 at 12:48 UTC
    I might be missing something, but before trying to duplicate the 99% of identical files, are you sure you are using environment variables such as PATH and PERL5LIB environment variables to full advantage, i.e. to define the runtime priorities for multiple directories?

    -M

    Free your mind

      Yes - I think you are missing something :-)

      The idea of having a developer branch (or sandbox or whatever - depending on your particular version control metaphor of choice) is to have a copy of everything so you can tweak to your heart's content rather without the risk of messing somebody else's work up.

      In any decent system branches are cheap since they only track changes to the files you tweak rather than duplicating them.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://597867]
Approved by GrandFather
Front-paged by grinder
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2024-03-19 08:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found