As is especially the case when developing software, the data that you maintain under version control is often closely related to, or perhaps dependent upon, someone else's data. Generally, the needs of your project will dictate that you stay as up to date as possible with the data provided by that external entity without sacrificing the stability of your own project. This scenario plays itself out all the time—anywhere that the information generated by one group of people has a direct effect on that which is generated by another group.
For example, software developers might be working on an application that makes use of a third-party library. Subversion has just such a relationship with the Apache Portable Runtime (APR) library (see the section called “The Apache Portable Runtime Library”). The Subversion source code depends on the APR library for all its portability needs. In earlier stages of Subversion's development, the project closely tracked APR's changing API, always sticking to the “bleeding edge” of the library's code churn. Now that both APR and Subversion have matured, Subversion attempts to synchronize with APR's library API only at well-tested, stable release points.
Now, if your project depends on someone else's information, you could attempt to synchronize that information with your own in several ways. Most painfully, you could issue oral or written instructions to all the contributors of your project, telling them to make sure they have the specific versions of that third-party information that your project needs. If the third-party information is maintained in a Subversion repository, you could also use Subversion's externals definitions to effectively “pin down” specific versions of that information to some location in your own working copy (see the section called “Externals Definitions”).
But sometimes you want to maintain custom modifications to third-party code in your own version control system. Returning to the software development example, programmers might need to make modifications to that third-party library for their own purposes. These modifications might include new functionality or bug fixes, maintained internally only until they become part of an official release of the third-party library. Or the changes might never be relayed back to the library maintainers, existing solely as custom tweaks to make the library further suit the needs of the software developers.
Now you face an interesting situation. Your project could house its custom modifications to the third-party data in some disjointed fashion, such as using patch files or full-fledged alternative versions of files and directories. But these quickly become maintenance headaches, requiring some mechanism by which to apply your custom changes to the third-party code and necessitating regeneration of those changes with each successive version of the third-party code that you track.
The solution to this problem is to use vendor branches. A vendor branch is a directory tree in your own version control system that contains information provided by a third-party entity, or vendor. Each version of the vendor's data that you decide to absorb into your project is called a vendor drop.
Vendor branches provide two benefits. First, by storing the currently supported vendor drop in your own version control system, you ensure that the members of your project never need to question whether they have the right version of the vendor's data. They simply receive that correct version as part of their regular working copy updates. Second, because the data lives in your own Subversion repository, you can store your custom changes to it in-place—you have no more need of an automated (or worse, manual) method for swapping in your customizations.
Unfortunately, there is no single best way to manage vendor branches in Subversion. The flexibility of the system offers several different approaches, each of which has its advantages and disadvantages, and none of which can be clearly considered a “silver bullet” for the problem. We'll cover a few of these approaches at a high level in the following sections, using the common example of a software project which depends on a third-party library.
Maintaining customizations to a third-party library involves three data sources: the version of the third-party library upon which your modifications were last based, the customized version (that is, the actual vendor branch) of that library which is used by your project, and any new version of the vendor's library to which you may be hoping to upgrade. Managing the vendor branch (which should live within your source code repository per our definition of the thing), then, essentially boils down to performing merge operations (in the general sense). But different teams take different approaches to the other data sources—the pristine versions of the third-party library code. Thus, there are likewise different specific ways to perform the requisite merges.
Strictly speaking, there are a couple of different ways that those merges can be performed in the general sense. For the sake of simplicity and with the goal of at least providing something concrete in this section of the book, we'll assume that there is but a single vendor branch which is upgraded to each successive new release of the third-party library by receiving updates that describe the differences between the current and new pristine versions of that library.
![[Note]](images/note.png)
|
Note |
|---|---|
|
Another approach is to create new vendor branches for each successive pristine library version, applying the differences between the current pristine library and the customized version thereof (from the current vendor branch) to the new branch. There's nothing wrong with that approach—we just don't feel compelled to document every legitimate possibility in this space. |
The following sections examine how to create and manage a
vendor branch in a few different scenarios. In the examples
which follow, we'll assume that the third-party library is
called libcomplex, and that we will be implementing a vendor
branch based on libcomplex 1.0.0 which lives in our repository
at ^/vendor/libcomplex-custom. We'll
then look at how we can upgrade to libcomplex 1.0.1 while
still preserving our customizations to the library.
Let's look first at a vendor branch management approach that is possible when the original third-party library is itself Subversion-accessible. For the sake of the example, we'll assume that the libcomplex library we previously discussed is developed in a publicly accessible Subversion repository, and that its developers use sane release procedures which include the creation of tags for each stable release version.
Since Subversion 1.5, svn merge has been able to perform so-called foreign repository merges, where the sources of the merge live in a different repository than the repository from which the merge target working copy was checked out. And in Subversion 1.8, the behavior of svn copy was changed so that when you perform a copy from a foreign repository into an existing working copy, the resulting tree is incorporated into that working copy and scheduled for addition. It's this foreign repository copy functionality that we'll use to bootstrap our vendor branch.
So let's create our vendor branch. We'll begin by creating a placeholder directory for all such vendor branches in our repository, and then checking out a working copy of that location.
$ svn mkdir http://svn.example.com/projects/vendor \
-m "Create a container for vendor branches."
Committed revision 1160.
$ svn checkout http://svn.example.com/projects/vendor \
/path/to/vendor
Checked out revision 1160.
$
Now, we'll take advantage of Subversion's foreign repository copy support to get an exact copy of libcomplex 1.0.0—including any Subversion properties stored on its files and directories—from the vendor repository.
$ cd /path/to/vendor
$ svn copy http://svn.othervendor.com/repos/libcomplex/tags/1.0.0 \
libcomplex-custom
--- Copying from foreign repository URL 'http://svn.othervendor.com/repos/lib\
complex/tags/1.0.0':
A libcomplex-custom
A libcomplex-custom/README
A libcomplex-custom/LICENSE
…
A libcomplex-custom/src/code.c
A libcomplex-custom/tests
A libcomplex-custom/tests/TODO
$ svn commit -m "Initialize libcomplex vendor branch from libcomplex 1.0.0."
Adding libcomplex-custom
Adding libcomplex-custom/README
Adding libcomplex-custom/LICENSE
…
Adding libcomplex-custom/src
Adding libcomplex-custom/src/code.h
Adding libcomplex-custom/src/code.c
Transmitting file data .......................................
Committed revision 1161.
$
|
|
Note |
|---|---|
|
If you happen to be using an older version of
Subversion, the closest available approximation of the new
foreign repository copy support in svn
copy is to instead import (via svn
import) a working copy of the vendor's tag,
including the |
Now that we have a vendor branch based on libcomplex 1.0.0, we can begin making the customizations to libcomplex required for our purposes, committing them directly to the vendor branch we've created. And of course, we can begin using libcomplex in our own application.
Some time later, libcomplex 1.0.1 is released. After reviewing its changes, we decide we'd like to upgrade our vendor branch to the new version. Here is where Subversion's foreign repository merge operation is useful. We have in our vendor branch the original libcomplex 1.0.0 plus our customizations to it. What we need now is to get the set of changes the vendor has made between 1.0.0 and 1.0.1 into our vendor branch, ideally without clobbering our own customizations. This is precisely what the 2-URL form of the svn merge command is for.
$ cd /path/to/vendor
$ svn merge http://svn.othervendor.com/repos/libcomplex/tags/1.0.0 \
http://svn.othervendor.com/repos/libcomplex/tags/1.0.1 \
libcomplex-custom
--- Merging differences between foreign repository URLs into '.':
U libcomplex-custom/src/code.h
C libcomplex-custom/src/code.c
U libcomplex-custom/README
Summary of conflicts:
Text conflicts: 1
Conflict discovered in file 'libcomplex-custom/src/code.c'.
Select: (p) postpone, (df) diff-full, (e) edit, (m) merge,
(mc) mine-conflict, (tc) theirs-conflict, (s) show all options:
As you can see, svn merge has merged the changes required to make libcomplex 1.0.0 look like libcomplex 1.0.1 into our working copy. In our example, it has even noticed and flagged a conflict on one file. It seems the vendor modified a region of one of the files we also customized. Subversion safely detects this conflict, and gives us the opportunity to resolve it so that our customizations to what is now libcomplex 1.0.1 continue to make sense. (See the section called “Resolve Any Conflicts” for more on resolving conflicts of this sort.)
Once we've resolved the conflicts and performed any testing or review we need, we can commit the changes to our vendor branch.
$ svn status libcomplex-custom
M libcomplex-custom/src/code.h
M libcomplex-custom/src/code.c
M libcomplex-custom/README
$ svn commit -m "Upgrade vendor branch to libcomplex 1.0.1." \
libcomplex-custom
Sending libcomplex-custom/README
Sending libcomplex-custom/src/code.h
Sending libcomplex-custom/src/code.c
Transmitting file data ...
Committed revision 1282.
$
That, in a nutshell, is how to manage vendor branches when the original source is Subversion-accessible. There are some notable shortcomings, though. First, foreign repository merges are not automatically tracked by Subversion itself like same-repository merges are. This means the burden falls to the user to know which merges have been performed on their vendor branch, and just how to construct the next merge when upgrading that branch. Also, as is the case for all of Subversion's merge support, renames in the merge sources can cause no small amount of complication and frustration. Unfortunately, at this time, we don't have a particularly solid recommendation to offer to alleviate that pain.
In the previous section (the section called “Vendor Branches from Foreign Repositories”) we looked at how to implement and maintain a vendor branch when the vendor drops are accessible via Subversion, which is the ideal scenario when it comes to vendor branches. Subversion is pretty good at handling merges of stuff that's been Subversion-managed. Unfortunately, it's not always the case that third-party libraries are publicly accessible via Subversion. Many times, a project depends on a library which is delivered via only non-Subversion mechanisms, such as a source code release distribution tarball. In such circumstances, we strongly recommend that you do all you can to get that non-Subversion information into Subversion as cleanly as possible. So let's examine an approach to vendor branches in which the third-party library's various releases are mirrored within our own repository.
Setting up the vendor branch the first time is pretty simple, really. For our example, we'll assume that libcomplex 1.0.0 is delivered via the common tarball mechanism. To create our vendor branch, we'll first get the contents of the libcomplex 1.0.0 tarball into our repository as a read-only (by convention only) vendor tag of sorts.
$ tar xvfz libcomplex-1.0.0.tar.gz
libcomplex-1.0.0/
libcomplex-1.0.0/README
libcomplex-1.0.0/LICENSE
…
libcomplex-1.0.0/src/code.c
libcomplex-1.0.0/tests
libcomplex-1.0.0/tests/TODO
$ svn import libcomplex-1.0.0 \
http://svn.example.com/projects/vendor/libcomplex-1.0.0 \
--no-ignore --no-auto-props \
-m "Import libcomplex 1.0.0 sources."
Adding libcomplex-custom
Adding libcomplex-custom/README
Adding libcomplex-custom/LICENSE
…
Adding libcomplex-custom/src
Adding libcomplex-custom/src/code.h
Adding libcomplex-custom/src/code.c
Transmitting file data .......................................
Committed revision 1160.
$
Note that in our example, we used
the --no-ignore option during import so that
Subversion is sure to pick up every file in the vendor drop
and not to omit any of them. We also supply
the --no-auto-props option so that our client
doesn't manufacture property information which isn't present
in the vendor drop.[44].
Now that the first vendor release drop is present in our repository, we can create our vendor branch from it just as we would create any other branch—using svn copy.
$ svn copy http://svn.example.com/projects/vendor/libcomplex-1.0.0 \
http://svn.example.com/projects/vendor/libcomplex-custom \
-m "Initialize libcomplex vendor branch from libcomplex 1.0.0."
Committed revision 1161.
$
Okay. At this point we have a vendor branch based on libcomplex 1.0.0. We are now poised to begin making the customizations to libcomplex required for our purposes—committing them directly to the vendor branch we've created—and then to start using our customized libcomplex in our own application.
Some time later, libcomplex 1.0.1 is released. After reviewing its changes, we decide we'd like to upgrade our vendor branch to the new version. In order to perform that upgrade on our branch, we need to essentially apply the same set of changes the vendor has made between 1.0.0 and 1.0.1 to our vendor branch without clobbering our own customizations. The safest way to perform that application is to first get libcomplex 1.0.1 into our repository as a delta against the libcomplex 1.0.0 code in our repository. Afterwards, we'll use the 2-URL form of the svn merge command to replicate those same changes into our vendor branch.
As it turns out, there are several different approaches we can take to to get libcomplex 1.0.1 into our repository in the right way.[45] The approach we'll describe here is relatively rudimentary, but it will serve our illustrative needs.
Remember, we want our mirror of the libcomplex 1.0.1 vendor drop to share ancestry with our 1.0.0 vendor drop, which will produce the best results later when we need to merge the changes between those drops to our vendor branch. So we'll start by creating a libcomplex-1.0.1 branch as copy of our previously created libcomplex-1.0.0 “vendor tag”—a copy which will eventually become a replica of libcomplex 1.0.1.
$ svn copy http://svn.example.com/projects/vendor/libcomplex-1.0.0 \
http://svn.example.com/projects/vendor/libcomplex-1.0.1 \
-m "Setup a construction zone for libcomplex 1.0.1."
Committed revision 1282.
$
What we need now is to make a working copy of our
libcomplex-1.0.1 branch, and then to make it actually look
like libcomplex 1.0.1. To do this, we'll take advantage of
the fact that svn checkout can overlay an
existing directory and, if the --force option
is provided, do so in manner that allows the differences
between the checked-out tree and the target tree that the
checkout overlayed to remain as local modifications in the new
working copy.
$ tar xvfz libcomplex-1.0.1.tar.gz
libcomplex-1.0.1/
libcomplex-1.0.1/README
libcomplex-1.0.1/LICENSE
…
libcomplex-1.0.1/src/code.c
libcomplex-1.0.1/tests
libcomplex-1.0.1/tests/TODO
$ svn checkout http://svn.example.com/projects/vendor/libcomplex-1.0.1 \
libcomplex-1.0.1 \
--force
E libcomplex-1.0.1/README
E libcomplex-1.0.1/LICENSE
E libcomplex-1.0.1/INSTALL
…
E libcomplex-1.0.1/src/code.c
E libcomplex-1.0.1/tests
E libcomplex-1.0.1/tests/TODO
Checked out revision 1282.
$ svn status libcomplex-1.0.1
M libcomplex-1.0.1/src/code.h
M libcomplex-1.0.1/src/code.c
M libcomplex-1.0.1/README
$
As you can see, after checking out what was really libcomplex 1.0.0 atop the libcomplex 1.0.1 exploded tarball, we are left with a working copy that contains local modifications—those modifications required to morph our previous vendor release drop into our new one.
Admittedly, this is a pretty simple example. The changes required to perform this particular upgrade involved merely content changes to existing files. In reality, new versions of third-party libraries might also add or remove files or directories, might rename files or directories, and so on. In those situations, it can be much more challenging to morph the new vendor tag into a state where it accurately reflects the vendor drop it claims to reflect. We'll leave the details of such transformations as an exercise to the reader.[46]
However we make it happen, once our new vendor tag working copy is reconciled with the original source distribution, we can commit those changes to our repository.
$ svn commit -m "Upgrade vendor branch to libcomplex 1.0.1." \
libcomplex-1.0.1
Sending libcomplex-1.0.1/README
Sending libcomplex-1.0.1/src/code.h
Sending libcomplex-1.0.1/src/code.c
Transmitting file data ...
Committed revision 1283.
$
We're finally ready to upgrade our vendor branch. Remember, our goal is to get the changes made by the vendor between the 1.0.0 and 1.0.1 releases of their library into our vendor branch. There is where a 2-URL svn merge operation, applied to a working copy of our vendor branch, comes into play.
$ svn checkout http://svn.example.com/projects/vendor/libcomplex-custom \
libcomplex-custom
E libcomplex-custom/README
E libcomplex-custom/LICENSE
E libcomplex-custom/INSTALL
…
E libcomplex-custom/src/code.c
E libcomplex-custom/tests
E libcomplex-custom/tests/TODO
Checked out revision 1283.
$ cd libcomplex-custom
$ svn merge ^/vendor/libcomplex-1.0.0 \
^/vendor/libcomplex-1.0.1
--- Merging differences between repository URLs into '.':
U src/code.h
C src/code.c
U README
Summary of conflicts:
Text conflicts: 1
Conflict discovered in file 'src/code.c'.
Select: (p) postpone, (df) diff-full, (e) edit, (m) merge,
(mc) mine-conflict, (tc) theirs-conflict, (s) show all options:
As you can see, svn merge has merged the requisite changes into our working copy, flagging a conflict where the vendor modified the same region of one of the files as we did during our customizations. Subversion detects this conflict, and gives us the opportunity to resolve it (using the methods described in the section called “Resolve Any Conflicts”) so that our customizations to what is now libcomplex 1.0.1 continue to make sense. Once we've resolved the conflicts and performed any testing or review we need, we can commit the changes to our vendor branch.
$ svn status M src/code.h M src/code.c M README $ svn commit -m "Upgrade vendor branch to libcomplex 1.0.1." Sending README Sending src/code.h Sending src/code.c Transmitting file data ... Committed revision 1284. $
Our vendor branch upgrade is complete. And the next time we need to upgrade that branch, we'll follow the same procedure we used to upgrade it this time.
[44] Technically, we could let the auto-props feature do its thing, but the key to making that work well is ensuring that each vendor drop gets identical auto-prop treatment.
[45] Using another svn import operation would be an incorrect approach, as the libcomplex 1.0.0 and 1.0.1 branches would not have any common ancestry.
[46] Here's a hint, though: svn
add --force /path/to/working-copy --no-ignore
--no-auto-props is super handy for adding any new
vendor drop items to version control in this
situation.
