Add GitAnnexBuilder, using git-annex to store blobs

This is an extension of NoTgzBuilder, which instead of storing blobs in source
control directly, allows the use of git-annex to reference blobs in an external
source (e.g. a web URL remote).  When preparing sources, it uses git-annex's
unlock command to temporarily restore the original file.

doc/builders.mkd

@@ -21,3 +21,48 @@ Once configured a build can be run with:
 Note that this does not require a tito tag to have been run, and technically the rel-eng/tito.props does not have to be committed to the git repo. This builder can be used on a repository you do not control or do not wish to push tito configuration into.
 
 The ArgSourceStrategy has a simple mechanism where it will try to parse the version and release to build from the filename with a regular expression. If you need something more advanced, you can override any or all of this behaviour by implementing a custom strategy for the fetch builder. (see lib_dir in **man 5 tito.props**)
+
+## tito.builder.GitAnnexBuilder
+
+A builder for packages with existing tarballs checked in using git-annex, e.g. referencing an external source (web remote) or special remotes used in the same way as a lookaside cache.
+
+This builder will "unlock" the source files to get the real contents, include them in the SRPM, then restore the automatic git-annex symlinks on completion.
+
+To create a new git repository using git-annex and tito init, run:
+
+    git init
+    git annex init
+    tito init
+
+Add the spec file as normal, but add the tarball/source via git-annex instead:
+
+    cp ~/tito.spec ~/tito-0.4.18.tar.gz .
+    git add tito.spec
+    git annex add tito-0.4.18.tar.gz
+    git commit -m "add tito 0.4.18"
+
+Edit rel-eng/tito.props to change the builder to the GitAnnexBuilder:
+
+    [globalconfig]
+    default_builder = tito.builder.GitAnnexBuilder
+
+Then finally tag and build as normal:
+
+    tito tag --keep-version
+    tito build --rpm
+
+Remotes and special remotes can be used now with git-annex to transfer file content elsewhere.  Special remotes can be used to sync bulky files to things other than git repositories, e.g. shared directories, S3, WebDAV etc.
+
+    git annex initremote usbdrive type=directory directory=/media/usb encryption=none
+    git annex copy --to=usbdrive
+
+Then files can be dropped or fetched again into the local repository, which acts as a local cache.  tito's GitAnnexBuilder will automatically "get" files during the build process.
+
+    git annex drop
+    git annex get
+
+Files can be added with the "web" remote, useful for RPM sources.  The downside is that like with other remotes, this configuration needs to be added to each git checkout but is also on a file-by-file basis, so a script may be required.
+
+    git annex addurl --file tito-0.4.18.tar.gz http://pkgs.fedoraproject.org/repo/pkgs/tito/tito-0.4.18.tar.gz/318e6f546c9b331317c94ea5d178876a/tito-0.4.18.tar.gz
+
+More information in the [git-annex walkthrough](http://git-annex.branchable.com/walkthrough/) and [special remotes](http://git-annex.branchable.com/special_remotes/) documentation.

src/tito/builder/__init__.py

@@ -8,6 +8,7 @@ from tito.builder.main import \
         UpstreamBuilder, \
         SatelliteBuilder, \
         MockBuilder, \
-        BrewDownloadBuilder
+        BrewDownloadBuilder, \
+        GitAnnexBuilder
 
 from tito.builder.fetch import FetchBuilder

src/tito/builder/main.py

@@ -20,6 +20,7 @@ import os
 import sys
 import re
 import commands
+import shutil
 from pkg_resources import require
 from distutils.version import LooseVersion as loose_version
 from tempfile import mkdtemp
@@ -1297,5 +1298,38 @@ class ExternalSourceBuilder(ConfigObject, BuilderBase):
         return version
 
 
+class GitAnnexBuilder(NoTgzBuilder):
+    """
+    Builder for packages with existing tarballs checked in using git-annex,
+    e.g. referencing an external source (web remote).  This builder will
+    "unlock" the source files to get the real contents, include them in the
+    SRPM, then restore the automatic git-annex symlinks on completion.
+    """
 
+    def _setup_sources(self):
+        super(GitAnnexBuilder, self)._setup_sources()
 
+        old_cwd = os.getcwd()
+        os.chdir(os.path.join(old_cwd, self.relative_project_dir))
+
+        (status, output) = commands.getstatusoutput("which git-annex")
+        if status != 0:
+            msg = "Please run 'yum install git-annex' as root."
+            error_out('%s' % msg)
+
+        run_command("git-annex lock")
+        annexed_files = run_command("git-annex find --include='*'").splitlines()
+        run_command("git-annex get")
+        run_command("git-annex unlock")
+        debug("  Annex files: %s" % annexed_files)
+
+        for annex in annexed_files:
+            debug("Copying unlocked file %s" % annex)
+            os.remove(os.path.join(self.rpmbuild_gitcopy, annex))
+            shutil.copy(annex, self.rpmbuild_gitcopy)
+
+        os.chdir(old_cwd)
+
+    def cleanup(self):
+        run_command("git-annex lock --force")
+        super(GitAnnexBuilder, self).cleanup()

tito.props.5.asciidoc

@@ -193,6 +193,14 @@ fetch_strategy = tito.builder.fetch.ArgSourceStrategy
 ArgSourceStrategy here could be replaced with a custom strategy if you were to
 have one in your lib_dir.
 
+tito.builder.GitAnnexBuilder::
+See doc/builders.mkd.
+
+Builder for packages with existing tarballs checked in using git-annex, e.g.
+referencing an external source (web remote).  This builder will "unlock" the
+source files to get the real contents, include them in the SRPM, then restore
+the automatic git-annex symlinks on completion.
+
 TAGGERS
 -------
 All taggers which inherit fom tito.tagger.VersionTagger (all to this date),