diff --git a/README.mkd b/README.md similarity index 100% rename from README.mkd rename to README.md diff --git a/doc/builders.mkd b/doc/builders.md similarity index 100% rename from doc/builders.mkd rename to doc/builders.md diff --git a/doc/index.md b/doc/index.md new file mode 100644 index 0000000..91f2a52 --- /dev/null +++ b/doc/index.md @@ -0,0 +1,230 @@ +## About +Tito is a tool for managing RPM based projects using git for their source code +repository. + +Tito offers the following features: + + - Tag new releases with incremented RPM version or release. + - Auto-generate spec file changelog based on git history since last tag. + - Create reliable tar.gz's with consistent checksums from any tag. + - Build source and binary rpms off any tag. + - Build source and binary "test" rpms off most recently committed code. + - Build multiple source rpms with appropriate disttag's for submission to the + Koji build system + - Build rpms via the "mock" tool. + - On a per-branch basis in git: + - Maintain concurrent version streams. + - Vary the way packages are built/tagged. + - Report on any diffs or commits messages missing since last tag. + - Define release targets to publish your packages to yum repositories, or + the Fedora build system. + - Define custom builder/releaser implementations for your own project needs. + - Build packages off an "upstream" git repository, where modifications in the + "downstream" git repository will be applied as a patch in the source rpm. + - Manage all of the above for a git repository with many disjoint packages + within it. + +## Related Projects +* `mockchain` from the [mock project](http://fedoraproject.org/wiki/Projects/Mock) +* `mock`'s built-in SCM support in `mock-scm` +* Fedora's [Koji](http://koji.fedoraproject.org/koji/) build engine and [fedpkg](https://fedorahosted.org/fedpkg/) tools +* The [OpenSUSE Build Service](https://build.opensuse.org/). + +## Install +To install from source + +``` +$ git clone https://github.com/dgoodwin/tito.git +$ cd tito/ +$ sudo yum install python-setuptools +$ ./setup.py build +$ sudo ./setup.py install +``` + +To make an rpm of tito to install elsewhere + +``` +$ sudo yum install python-devel asciidoc +$ tito build --rpm +# see what's in the package +$ rpm -ql -p /tmp/tito/noarch/tito-*.noarch.rpm +``` + +## Getting Started +From your git repository: + +``` +$ tito init +``` + +This will create a top-level metadata directory called `.tito/` and commit it +to git. This directory will store tito's configuration and package metadata on +a per branch basis. It will be filtered out when creating .tar.gz's. + +## Tagging Packages +Before doing most everything you'll need to tag your package(s). + +First, ensure that your package spec files are at the top of the relative source tree for that package. + +The most common case, a single project git repository, has the spec file +and root of the project at the top level of the git repository: + +``` +docs/ +mypackage.spec +README +.tito/ +src/ +test/ +``` + +For a multi-project git repository, packages can be defined in various +sub-directories, provided they do not nest (i.e. walking up the tree, two spec +files will never be encountered): + +``` +.tito/ + package1/ + docs/ + mypackage.spec + README + src/ + test/ +subdir/ + package2/ + anotherpkg.spec + docs/ + README + src/ + test/ +``` + +The packages can be organized in any hierarchy you like and even be moved +around and re-tagged, we only need to have the spec file in the top level +directory for that package. + +Tagging packages is normally done with: + +``` +$ tito tag +``` + +This will: + + - bump the version or release in the spec file (use --keep-version to use whatever is defined in the spec file) + - auto-generate a changelog from first line of each commit since last tag (use --no-auto-changelog if you do not want this) + - open an editor allowing you a chance to edit that changelog + - insert the changelog into your spec + - commit these changes, and generate a git tag + +By default if you omit --keep-version, tito will tag by bumping the rpm +version. (i.e. we bump the Z in X.Y.Z. If you'd prefer to bump the package +release instead (normally should just be used for changes to the spec file or +patches applied within it), you can change the 'tagger' class in +.tito/tito.props to ReleaseTagger. This will affect all packages in this git +branch, if you'd prefer to do this on a per-package basis you can do so in a +package specific tito.props. (see section below) + +Once a package is tagged you will need to push both the auto-commit and the +tag to your remote git repository before tito will let you build it. (better +support for standalone git repositories is coming, for now --offline will +help) + +See `man tito` for more options. + +## Building Packages +To build the most recent .tar.gz for a package, cd into that packages +directory and run: + +``` +$ tito build --tgz +``` + +Note that this tarball will have a consistent checksum every time. + +Likewise the `--srpm` and `--rpm` options allow you to build both binary and +source rpms. + +Add in the `--tag=TAG` option to build any of the above for any past tag. + +If you're working on something locally and would like to check that your +package is still building correctly without pushing your changes to the remote +repository, add the `--test` option. This will build a test rpm from your most +recently committed work. (**NOTE: does *not* include uncommitted changes**) + +TODO: Document the use of --release, which is complicated and untested against +Fedora's Koji. + +See `man tito` for more options. + +## Releasing Packages +Tito supports a mechanism where you can define multiple release targets. + +In `.tito/releasers.conf`, create a section like: + +``` +[yum-f15-x86_64] +releaser = tito.release.YumRepoReleaser +builder = tito.builder.MockBuilder +builder.mock = fedora-15-x86_64 +rsync = fedorapeople.org:/srv/repos/dgoodwin/tito/fedora-15/x86_64/ +``` + +You can define as many release targets as you like with various +configurations. To publish the most recently tagged build in your current +branch you would run: + +``` +$ tito release yum-f15-x86_64 +``` + +You can specify multiple targets on the CLI. + +See `man 8 releasers.conf` for more information on defining release targets. + +See `man tito` for more information on CLI arguments to `tito release`. + +## Custom Builders / Taggers / Releasers +If the existing implementations Tito provides are not sufficient for +your needs, it is possible to define a lib_dir in tito.props globalconfig +section. This is a directory that tito will add to the python path during +execution, allowing you a place to define your own custom implementations of +builders, taggers, and releasers. + +The process of actually writing a custom Builder/Tagger/Releaser is an +exercise left to the reader, but essentially you will want to work off the +code in the tito.builder module. Inherit from the base Builder, and override +the methods you need to. + +Please note that if you store your custom implementations inside your source +tree, they will need to be kept in sync in all branches you are using for +consistent behavior. Also, there are no guarantees that tito will not change +in future releases, meaning that your custom implementations may occasionally +need to be updated. + +## Troubleshooting +If you create a tag accidentally or that you wish to re-do, make sure you have +not git pushed the tag yet, the auto-commit is the most recent in your git +history, and run: + +``` +git tag -d YOURNEWTAG +git reset --hard HEAD^1 +``` + +If your project is standalone (no remote reference you communicate with as +authoritative) you may wish to set `offline = "true"` in `.tito/tito.props` +under the globalconfig section, so you do not need to specify `--offline` +with each invocation. + +## Configuration +See: + +* `man 5 tito.props` +* `man 5 releasers.conf` +* `man 5 titorc` + +## External Docs +* [Tito release announcements](http://rm-rf.ca/blogs/dgoodwin) +* [How to create new release of RPM package in 5 seconds](http://miroslav.suchy.cz/blog/archives/2013/12/17/how_to_create_new_release_of_rpm_package_in_5_seconds) +* [How to build in Copr](http://miroslav.suchy.cz/blog/archives/2013/12/29/how_to_build_in_copr) diff --git a/doc/mead.md b/doc/mead.md new file mode 100644 index 0000000..78883c7 --- /dev/null +++ b/doc/mead.md @@ -0,0 +1,173 @@ +## Introduction +Mead is a Maven-based build system for Koji. It works by building your +project via Maven, rendering a RPM spec file from a template, and then +packaging everything as an RPM with the rendered spec file. + +Tito has support for Mead in its tagging, building, and releasing capacities. + +## Setup +For a Maven project, Tito needs two things: a `pom.xml` and a `.tmpl` file. +The POM file should already be present since Maven itself requires one. The +`.tmpl` file is a [Cheetah](http://www.cheetahtemplate.org/) template of your +RPM spec file that Mead fills in when it is building. + +## Tagging +Tito will increase the version in the `pom.xml` for you when you tag. +However, it **does not** set the new version to a `-SNAPSHOT` +version as is the convention in many Maven projects. + +Tito uses the maven-version plugin to increase the version. + +## Building +Tito attempts to mimic the Mead build process although it is not perfect. +Mead offers the ability to descend into different directories and run the +build from there. Tito does not do this. Instead, Tito ascends to the root +of the SCM checkout and runs a `maven deploy` against the top-level POM file. +The artifacts are deployed to a temporary Maven repository under `/tmp/tito`. +Tito then attempts to render the `.tmpl` file for the package. The following +variables are available (these values are also provided by Mead when Koji +runs a build): + +* `$artifacts` - A hash of the Maven generated artifacts with the file + extensions as keys. For example: `{'.md5': 'my_project.jar.md5', '.jar': + my_project.jar'} +* `$all_artifacts` - All Maven generated artifacts in a list (including MD5 + sums, pom files, etc) +* `$all_artifacts_with_path` - All artifacts but with the full path to the + artifact within the project +* `$version` - The version of the **top level** project +* `$revision` - The revision of the **top_level** project +* `$epoch` - Currently always set to `None` +* `$name` - The name of the **top_level** project + +Note that the `name`, `version`, and `revision` variables are not very useful +as they receive their values from the top-level POM file. + +Once the spec file is rendered, Tito proceeds with a standard RPM build. + +### Assemblies +Tito attempts to use the maven-assembly plugin to build a project tarball so +its best if you have that plugin configured. Tito will fall back to a `git +archive` command, but the assembly plugin is the best way. + +### Builder Arguments +The Mead builder can take several arguments on the command line: `maven_arg` +and `maven_property`. These arguments can be specified multiple times. The +`maven_property` arguments are passed to Maven as `-D` style properties and +the `maven_arg` arguments are passed as command-line arguments. For example: + +``` +$ tito build --rpm --arg "maven_property=maven.test.skip=true" --arg "maven_arg=-X" -arg "maven_arg=-fae" +``` + +### Other Cheetah Notes + +* Dollar signs are meaningful in Cheetah. If your spec file contains shell + variables, you will need to surround the relevant block with `#raw` and + `#end raw` tags. + +``` +#raw +for selinuxvariant in %{selinux_variants} +do + make NAME=$selinuxvariant -f /usr/share/selinux/devel/Makefile + mv %{modulename}.pp %{modulename}.pp.$selinuxvariant + make NAME=$selinuxvariant -f /usr/share/selinux/devel/Makefile clean +done +#end raw +``` +* Do not use line continuations (i.e. ending a line with `\`)! + Cheetah doesn't like them for some reason. +* Do not use non-ASCII characters anywhere in the Cheetah template (including + names with non-ASCII characters in the changelog.) I ran into difficulties + with the template failing to render when non-ASCII characters appeared in + it. +* Always surround the entire `%changelog` section in `#raw` and `#end raw` + tags so that people's changelog entries won't break the template. + +## Releasing + +### Configuring the `DistGitMeadReleaser` +The first thing to do is to create a section in your `releasers.conf` for the +mead releaser. In addition to the releaser class and branch, you need to +provide values for `mead_scm` and `mead_push_url`. Mead does not build from +tarballs located in the Koji lookaside cache. Instead it builds from URLs +pointing to an SCM repository. + +* `mead_scm` - The SCM URL used to checkout the code +* `mead_target` - The SCM URL that Tito will use to push a copy of its tags + to. A `MEAD_SCM_USERNAME` value set in `~/.titorc` can be used here. + +For example: + +``` +[mead] +releaser = tito.release.DistGitMeadReleaser +branches = fedora-20 +mead_scm = git://example.com/my_project.git +mead_push_url = git+ssh://MEAD_SCM_USERNAME@example.com/my_project.git +``` + +### Configuring the Koji Build Process +The `tito.release.DistGitMeadReleaser` releaser using Mead's `maven-chain` +functionality. The top-level package directory (the same directory with the +`.tmpl` file) must have a `.chain` file. + +The chain file is an INI style file with each section as a different link in +the chain. The section title is in the "groupId-artifactId" format and +contains a value for `scmurl` which is a pointer to the SCM containing the +code and a pointer to the git ref to use. For example + +``` +scmurl=git://example.com/my_project.git?app#app-1.2.3-1 +``` + +Note that the SCM URL can be followed by a "?" and a subdirectory; and that +the git ref is placed at the end after a "#". + +Other options: + +* `buildrequires` - a space delimited list of all sections that must be built + beforehand +* `type` - set this to `wrapper` to add a wrapper RPM step. Wrapper RPM steps + need to have a `buildrequires` on the actual Maven build step. +* `packages` - a space delimited list of additional YUM packages to install + before building +* `maven_options` - command line flags to pass to Maven +* `properties` - "key=value" list of properties to set with `-D` + +For Tito, the `.chain` file needs to have a few variables in it that will be +filled in to alleviate the need for setting the git ref by hand. + +The variables passed in are + +* `mead_scm`: The value of `mead_scm` in `releasers.conf` +* `git_ref`: The git reference Tito is trying to build +* `maven_properties`: The value of any `maven_properties` provided as builder arguments +* `maven_options`: The value of any `maven_args` provided as builder arguments + +Here's a short example that will ultimately produce an RPM named "app". + +``` +[org.example-foobar-parent] +scmurl=${mead_scm}#${git_ref} +maven_options=-N ${maven_options} + +[org.example-foobar-common] +scmurl=${mead_scm}?common#${git_ref} +buildrequires=org.example-foobar-parent +packages=gettext + +[org.example-app] +scmurl=${mead_scm}?app#${git_ref} +buildrequires=org.example-foobar-common + +[app] +type=wrapper +scmurl=${mead_scm}?app#${git_ref} +buildrequires=org.example-app +``` + +During the release process, Tito will check in a rendered version of your +chain file to dist-git (as well as upload a tarball of the source it is +building to the lookaside cache). It will then fire off a build. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..3febff3 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,10 @@ +site_name: Tito +repo_url: https://github.com/dgoodwin/tito +docs_dir: doc +markdown_extensions: + - codehilite + - extra +pages: + - 'Introduction': 'index.md' + - 'Builders': 'builders.md' + - 'Mead': 'mead.md'