mirror of
https://github.com/release-engineering/dist-git.git
synced 2025-02-23 15:02:54 +00:00
README.md updated
This commit is contained in:
clime
parent
99f4735d7a
commit
6d669cfb65
1 changed files with 32 additions and 99 deletions
131
README.md
131
README.md
|
|
@ -1,134 +1,63 @@
|
|||
**This project is under developement. Please do not use it in production. Ideas, issues and patches are very welcome.**
|
||||
DistGit
|
||||
=======
|
||||
|
||||
Dist Git
|
||||
========
|
||||
DistGit (Distributed Git) is Git with additional data storage. It is designed to hold content of source rpms and consists of these three main components:
|
||||
|
||||
Dist Git is a remote Git repository specificaly designed to hold RPM package sources. It consists of three main modules:
|
||||
|
||||
1. Git repository with permissions managed by [Gitolite](http://gitolite.com/gitolite/index.html)
|
||||
1. Git repositories
|
||||
2. Lookaside cache to store source tarballs
|
||||
3. Scripts to manage
|
||||
3. Scripts to manage both
|
||||
|
||||
How Does It Work
|
||||
----------------
|
||||
|
||||
### Hosting Files
|
||||
|
||||
An RPM package repository typically consists of a spec file and the sources itself. Sources are most often taken from the upstream as they are and packed as a tarball. The sources can contain large files like virtual machine images, which, in some cases, can grow up to several GB. Those binary files can not be stored in git effectively - so the Dist Git stores them in a separate place called Lookaside Cache and only a text link to the cache is stored in the git itself.
|
||||
RPM source package typically contains a spec file and the sources (upstream tarball + additional patches). Source tarballs, being binary and potentially large, are not very well suited to be placed in a Git repository. On each their update, Git would produce a huge, meaningless diff. That's why DistGit was introduced as it employs an efficient lookaside cache where the tarballs can be stored. The Git repo itself can then be left to do what it does best: keep track of changes on the spec file, downstream patches, and an additional text file called `sources` that contains link to the source tarball in the lookaside cache.
|
||||
|
||||
|
||||
|
||||
### Communication
|
||||
|
||||
The Dist Git server repeatedly asks a package database for information about packages. This information contains a list of packages and other information. Each package can have a list of users or groups entitled to commit to this package and a list of platforms for which the package is built. Sources for each platform are held in corresponding branches.
|
||||
|
||||
User cat interact with the Dist Git server using client probably based on [rpkg](https://fedorahosted.org/rpkg/). The client authenticates with an ssh certificate for git communication and with an http client certificate for uploads to the lookaside cache.
|
||||
|
||||
|
||||
|
||||
#### Package Database Communication
|
||||
The following is an example JSON data comming from the Package Database which would create two packages: *copr-frontend* and *copr-backend*. The first package would be for Fedora 21 only and permissions to commit into this repo would be granted to users *mirek*, *adam* and anyone in the group *provenpackager*. The *copr-backend* package would be for Fedora 21 and CentOS 7. The permissions would be processed the same way as for the first package.
|
||||
|
||||
```JSON
|
||||
"packageAcls": {
|
||||
"copr-frontend": {
|
||||
"fedora-21": {
|
||||
"commit": {
|
||||
"groups": ["provenpackager"],
|
||||
"people": ["mirek", "adam"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"copr-backend": {
|
||||
"fedora-21": {
|
||||
"commit": {
|
||||
"groups": ["provenpackager"],
|
||||
"people": ["mirek", "valentin"]
|
||||
}
|
||||
},
|
||||
"centos-7": {
|
||||
"commit": {
|
||||
"groups": ["provenpackager"],
|
||||
"people": ["mirek", "valentin"]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The final result would consist of two package repositories:
|
||||
- *copr-frontend* with a single branch: *fedora-21*
|
||||
- *copr-backend* with two branches: *fedora-21* and *centos-7*
|
||||
|
||||
#### Client Authentication and Authorization
|
||||
|
||||
In order to make changes in the package repositories, client needs to have a permission to do that. Both Git and Lookaside Cache have their own auth process.
|
||||
|
||||
Git uses ssh communication and client authenticates with public key. Each user needs to have an account on the server and be in a *packager* group. Their ssh shell must be set to "`HOME=/var/lib/dist-git/git /usr/share/gitolite3/gitolite-shell $USERNAME`" in order to have authorization working.
|
||||
|
||||
Authorization is done by Gitolte. The configuration file describing all the permisions is automaticaly generated each time a Package Database is queried. Gitolite uses system users and groups.
|
||||
|
||||
Lookaside Cache uses https communication and client authenticates with ssl client certificate. The Dist Git service provider needs to issue the client certificate for every user.
|
||||
|
||||
There is no authentication needed in order to read from the server.
|
||||
|
||||
|
||||
Instalation Guide
|
||||
-----------------
|
||||
User Guide
|
||||
----------
|
||||
|
||||
The project is prepared to be built as an RPM package. You can easily build it on [Fedora](https://getfedora.org/) or [CentOS](https://www.centos.org/) using a tool called [Tito](https://github.com/dgoodwin/tito).
|
||||
|
||||
#### 1. Build and Install the Package:
|
||||
|
||||
To build the current release, use the following command in the repo directory:
|
||||
`$ tito build --rpm`
|
||||
To build the current release, use the following command in the repo directory:
|
||||
`$ tito build --rpm`
|
||||
|
||||
Install the resulting RPM package:
|
||||
`# yum install /path/to/the-package.rpm`
|
||||
Install the resulting RPM package:
|
||||
`# dnf install /path/to/the-package.rpm`
|
||||
|
||||
#### 2. Configuration:
|
||||
|
||||
Edit the configuration file at `/etc/dist-git/dist-git.conf` to match your requirements. The file contains several examples and tips that should help you with your setup.
|
||||
Enable the lookaside cache by using and modifying the example httpd config:
|
||||
|
||||
Enable the lookaside cache by using and modifying the example httpd scripts:
|
||||
```
|
||||
# cd /etc/httpd/conf.d/
|
||||
# cp ssl.conf.example ssl.conf
|
||||
|
||||
# cd /etc/httpd/conf.d/dist-git/
|
||||
# cp lookaside-upload.conf.example lookaside-upload.conf
|
||||
# vim lookaside-upload.conf
|
||||
```
|
||||
|
||||
Lookaside Cache uses https communication and client authenticates with ssl client certificate. The Dist Git service provider needs to issue the client certificate for every user.
|
||||
|
||||
#### 3. Users and Groups:
|
||||
|
||||
All users need to:
|
||||
1. have an ssh access with private key authentication
|
||||
2. be in a *packager* group
|
||||
3. have their ssh shell restricted to "`HOME=/var/lib/dist-git/git /usr/share/gitolite3/gitolite-shell $USERNAME`"
|
||||
4. be provided with an ssl client certificate to authenticate with the lookaside cache
|
||||
All DistGit users need to:
|
||||
1. have an ssh server access with private key authentication
|
||||
2. be in a *packager* group on the server
|
||||
3. be provided with an ssl client certificate to authenticate with the lookaside cache
|
||||
|
||||
An example setup of the first three steps could look like this:
|
||||
#### 4. Install DistGit Web Interface:
|
||||
|
||||
Install Cgit, the web interface for Git:
|
||||
`# dnf install cgit`
|
||||
|
||||
And point it to the DistGit repositories:
|
||||
```
|
||||
USER="frank"
|
||||
RSA="ssh-rsa AAA...YqfTP frank@example.com"
|
||||
|
||||
useradd $USER
|
||||
usermod -aG packager $USER
|
||||
mkdir /home/$USER/.ssh
|
||||
echo "command=\"HOME=/var/lib/dist-git/git/ /usr/share/gitolite3/gitolite-shell $USER\" $RSA" > /home/$USER/.ssh/authorized_keys
|
||||
# echo "project-list=/srv/git/pkgs-git-repos-list" >> /etc/cgitrc
|
||||
# echo "scan-path=/srv/git/repositories/" >> /etc/cgitrc
|
||||
```
|
||||
|
||||
#### 4. Install the Web Interface:
|
||||
|
||||
Install Cgit, the web interface for git:
|
||||
`# yum install cgit`
|
||||
|
||||
And point it to the distgit repositories:
|
||||
```
|
||||
# echo "project-list=/var/lib/dist-git/git/pkgs-git-repos-list" >> /etc/cgitrc
|
||||
# echo "scan-path=/var/lib/dist-git/git/rpms/" >> /etc/cgitrc
|
||||
```
|
||||
It is useful to comment out `cache-size` entry in /etc/cgitrc (or set it to zero) to always get up-to-date repository state at each page refresh.
|
||||
|
||||
The web interface will be available on address like `http://your-server/cgit`.
|
||||
|
||||
|
|
@ -139,3 +68,7 @@ The web interface will be available on address like `http://your-server/cgit`.
|
|||
# systemctl start httpd
|
||||
# systemctl start dist-git.socket
|
||||
```
|
||||
|
||||
#### 6. DistGit client tools:
|
||||
|
||||
To interact with DistGit server, you can use use rpkg command-line tool or python pyrpkg library.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue