release-engineering/dist-git
1
0
Fork 0
mirror of https://github.com/release-engineering/dist-git.git synced 2025-02-23 23:12:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #10 from clime/master

Browse source 
Compatibility patch with Fedora Infra and copr-dist-git
This commit is contained in:
Miroslav Suchý 2017-02-22 16:32:47 +01:00 • committed by GitHub
commit 226fa6586d
41 changed files with 5470 additions and 1314 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

138
README.md
View file

@ -1,134 +1,74 @@
**This project is under developement. Please do not use it in production. Ideas, issues and patches are very welcome.**
DistGit
=======
Dist Git
========
DistGit (Distributed Git) is Git with additional data storage. It is designed to hold content of source rpms and consists of these three main components:
Dist Git is a remote Git repository specificaly designed to hold RPM package sources. It consists of three main modules:
1. Git repository with permissions managed by [Gitolite](http://gitolite.com/gitolite/index.html)
1. Git repositories
2. Lookaside cache to store source tarballs
3. Scripts to manage
3. Scripts to manage both
How Does It Work
----------------
### Hosting Files
An RPM package repository typically consists of a spec file and the sources itself. Sources are most often taken from the upstream as they are and packed as a tarball. The sources can contain large files like virtual machine images, which, in some cases, can grow up to several GB. Those binary files can not be stored in git effectively - so the Dist Git stores them in a separate place called Lookaside Cache and only a text link to the cache is stored in the git itself.
RPM source package typically contains a spec file and the sources (upstream tarball + additional patches). Source tarballs, being binary and potentially large, are not very well suited to be placed in a Git repository. On each their update, Git would produce a huge, meaningless diff. That's why DistGit was introduced as it employs an efficient lookaside cache where the tarballs can be stored. The Git repo itself can then be left to do what it does best: keep track of changes on the spec file, downstream patches, and an additional text file called `sources` that contains link to the source tarball in the lookaside cache.
![storage](/images/storage.png)
### Communication
The Dist Git server repeatedly asks a package database for information about packages. This information contains a list of packages and other information. Each package can have a list of users or groups entitled to commit to this package and a list of platforms for which the package is built. Sources for each platform are held in corresponding branches.
User cat interact with the Dist Git server using client probably based on [rpkg](https://fedorahosted.org/rpkg/). The client authenticates with an ssh certificate for git communication and with an http client certificate for uploads to the lookaside cache.
![server-communication](/images/server-communication.png)
#### Package Database Communication
The following is an example JSON data comming from the Package Database which would create two packages: *copr-frontend* and *copr-backend*. The first package would be for Fedora 21 only and permissions to commit into this repo would be granted to users *mirek*, *adam* and anyone in the group *provenpackager*. The *copr-backend* package would be for Fedora 21 and CentOS 7. The permissions would be processed the same way as for the first package.
```JSON
"packageAcls": {
"copr-frontend": {
"fedora-21": {
"commit": {
"groups": ["provenpackager"],
"people": ["mirek", "adam"]
}
}
},
"copr-backend": {
"fedora-21": {
"commit": {
"groups": ["provenpackager"],
"people": ["mirek", "valentin"]
}
},
"centos-7": {
"commit": {
"groups": ["provenpackager"],
"people": ["mirek", "valentin"]
}
}
}
}
```
The final result would consist of two package repositories:
- *copr-frontend* with a single branch: *fedora-21*
- *copr-backend* with two branches: *fedora-21* and *centos-7*
#### Client Authentication and Authorization
In order to make changes in the package repositories, client needs to have a permission to do that. Both Git and Lookaside Cache have their own auth process.
Git uses ssh communication and client authenticates with public key. Each user needs to have an account on the server and be in a *packager* group. Their ssh shell must be set to "`HOME=/var/lib/dist-git/git /usr/share/gitolite3/gitolite-shell $USERNAME`" in order to have authorization working.
Authorization is done by Gitolte. The configuration file describing all the permisions is automaticaly generated each time a Package Database is queried. Gitolite uses system users and groups.
Lookaside Cache uses https communication and client authenticates with ssl client certificate. The Dist Git service provider needs to issue the client certificate for every user.
There is no authentication needed in order to read from the server.
Instalation Guide
-----------------
User Guide
----------
The project is prepared to be built as an RPM package. You can easily build it on [Fedora](https://getfedora.org/) or [CentOS](https://www.centos.org/) using a tool called [Tito](https://github.com/dgoodwin/tito).
#### 1. Build and Install the Package:
To build the current release, use the following command in the repo directory:
`$ tito build --rpm`
To build the current release, use the following command in the repo directory:
Install the resulting RPM package:
`# yum install /path/to/the-package.rpm`
```
$ tito build --rpm
```
Install the resulting RPM package:
```
# dnf install /path/to/the-package.rpm
```
#### 2. Configuration:
Edit the configuration file at `/etc/dist-git/dist-git.conf` to match your requirements. The file contains several examples and tips that should help you with your setup.
Enable the lookaside cache by using and modifying the example httpd config:
Enable the lookaside cache by using and modifying the example httpd scripts:
```
# cd /etc/httpd/conf.d/
# cp ssl.conf.example ssl.conf
# cd /etc/httpd/conf.d/dist-git/
# cp lookaside-upload.conf.example lookaside-upload.conf
# vim lookaside-upload.conf
```
Lookaside Cache uses https communication and client authenticates with ssl client certificate. The Dist Git service provider needs to issue the client certificate for every user.
#### 3. Users and Groups:
All users need to:
1. have an ssh access with private key authentication
2. be in a *packager* group
3. have their ssh shell restricted to "`HOME=/var/lib/dist-git/git /usr/share/gitolite3/gitolite-shell $USERNAME`"
4. be provided with an ssl client certificate to authenticate with the lookaside cache
All DistGit users need to:
1. have an ssh server access with private key authentication
2. be in a *packager* group on the server
3. be provided with an ssl client certificate to authenticate with the lookaside cache
#### 4. Install DistGit Web Interface:
Install Cgit, the web interface for Git:
An example setup of the first three steps could look like this:
```
USER="frank"
RSA="ssh-rsa AAA...YqfTP frank@example.com"
useradd $USER
usermod -aG packager $USER
mkdir /home/$USER/.ssh
echo "command=\"HOME=/var/lib/dist-git/git/ /usr/share/gitolite3/gitolite-shell $USER\" $RSA" > /home/$USER/.ssh/authorized_keys
# dnf install cgit
```
#### 4. Install the Web Interface:
And point it to the DistGit repositories:
Install Cgit, the web interface for git:
`# yum install cgit`
```
# echo "project-list=/srv/git/pkgs-git-repos-list" >> /etc/cgitrc
# echo "scan-path=/srv/git/repositories/" >> /etc/cgitrc
```
And point it to the distgit repositories:
```
# echo "project-list=/var/lib/dist-git/git/pkgs-git-repos-list" >> /etc/cgitrc
# echo "scan-path=/var/lib/dist-git/git/rpms/" >> /etc/cgitrc
```
It is useful to comment out `cache-size` entry in /etc/cgitrc (or set it to zero) to always get up-to-date repository state at each page refresh.
The web interface will be available on address like `http://your-server/cgit`.
@ -139,3 +79,7 @@ The web interface will be available on address like `http://your-server/cgit`.
# systemctl start httpd
# systemctl start dist-git.socket
```
#### 6. DistGit client tools:
To interact with DistGit server, you can use use rpkg command-line tool or python pyrpkg library.

70
Vagrantfile vendored Normal file
View file

@ -0,0 +1,70 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
### distgit ###################################################
config.vm.define "distgit" do |distgit|
distgit.vm.box = "fedora/25-cloud-base"
distgit.vm.synced_folder ".", "/vagrant", type: "rsync"
distgit.vm.provision "shell",
inline: "echo 'nameserver 8.8.8.8' >> /etc/resolv.conf",
run: "always"
# Update the system
distgit.vm.provision "shell",
inline: "dnf clean all && sudo dnf -y update || true" # || true cause dnf might return non-zero status (probly delta rpm rebuilt failed)
distgit.vm.provision "shell",
inline: "dnf install -y tito wget"
distgit.vm.provision "shell",
inline: "dnf builddep -y /vagrant/dist-git.spec",
run: "always"
distgit.vm.provision "shell",
inline: "dnf builddep -y /vagrant/dist-git.spec",
run: "always"
distgit.vm.provision "shell",
inline: "cd /vagrant/ && tito build -i --test --rpm",
run: "always"
distgit.vm.provision "shell",
inline: "cp /etc/httpd/conf.d/dist-git/lookaside-upload.conf.example /etc/httpd/conf.d/dist-git/lookaside-upload.conf",
run: "always"
distgit.vm.provision "file",
source: "./beaker-tests/pkgs-files/pkgs.example.org.pem", destination: "/tmp/pkgs.example.org.pem",
run: "always"
distgit.vm.provision "shell",
inline: "mv /tmp/pkgs.example.org.pem /etc/pki/tls/certs/pkgs.example.org.pem && restorecon -R /etc/pki/tls/certs/",
run: "always"
distgit.vm.provision "file",
source: "./beaker-tests/pkgs-files/ca-bundle.crt", destination: "/tmp/ca-bundle.crt",
run: "always"
distgit.vm.provision "shell",
inline: "mv /tmp/ca-bundle.crt /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem && restorecon -R /etc/pki/ca-trust/extracted/pem/",
run: "always"
distgit.vm.provision "shell",
inline: "systemctl enable dist-git.socket && systemctl start dist-git.socket",
run: "always"
distgit.vm.provision "shell",
inline: "systemctl enable httpd && systemctl start httpd",
run: "always"
distgit.vm.provision "shell",
inline: "useradd clime -G packager"
distgit.vm.provision "shell",
inline: "echo 'clime ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers"
end
end

63
beaker-tests/Makefile Normal file
View file

@ -0,0 +1,63 @@
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# Makefile of /tools/dist-git/Regression/
# Description: Test dist-git functionality.
# Author: clime <clime@redhat.com>
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# Copyright (c) 2016 Red Hat, Inc.
#
# This program is free software: you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation, either version 2 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
# PURPOSE. See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/.
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
export TEST=/tools/dist-git/Regression/
export TESTVERSION=1.0
BUILT_FILES=
FILES=$(METADATA) run.sh Makefile PURPOSE pkgs-files files setup.sh tests
.PHONY: all install download clean
run: $(FILES) build
./run.sh
build: $(BUILT_FILES)
test -x run.sh || chmod a+x run.sh
clean:
rm -f *~ $(BUILT_FILES)
include /usr/share/rhts/lib/rhts-make.include
$(METADATA): Makefile
@echo "Owner: clime <clime@redhat.com>" > $(METADATA)
@echo "Name: $(TEST)" >> $(METADATA)
@echo "TestVersion: $(TESTVERSION)" >> $(METADATA)
@echo "Path: $(TEST_DIR)" >> $(METADATA)
@echo "Description: Test dist-git functionality." >> $(METADATA)
@echo "Type: Regression" >> $(METADATA)
@echo "TestTime: 2h" >> $(METADATA)
@echo "RunFor: dist-git" >> $(METADATA)
@echo "Requires: dist-git" >> $(METADATA)
@echo "Priority: Normal" >> $(METADATA)
@echo "License: GPLv2+" >> $(METADATA)
@echo "Confidential: no" >> $(METADATA)
@echo "Destructive: no" >> $(METADATA)
@echo "Releases: -RHEL4 -RHELClient5 -RHELServer5" >> $(METADATA)
rhts-lint $(METADATA)

3
beaker-tests/PURPOSE Normal file
View file

@ -0,0 +1,3 @@
PURPOSE of /tools/dist-git/Regression/
Description: Test dist-git functionality.
Author: clime <clime@redhat.com>

32
beaker-tests/files/etc/rpkg/fedpkg.conf Normal file
View file

@ -0,0 +1,32 @@
[fedpkg]
lookaside = http://pkgs.example.org/repo/pkgs
lookasidehash = md5
lookaside_cgi = https://pkgs.example.org/repo/pkgs/upload.cgi
gitbaseurl = ssh://clime@pkgs.example.org/%(module)s
anongiturl = git://pkgs.example.org/%(module)s
tracbaseurl = https://clime:%(password)s@fedorahosted.org/rel-eng/login/xmlrpc
branchre = f\d$|f\d\d$|el\d$|olpc\d$|master$
kojiconfig = /etc/koji.conf
build_client = koji
clone_config =
bz.default-tracker bugzilla.redhat.com
bz.default-product Fedora
bz.default-version rawhide
bz.default-component %(module)s
sendemail.to %(module)s-owner@fedoraproject.org
push.default simple
#distgit_namespaced = True
kerberos_realms = FEDORAPROJECT.ORG
[fedpkg.bodhi]
# This is for the bodhi-client 1.x. that accepts --bodhi-dir option to switch
# to different instances including the production and stage.
url = https://bodhi.fedoraproject.org/
# This is for the bodhi-client 2.x, that do not require an option to switch to
# different instance. Instead, --staging is available to switch to the stage
# bodhi, and production is used without providing --staging.
staging = False
[fedpkg.pkgdb]
url = https://admin.fedoraproject.org/pkgdb/

25
beaker-tests/files/root/.fedora-server-ca.cert Normal file
View file

@ -0,0 +1,25 @@
-----BEGIN CERTIFICATE-----
MIIEJTCCAw2gAwIBAgIJAMqvMDXuzzLXMA0GCSqGSIb3DQEBCwUAMIGoMQswCQYD
VQQGEwJDWjEXMBUGA1UECAwOQ3plY2ggUmVwdWJsaWMxDTALBgNVBAcMBEJybm8x
GDAWBgNVBAoMD0V4YW1wbGUgUHJvamVjdDEbMBkGA1UECwwSRXhhbXBsZSBQcm9q
ZWN0IENBMRkwFwYDVQQDDBBwa2dzLmV4YW1wbGUub3JnMR8wHQYJKoZIhvcNAQkB
FhBjbGltZUByZWRoYXQuY29tMB4XDTE3MDIxMTEzNTM0MVoXDTE4MDIxMTEzNTM0
MVowgagxCzAJBgNVBAYTAkNaMRcwFQYDVQQIDA5DemVjaCBSZXB1YmxpYzENMAsG
A1UEBwwEQnJubzEYMBYGA1UECgwPRXhhbXBsZSBQcm9qZWN0MRswGQYDVQQLDBJF
eGFtcGxlIFByb2plY3QgQ0ExGTAXBgNVBAMMEHBrZ3MuZXhhbXBsZS5vcmcxHzAd
BgkqhkiG9w0BCQEWEGNsaW1lQHJlZGhhdC5jb20wggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDdAN9IwCYTG+lTHPxSR1/UyqVORieUiBxFNGFA0lwfpT0A
2KXN25oUaPTU3YmXOp4itt44v2eHbK2C+iuyhbR5tw5SS9GH9bTXxWmQv7WgwJi1
Jg1zRWANgkeqe4aHPUm0SCbC14q/BBqAq+zcIJSb4/tN05CQl/KJ6bz459vfj1bR
qnmaJwyClSg2++vnCr2E+F2+rFtJzmx+gOZjd/nrDaUJJrUHgfWDl989QpMlah+y
Lq/Nm7Ie5a25eE+u7Z/H7HHuoPSv54fdq+f4viFAqeJmg5pEEzjDTcxsIAKDPjGl
qrv1F1798ZXw5/L3j5op5M3Zi4NZDmn8NZQsxWEjAgMBAAGjUDBOMB0GA1UdDgQW
BBR0Xs+1RuRmK2feWpT0KHEXYRKytjAfBgNVHSMEGDAWgBR0Xs+1RuRmK2feWpT0
KHEXYRKytjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAtFVzpzHuQ
SvuJNlB3loXDYspInmA6dtOp+rfjKaPX2HOx1VV/NN6lgRTY8oJYajhL/1Q4kr+B
33Vh+DbKnq/tEnQbNHSNI3OMwcWHedZ7SAlsNaPLfxUpRyjAOFTv+6kDgYs5M2VN
uPbY06o+VUg1UQqtaP0PiZ5qlxQeY44Lno6lcG0wACZYNCt/f+MtCyvowNP297dl
HKDijjui/OKrAEab3dXd9HCPOJUDlJoz3DUVzjEBMtw+Xbs9wwfj1zdu4W8pFMip
NujsR824a7Ys5+JcIyJxDWGsMYx6RsBQcBpx/E/40/ehodtpc2RvQEDNdpxiW32w
4oTW+0s+0z3u
-----END CERTIFICATE-----

25
beaker-tests/files/root/.fedora-upload-ca.cert Normal file
View file

@ -0,0 +1,25 @@
-----BEGIN CERTIFICATE-----
MIIEJTCCAw2gAwIBAgIJAMqvMDXuzzLXMA0GCSqGSIb3DQEBCwUAMIGoMQswCQYD
VQQGEwJDWjEXMBUGA1UECAwOQ3plY2ggUmVwdWJsaWMxDTALBgNVBAcMBEJybm8x
GDAWBgNVBAoMD0V4YW1wbGUgUHJvamVjdDEbMBkGA1UECwwSRXhhbXBsZSBQcm9q
ZWN0IENBMRkwFwYDVQQDDBBwa2dzLmV4YW1wbGUub3JnMR8wHQYJKoZIhvcNAQkB
FhBjbGltZUByZWRoYXQuY29tMB4XDTE3MDIxMTEzNTM0MVoXDTE4MDIxMTEzNTM0
MVowgagxCzAJBgNVBAYTAkNaMRcwFQYDVQQIDA5DemVjaCBSZXB1YmxpYzENMAsG
A1UEBwwEQnJubzEYMBYGA1UECgwPRXhhbXBsZSBQcm9qZWN0MRswGQYDVQQLDBJF
eGFtcGxlIFByb2plY3QgQ0ExGTAXBgNVBAMMEHBrZ3MuZXhhbXBsZS5vcmcxHzAd
BgkqhkiG9w0BCQEWEGNsaW1lQHJlZGhhdC5jb20wggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDdAN9IwCYTG+lTHPxSR1/UyqVORieUiBxFNGFA0lwfpT0A
2KXN25oUaPTU3YmXOp4itt44v2eHbK2C+iuyhbR5tw5SS9GH9bTXxWmQv7WgwJi1
Jg1zRWANgkeqe4aHPUm0SCbC14q/BBqAq+zcIJSb4/tN05CQl/KJ6bz459vfj1bR
qnmaJwyClSg2++vnCr2E+F2+rFtJzmx+gOZjd/nrDaUJJrUHgfWDl989QpMlah+y
Lq/Nm7Ie5a25eE+u7Z/H7HHuoPSv54fdq+f4viFAqeJmg5pEEzjDTcxsIAKDPjGl
qrv1F1798ZXw5/L3j5op5M3Zi4NZDmn8NZQsxWEjAgMBAAGjUDBOMB0GA1UdDgQW
BBR0Xs+1RuRmK2feWpT0KHEXYRKytjAfBgNVHSMEGDAWgBR0Xs+1RuRmK2feWpT0
KHEXYRKytjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAtFVzpzHuQ
SvuJNlB3loXDYspInmA6dtOp+rfjKaPX2HOx1VV/NN6lgRTY8oJYajhL/1Q4kr+B
33Vh+DbKnq/tEnQbNHSNI3OMwcWHedZ7SAlsNaPLfxUpRyjAOFTv+6kDgYs5M2VN
uPbY06o+VUg1UQqtaP0PiZ5qlxQeY44Lno6lcG0wACZYNCt/f+MtCyvowNP297dl
HKDijjui/OKrAEab3dXd9HCPOJUDlJoz3DUVzjEBMtw+Xbs9wwfj1zdu4W8pFMip
NujsR824a7Ys5+JcIyJxDWGsMYx6RsBQcBpx/E/40/ehodtpc2RvQEDNdpxiW32w
4oTW+0s+0z3u
-----END CERTIFICATE-----

111
beaker-tests/files/root/.fedora.cert Normal file
View file

@ -0,0 +1,111 @@
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 4390 (0x1126)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=CZ, ST=Czech Republic, L=Brno, O=Example Project, OU=Example Project CA, CN=pkgs.example.org/emailAddress=clime@redhat.com
Validity
Not Before: Feb 11 14:48:08 2017 GMT
Not After : Feb 11 14:48:08 2018 GMT
Subject: C=CZ, ST=Czech Republic, O=Example Project, OU=Example User Cert, CN=clime/emailAddress=clime@redhat.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:bb:47:87:4b:7c:63:b7:44:22:05:6f:18:c5:ba:
c8:72:ff:76:88:3b:40:a3:f9:c3:0b:1d:1d:06:e3:
9e:dc:f0:f8:5d:1c:51:94:f0:55:6c:df:86:df:72:
95:78:ec:e7:17:2e:b8:82:d2:9e:0b:8e:db:13:7f:
13:8e:57:27:e3:30:a2:19:ad:22:3e:4e:b3:c0:35:
33:1f:32:cf:49:aa:1f:eb:0a:0f:c0:4c:9c:33:df:
fb:42:0f:dc:f1:7c:79:7c:c1:ab:bf:f8:c4:99:a5:
0c:ed:9d:85:eb:d7:41:50:3c:b8:d8:f8:56:f0:04:
c5:c9:34:8f:c0:82:2c:8f:96:0b:f6:1b:33:72:66:
f1:4c:fe:24:70:8a:6a:c6:01:1e:31:c3:fa:ad:3c:
5b:78:6c:c4:4d:c1:eb:1b:c3:d1:a8:ac:a5:ff:e9:
db:ff:b2:36:c3:f8:02:a7:56:3e:c8:bb:fa:a6:1e:
27:71:d7:f2:b8:fe:ba:61:47:03:fe:c5:41:4c:de:
40:d2:83:39:03:a8:ae:50:cd:9e:8b:e7:7b:b6:72:
96:8c:0a:f0:54:27:6f:5b:01:4e:5c:cd:d7:54:89:
97:64:56:9d:15:99:95:65:8d:cd:20:5a:df:34:21:
6c:fe:d7:c5:dd:a8:a8:7c:cb:b9:4a:d8:55:86:ea:
1b:9f
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
F4:A8:04:D2:47:5E:F1:21:D8:B6:58:1F:04:52:47:EE:53:06:99:1A
X509v3 Authority Key Identifier:
keyid:74:5E:CF:B5:46:E4:66:2B:67:DE:5A:94:F4:28:71:17:61:12:B2:B6
Signature Algorithm: sha256WithRSAEncryption
06:8e:59:4a:fa:24:d2:5b:5d:89:67:ff:a2:1e:9e:5b:ca:63:
a4:fa:16:4a:69:0f:7c:4f:ac:6f:7d:f3:a7:18:97:7f:f6:29:
82:89:30:eb:00:63:ab:aa:75:a4:99:47:f1:6e:fe:c7:d2:6d:
f7:51:31:dc:ac:7f:76:ad:0f:cc:2c:46:63:6d:86:e7:35:7d:
0c:ad:4a:27:d7:26:c9:19:a5:eb:75:aa:07:45:df:c4:bb:23:
0a:16:7a:38:c3:0b:fa:7b:13:c9:88:30:98:d0:c5:78:57:85:
46:8d:d9:f2:95:12:1c:26:04:4a:73:42:4f:2b:6c:61:c6:c0:
8c:be:68:f2:ce:be:a8:d4:77:2c:f5:c5:1e:c3:97:2f:21:83:
f2:7e:ef:b3:e1:fa:8c:17:9a:e6:59:e6:4a:94:16:b1:22:7b:
15:6e:a4:51:42:2f:4e:44:35:15:9d:6b:6f:90:4a:32:ab:82:
01:e3:35:b1:aa:0e:73:bf:42:87:36:6b:cb:95:ce:9b:ae:ca:
0f:a7:ac:94:14:b3:1c:5c:44:b0:c1:e1:78:f9:3b:ea:67:6a:
c5:c2:ef:b9:78:f3:f8:ca:ab:ce:f6:82:be:01:dd:a9:f8:fc:
6b:c4:33:91:a3:dc:fe:e2:56:e9:59:45:04:37:ee:d1:64:f0:
9a:38:ff:ef
-----BEGIN CERTIFICATE-----
MIIELjCCAxagAwIBAgICESYwDQYJKoZIhvcNAQELBQAwgagxCzAJBgNVBAYTAkNa
MRcwFQYDVQQIDA5DemVjaCBSZXB1YmxpYzENMAsGA1UEBwwEQnJubzEYMBYGA1UE
CgwPRXhhbXBsZSBQcm9qZWN0MRswGQYDVQQLDBJFeGFtcGxlIFByb2plY3QgQ0Ex
GTAXBgNVBAMMEHBrZ3MuZXhhbXBsZS5vcmcxHzAdBgkqhkiG9w0BCQEWEGNsaW1l
QHJlZGhhdC5jb20wHhcNMTcwMjExMTQ0ODA4WhcNMTgwMjExMTQ0ODA4WjCBjTEL
MAkGA1UEBhMCQ1oxFzAVBgNVBAgMDkN6ZWNoIFJlcHVibGljMRgwFgYDVQQKDA9F
eGFtcGxlIFByb2plY3QxGjAYBgNVBAsMEUV4YW1wbGUgVXNlciBDZXJ0MQ4wDAYD
VQQDDAVjbGltZTEfMB0GCSqGSIb3DQEJARYQY2xpbWVAcmVkaGF0LmNvbTCCASIw
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALtHh0t8Y7dEIgVvGMW6yHL/dog7
QKP5wwsdHQbjntzw+F0cUZTwVWzfht9ylXjs5xcuuILSnguO2xN/E45XJ+Mwohmt
Ij5Os8A1Mx8yz0mqH+sKD8BMnDPf+0IP3PF8eXzBq7/4xJmlDO2dhevXQVA8uNj4
VvAExck0j8CCLI+WC/YbM3Jm8Uz+JHCKasYBHjHD+q08W3hsxE3B6xvD0aispf/p
2/+yNsP4AqdWPsi7+qYeJ3HX8rj+umFHA/7FQUzeQNKDOQOorlDNnovne7ZylowK
8FQnb1sBTlzN11SJl2RWnRWZlWWNzSBa3zQhbP7Xxd2oqHzLuUrYVYbqG58CAwEA
AaN7MHkwCQYDVR0TBAIwADAsBglghkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0
ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFPSoBNJHXvEh2LZYHwRSR+5TBpkaMB8G
A1UdIwQYMBaAFHRez7VG5GYrZ95alPQocRdhErK2MA0GCSqGSIb3DQEBCwUAA4IB
AQAGjllK+iTSW12JZ/+iHp5bymOk+hZKaQ98T6xvffOnGJd/9imCiTDrAGOrqnWk
mUfxbv7H0m33UTHcrH92rQ/MLEZjbYbnNX0MrUon1ybJGaXrdaoHRd/EuyMKFno4
wwv6exPJiDCY0MV4V4VGjdnylRIcJgRKc0JPK2xhxsCMvmjyzr6o1Hcs9cUew5cv
IYPyfu+z4fqMF5rmWeZKlBaxInsVbqRRQi9ORDUVnWtvkEoyq4IB4zWxqg5zv0KH
NmvLlc6brsoPp6yUFLMcXESwweF4+TvqZ2rFwu+5ePP4yqvO9oK+Ad2p+PxrxDOR
o9z+4lbpWUUEN+7RZPCaOP/v
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAu0eHS3xjt0QiBW8YxbrIcv92iDtAo/nDCx0dBuOe3PD4XRxR
lPBVbN+G33KVeOznFy64gtKeC47bE38Tjlcn4zCiGa0iPk6zwDUzHzLPSaof6woP
wEycM9/7Qg/c8Xx5fMGrv/jEmaUM7Z2F69dBUDy42PhW8ATFyTSPwIIsj5YL9hsz
cmbxTP4kcIpqxgEeMcP6rTxbeGzETcHrG8PRqKyl/+nb/7I2w/gCp1Y+yLv6ph4n
cdfyuP66YUcD/sVBTN5A0oM5A6iuUM2ei+d7tnKWjArwVCdvWwFOXM3XVImXZFad
FZmVZY3NIFrfNCFs/tfF3aiofMu5SthVhuobnwIDAQABAoIBAGV5Tk+SaJ1GZ4C2
P5/zouTckqciI9Y1YH9HGRFAiDWCw6//xhyDVGYXf4bXGTJaeqRPBl6sLEJq9z1L
H5eClfQ9x7dN3j4QesjsinSfHT321jCOVRaG7CQXepbC7Z5TKah1b9AmxK/ThHvD
XGr7SUJmhGnbtY97pods2W8fmXF6l/+Pyrbqjg6iPHf5LzyhcgaO9JIAbi4Mz8Yt
pJZPqC3vDYYgdiBSxqZlfNU3u/5fDVoVhJAsgvAzOzIT0PCxbnE6mpRIcStCUyR0
Dr4+A3b6iTbh+cfNuS+FyeNVVq3oQx+7vOBggGqxDNH7se9cvLa2UOA1ZSbRr0BH
uSUw+6ECgYEA52Ud/9VU2l5RF3dO99hjpZlUSqPuzt4qMK7ag8pXGUhuPklS2F9G
Jqs/HBfeNtQ+4hahG53E4qy3oc6fBdwpjSfl9iIXksbaJKei+AYiDGm20fIpSMnJ
x8MM27wz/3tW5C/MVJ2AetSWzgwwX+jYi5sxJc9Nb+ngDpVs4+/JmccCgYEAzzGG
n0G8IlB5cGMugcBv52py3bFX0Fn9YjdEpHA01KdNtQCOiMaNb0Oz5ykxzet/l6MB
OuguNGRNIJSCR/RACYpPTwIfZN3HPTVc+gvt/vyyTt0xVzAcGeTtlrzPHowyVQYX
kLgijntxH9gyI75aiuHkQLPzVysZ/i1umwINr2kCgYEAwbYU9ydQcb6jhFGM+k9c
nDVQK2wxb1ztVPQMWmf5omDQabRYdmrL/3M80ffpP11Ph1YKaWQafui4key3k+BL
qK/OA89mjKmL1UPBWoFH37xgIEHp7dV/5ouTKCDWJvrKMx0QoYV5GVr/JF9v39q9
7aFt84kn+ph/ybWuAuOqkeECgYAPTs9tcMPfnSZ7wTgsafVamajG7FValWt9GCxA
IfIlS/8XwmTFgjXs5ImkKIz/c8jHgUmT+pd/U3HHWHGPs2IP0mItqjB7mU8hHJIz
/zGkRzM9VGZS1MEewKU1H9oFncISOhL/pDsZX9Fp0uVRcrzpyeq9RQ5TgGG5P84O
McalEQKBgGcdtCrpRsdEN8cUdW5Cv9ESWJ9Pupw0tcz5faRq3Pcd3tmumDUSPJNj
WdcV4NGGZ+Wl291dxscnn89zD0DzYGr4m+HcsukEtntI5wjv8ncfTzb4PuZz6bsQ
YMHs+SbmQBK5mzL6jLkf8xASZaS5Rd9v55twi2UZ4JWyu8RPRWd4
-----END RSA PRIVATE KEY-----

84
beaker-tests/files/root/user.crt Normal file
View file

@ -0,0 +1,84 @@
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 4390 (0x1126)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=CZ, ST=Czech Republic, L=Brno, O=Example Project, OU=Example Project CA, CN=pkgs.example.org/emailAddress=clime@redhat.com
Validity
Not Before: Feb 11 14:48:08 2017 GMT
Not After : Feb 11 14:48:08 2018 GMT
Subject: C=CZ, ST=Czech Republic, O=Example Project, OU=Example User Cert, CN=clime/emailAddress=clime@redhat.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:bb:47:87:4b:7c:63:b7:44:22:05:6f:18:c5:ba:
c8:72:ff:76:88:3b:40:a3:f9:c3:0b:1d:1d:06:e3:
9e:dc:f0:f8:5d:1c:51:94:f0:55:6c:df:86:df:72:
95:78:ec:e7:17:2e:b8:82:d2:9e:0b:8e:db:13:7f:
13:8e:57:27:e3:30:a2:19:ad:22:3e:4e:b3:c0:35:
33:1f:32:cf:49:aa:1f:eb:0a:0f:c0:4c:9c:33:df:
fb:42:0f:dc:f1:7c:79:7c:c1:ab:bf:f8:c4:99:a5:
0c:ed:9d:85:eb:d7:41:50:3c:b8:d8:f8:56:f0:04:
c5:c9:34:8f:c0:82:2c:8f:96:0b:f6:1b:33:72:66:
f1:4c:fe:24:70:8a:6a:c6:01:1e:31:c3:fa:ad:3c:
5b:78:6c:c4:4d:c1:eb:1b:c3:d1:a8:ac:a5:ff:e9:
db:ff:b2:36:c3:f8:02:a7:56:3e:c8:bb:fa:a6:1e:
27:71:d7:f2:b8:fe:ba:61:47:03:fe:c5:41:4c:de:
40:d2:83:39:03:a8:ae:50:cd:9e:8b:e7:7b:b6:72:
96:8c:0a:f0:54:27:6f:5b:01:4e:5c:cd:d7:54:89:
97:64:56:9d:15:99:95:65:8d:cd:20:5a:df:34:21:
6c:fe:d7:c5:dd:a8:a8:7c:cb:b9:4a:d8:55:86:ea:
1b:9f
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
F4:A8:04:D2:47:5E:F1:21:D8:B6:58:1F:04:52:47:EE:53:06:99:1A
X509v3 Authority Key Identifier:
keyid:74:5E:CF:B5:46:E4:66:2B:67:DE:5A:94:F4:28:71:17:61:12:B2:B6
Signature Algorithm: sha256WithRSAEncryption
06:8e:59:4a:fa:24:d2:5b:5d:89:67:ff:a2:1e:9e:5b:ca:63:
a4:fa:16:4a:69:0f:7c:4f:ac:6f:7d:f3:a7:18:97:7f:f6:29:
82:89:30:eb:00:63:ab:aa:75:a4:99:47:f1:6e:fe:c7:d2:6d:
f7:51:31:dc:ac:7f:76:ad:0f:cc:2c:46:63:6d:86:e7:35:7d:
0c:ad:4a:27:d7:26:c9:19:a5:eb:75:aa:07:45:df:c4:bb:23:
0a:16:7a:38:c3:0b:fa:7b:13:c9:88:30:98:d0:c5:78:57:85:
46:8d:d9:f2:95:12:1c:26:04:4a:73:42:4f:2b:6c:61:c6:c0:
8c:be:68:f2:ce:be:a8:d4:77:2c:f5:c5:1e:c3:97:2f:21:83:
f2:7e:ef:b3:e1:fa:8c:17:9a:e6:59:e6:4a:94:16:b1:22:7b:
15:6e:a4:51:42:2f:4e:44:35:15:9d:6b:6f:90:4a:32:ab:82:
01:e3:35:b1:aa:0e:73:bf:42:87:36:6b:cb:95:ce:9b:ae:ca:
0f:a7:ac:94:14:b3:1c:5c:44:b0:c1:e1:78:f9:3b:ea:67:6a:
c5:c2:ef:b9:78:f3:f8:ca:ab:ce:f6:82:be:01:dd:a9:f8:fc:
6b:c4:33:91:a3:dc:fe:e2:56:e9:59:45:04:37:ee:d1:64:f0:
9a:38:ff:ef
-----BEGIN CERTIFICATE-----
MIIELjCCAxagAwIBAgICESYwDQYJKoZIhvcNAQELBQAwgagxCzAJBgNVBAYTAkNa
MRcwFQYDVQQIDA5DemVjaCBSZXB1YmxpYzENMAsGA1UEBwwEQnJubzEYMBYGA1UE
CgwPRXhhbXBsZSBQcm9qZWN0MRswGQYDVQQLDBJFeGFtcGxlIFByb2plY3QgQ0Ex
GTAXBgNVBAMMEHBrZ3MuZXhhbXBsZS5vcmcxHzAdBgkqhkiG9w0BCQEWEGNsaW1l
QHJlZGhhdC5jb20wHhcNMTcwMjExMTQ0ODA4WhcNMTgwMjExMTQ0ODA4WjCBjTEL
MAkGA1UEBhMCQ1oxFzAVBgNVBAgMDkN6ZWNoIFJlcHVibGljMRgwFgYDVQQKDA9F
eGFtcGxlIFByb2plY3QxGjAYBgNVBAsMEUV4YW1wbGUgVXNlciBDZXJ0MQ4wDAYD
VQQDDAVjbGltZTEfMB0GCSqGSIb3DQEJARYQY2xpbWVAcmVkaGF0LmNvbTCCASIw
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALtHh0t8Y7dEIgVvGMW6yHL/dog7
QKP5wwsdHQbjntzw+F0cUZTwVWzfht9ylXjs5xcuuILSnguO2xN/E45XJ+Mwohmt
Ij5Os8A1Mx8yz0mqH+sKD8BMnDPf+0IP3PF8eXzBq7/4xJmlDO2dhevXQVA8uNj4
VvAExck0j8CCLI+WC/YbM3Jm8Uz+JHCKasYBHjHD+q08W3hsxE3B6xvD0aispf/p
2/+yNsP4AqdWPsi7+qYeJ3HX8rj+umFHA/7FQUzeQNKDOQOorlDNnovne7ZylowK
8FQnb1sBTlzN11SJl2RWnRWZlWWNzSBa3zQhbP7Xxd2oqHzLuUrYVYbqG58CAwEA
AaN7MHkwCQYDVR0TBAIwADAsBglghkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0
ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFPSoBNJHXvEh2LZYHwRSR+5TBpkaMB8G
A1UdIwQYMBaAFHRez7VG5GYrZ95alPQocRdhErK2MA0GCSqGSIb3DQEBCwUAA4IB
AQAGjllK+iTSW12JZ/+iHp5bymOk+hZKaQ98T6xvffOnGJd/9imCiTDrAGOrqnWk
mUfxbv7H0m33UTHcrH92rQ/MLEZjbYbnNX0MrUon1ybJGaXrdaoHRd/EuyMKFno4
wwv6exPJiDCY0MV4V4VGjdnylRIcJgRKc0JPK2xhxsCMvmjyzr6o1Hcs9cUew5cv
IYPyfu+z4fqMF5rmWeZKlBaxInsVbqRRQi9ORDUVnWtvkEoyq4IB4zWxqg5zv0KH
NmvLlc6brsoPp6yUFLMcXESwweF4+TvqZ2rFwu+5ePP4yqvO9oK+Ad2p+PxrxDOR
o9z+4lbpWUUEN+7RZPCaOP/v
-----END CERTIFICATE-----

18
beaker-tests/files/root/user.csr Normal file
View file

@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE REQUEST-----
MIIC4jCCAcoCAQAwgZwxCzAJBgNVBAYTAkNaMRcwFQYDVQQIDA5DemVjaCBSZXB1
YmxpYzENMAsGA1UEBwwEQnJubzEYMBYGA1UECgwPRXhhbXBsZSBQcm9qZWN0MRow
GAYDVQQLDBFFeGFtcGxlIFVzZXIgQ2VydDEOMAwGA1UEAwwFY2xpbWUxHzAdBgkq
hkiG9w0BCQEWEGNsaW1lQHJlZGhhdC5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQC7R4dLfGO3RCIFbxjFushy/3aIO0Cj+cMLHR0G457c8PhdHFGU
8FVs34bfcpV47OcXLriC0p4LjtsTfxOOVyfjMKIZrSI+TrPANTMfMs9Jqh/rCg/A
TJwz3/tCD9zxfHl8wau/+MSZpQztnYXr10FQPLjY+FbwBMXJNI/AgiyPlgv2GzNy
ZvFM/iRwimrGAR4xw/qtPFt4bMRNwesbw9GorKX/6dv/sjbD+AKnVj7Iu/qmHidx
1/K4/rphRwP+xUFM3kDSgzkDqK5QzZ6L53u2cpaMCvBUJ29bAU5czddUiZdkVp0V
mZVljc0gWt80IWz+18XdqKh8y7lK2FWG6hufAgMBAAGgADANBgkqhkiG9w0BAQsF
AAOCAQEAZiXjgaUCc0E/T2n62DfqtJK4FasXy9bdZPy8xleeDRMt7NrT4OLe8uPy
XdBDGQjn+zd/GaA+/jf6CIccgg3EeGMRQzo9xyYJLjiOKES0Ep7oUQ63jW/5jSev
wJbUQ3QxhXX2WYtFAG/OOO/qbTxs1tSfG9aziKpC3JE7bGQ7Jp6eeuk0JHla+41a
Rliij9wd+0z9TII8f9yzFRqPPVM0fwP0w1yUIaNtGyKJY6/jJ/5AVbh/ARNxkzSY
gsJuCIUAR2o8OKHhewGpc4pFv3UzMLVa5RbSWedV5VdDML01tn9mIxrYe7vxK10C
3qRDpy+fRE8abNqNNBacna0fkYr4JQ==
-----END CERTIFICATE REQUEST-----

27
beaker-tests/files/root/user.key Normal file
View file

@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAu0eHS3xjt0QiBW8YxbrIcv92iDtAo/nDCx0dBuOe3PD4XRxR
lPBVbN+G33KVeOznFy64gtKeC47bE38Tjlcn4zCiGa0iPk6zwDUzHzLPSaof6woP
wEycM9/7Qg/c8Xx5fMGrv/jEmaUM7Z2F69dBUDy42PhW8ATFyTSPwIIsj5YL9hsz
cmbxTP4kcIpqxgEeMcP6rTxbeGzETcHrG8PRqKyl/+nb/7I2w/gCp1Y+yLv6ph4n
cdfyuP66YUcD/sVBTN5A0oM5A6iuUM2ei+d7tnKWjArwVCdvWwFOXM3XVImXZFad
FZmVZY3NIFrfNCFs/tfF3aiofMu5SthVhuobnwIDAQABAoIBAGV5Tk+SaJ1GZ4C2
P5/zouTckqciI9Y1YH9HGRFAiDWCw6//xhyDVGYXf4bXGTJaeqRPBl6sLEJq9z1L
H5eClfQ9x7dN3j4QesjsinSfHT321jCOVRaG7CQXepbC7Z5TKah1b9AmxK/ThHvD
XGr7SUJmhGnbtY97pods2W8fmXF6l/+Pyrbqjg6iPHf5LzyhcgaO9JIAbi4Mz8Yt
pJZPqC3vDYYgdiBSxqZlfNU3u/5fDVoVhJAsgvAzOzIT0PCxbnE6mpRIcStCUyR0
Dr4+A3b6iTbh+cfNuS+FyeNVVq3oQx+7vOBggGqxDNH7se9cvLa2UOA1ZSbRr0BH
uSUw+6ECgYEA52Ud/9VU2l5RF3dO99hjpZlUSqPuzt4qMK7ag8pXGUhuPklS2F9G
Jqs/HBfeNtQ+4hahG53E4qy3oc6fBdwpjSfl9iIXksbaJKei+AYiDGm20fIpSMnJ
x8MM27wz/3tW5C/MVJ2AetSWzgwwX+jYi5sxJc9Nb+ngDpVs4+/JmccCgYEAzzGG
n0G8IlB5cGMugcBv52py3bFX0Fn9YjdEpHA01KdNtQCOiMaNb0Oz5ykxzet/l6MB
OuguNGRNIJSCR/RACYpPTwIfZN3HPTVc+gvt/vyyTt0xVzAcGeTtlrzPHowyVQYX
kLgijntxH9gyI75aiuHkQLPzVysZ/i1umwINr2kCgYEAwbYU9ydQcb6jhFGM+k9c
nDVQK2wxb1ztVPQMWmf5omDQabRYdmrL/3M80ffpP11Ph1YKaWQafui4key3k+BL
qK/OA89mjKmL1UPBWoFH37xgIEHp7dV/5ouTKCDWJvrKMx0QoYV5GVr/JF9v39q9
7aFt84kn+ph/ybWuAuOqkeECgYAPTs9tcMPfnSZ7wTgsafVamajG7FValWt9GCxA
IfIlS/8XwmTFgjXs5ImkKIz/c8jHgUmT+pd/U3HHWHGPs2IP0mItqjB7mU8hHJIz
/zGkRzM9VGZS1MEewKU1H9oFncISOhL/pDsZX9Fp0uVRcrzpyeq9RQ5TgGG5P84O
McalEQKBgGcdtCrpRsdEN8cUdW5Cv9ESWJ9Pupw0tcz5faRq3Pcd3tmumDUSPJNj
WdcV4NGGZ+Wl291dxscnn89zD0DzYGr4m+HcsukEtntI5wjv8ncfTzb4PuZz6bsQ
YMHs+SbmQBK5mzL6jLkf8xASZaS5Rd9v55twi2UZ4JWyu8RPRWd4
-----END RSA PRIVATE KEY-----

4209
beaker-tests/pkgs-files/ca-bundle.crt Normal file
View file

File diff suppressed because it is too large Load diff

54
beaker-tests/pkgs-files/pkgs.example.org.pem Normal file
View file

@ -0,0 +1,54 @@
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDdAN9IwCYTG+lT
HPxSR1/UyqVORieUiBxFNGFA0lwfpT0A2KXN25oUaPTU3YmXOp4itt44v2eHbK2C
+iuyhbR5tw5SS9GH9bTXxWmQv7WgwJi1Jg1zRWANgkeqe4aHPUm0SCbC14q/BBqA
q+zcIJSb4/tN05CQl/KJ6bz459vfj1bRqnmaJwyClSg2++vnCr2E+F2+rFtJzmx+
gOZjd/nrDaUJJrUHgfWDl989QpMlah+yLq/Nm7Ie5a25eE+u7Z/H7HHuoPSv54fd
q+f4viFAqeJmg5pEEzjDTcxsIAKDPjGlqrv1F1798ZXw5/L3j5op5M3Zi4NZDmn8
NZQsxWEjAgMBAAECggEAMfPJhCsSMA2F/mg1y+8gBPvMw1VuCs98/FNri/hAJFL3
5+IQD+iL+gMDG737JHh2V8hXkCILJ3vz9//Nsqsv8LbkgMtsT7vEpWSUiMgcKHp/
yaoRLjXHESaaRjlnXdgHeb05LZ8EWEp4Jw81GUgv7tBn895W9aeyhWOUd/oII45Z
/UBe8YLXtApalsZR0ll18FqEisHe+jqLzWtjcizzbV1R3DdKBLlcAQq5fckgCwIV
TsdzNW0VwRfW+Z2EHdIf2sMuEtamS1TZ6iIAEHrhQWyXxbn1FXtaWgyPMqLeMDzW
UdksmHSKi+qDkGjPbTcTUrpg0fLsqoyS4HseSsGe0QKBgQDvMl5vM30GB8VXIKe6
l7sHVQTjBcVe64yB9zPGUXxOr0W2GfcUo98HkSwQUkoUUQy4lREZ8DIC7OXSJVlN
cDf0FYffxQVkDkfJlWcDWXe5pM7Tklh+Pz0SY29C8jhpJ2YPIvYtla8jh4gVmllx
UFaf4yiN+ykxjRU1tikWM0yPqwKBgQDsh1INKw/Un3LqHAiXH935G1w1+CFi/Niv
TaUTh+06V14cbcrQ3YVLm+3HIdA/kx9oCDVZOV2rkB/3ezn0yyyuU2ALve12bpZf
Q/ejekH3wPIb3wskDCgWWNZFmx6ptLDyVV2i/yS2BPikC1ai3nb+bWr6wHkfDFzL
Lhz3QzdcaQKBgF8cLRaUZ5MdReeWEXuNV43bzXyjz84lZRJMhCbIQeBbqyF0g52O
Z2Fv5RB8V/5qF9/RTiJ9MrWdEukTdsXm6CjC4uskQTd+wiaypGpdidCkaqUP3OYe
KR+FsDjzbWDKjLdaFH8dtK0w9/h3DjeTVU5wGbES/Fc6nQNlckLklP3PAoGBAMc4
s0OkIZFVO9U0B4hCrxRCl0hd1sEpdtkw7PmmY4ngTMRlNY73Bm7Ix14SfPUPVkSp
pUWhq+P1YPdyMz6cYuXyE9PQBVLbkoxrEwn+z9F+Em6nTxrtolYwPUojfZAzQVbb
0CKaBLHLPiHt6usDtfZW8uSBeYNNRltcW8loNBkBAoGBAIj9NEbjuvXjJbiSWACf
O3SvNemH9EG5eohRecfgjcKvE7SrHuyvRgusfx7Fcajbp1QKdqNMoqfKqMoNiqfU
SVucZvypRuzQON4E9bwlN4Dnkt78rGWFSshZuZ5Fr6DhVtmFCNVNcHxQMAu7qXhP
Chp3C3cLlrfBXevD74QGoXXP
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIEJTCCAw2gAwIBAgIJAMqvMDXuzzLXMA0GCSqGSIb3DQEBCwUAMIGoMQswCQYD
VQQGEwJDWjEXMBUGA1UECAwOQ3plY2ggUmVwdWJsaWMxDTALBgNVBAcMBEJybm8x
GDAWBgNVBAoMD0V4YW1wbGUgUHJvamVjdDEbMBkGA1UECwwSRXhhbXBsZSBQcm9q
ZWN0IENBMRkwFwYDVQQDDBBwa2dzLmV4YW1wbGUub3JnMR8wHQYJKoZIhvcNAQkB
FhBjbGltZUByZWRoYXQuY29tMB4XDTE3MDIxMTEzNTM0MVoXDTE4MDIxMTEzNTM0
MVowgagxCzAJBgNVBAYTAkNaMRcwFQYDVQQIDA5DemVjaCBSZXB1YmxpYzENMAsG
A1UEBwwEQnJubzEYMBYGA1UECgwPRXhhbXBsZSBQcm9qZWN0MRswGQYDVQQLDBJF
eGFtcGxlIFByb2plY3QgQ0ExGTAXBgNVBAMMEHBrZ3MuZXhhbXBsZS5vcmcxHzAd
BgkqhkiG9w0BCQEWEGNsaW1lQHJlZGhhdC5jb20wggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDdAN9IwCYTG+lTHPxSR1/UyqVORieUiBxFNGFA0lwfpT0A
2KXN25oUaPTU3YmXOp4itt44v2eHbK2C+iuyhbR5tw5SS9GH9bTXxWmQv7WgwJi1
Jg1zRWANgkeqe4aHPUm0SCbC14q/BBqAq+zcIJSb4/tN05CQl/KJ6bz459vfj1bR
qnmaJwyClSg2++vnCr2E+F2+rFtJzmx+gOZjd/nrDaUJJrUHgfWDl989QpMlah+y
Lq/Nm7Ie5a25eE+u7Z/H7HHuoPSv54fdq+f4viFAqeJmg5pEEzjDTcxsIAKDPjGl
qrv1F1798ZXw5/L3j5op5M3Zi4NZDmn8NZQsxWEjAgMBAAGjUDBOMB0GA1UdDgQW
BBR0Xs+1RuRmK2feWpT0KHEXYRKytjAfBgNVHSMEGDAWgBR0Xs+1RuRmK2feWpT0
KHEXYRKytjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAtFVzpzHuQ
SvuJNlB3loXDYspInmA6dtOp+rfjKaPX2HOx1VV/NN6lgRTY8oJYajhL/1Q4kr+B
33Vh+DbKnq/tEnQbNHSNI3OMwcWHedZ7SAlsNaPLfxUpRyjAOFTv+6kDgYs5M2VN
uPbY06o+VUg1UQqtaP0PiZ5qlxQeY44Lno6lcG0wACZYNCt/f+MtCyvowNP297dl
HKDijjui/OKrAEab3dXd9HCPOJUDlJoz3DUVzjEBMtw+Xbs9wwfj1zdu4W8pFMip
NujsR824a7Ys5+JcIyJxDWGsMYx6RsBQcBpx/E/40/ehodtpc2RvQEDNdpxiW32w
4oTW+0s+0z3u
-----END CERTIFICATE-----

80
beaker-tests/run.sh Executable file
View file

@ -0,0 +1,80 @@
#!/bin/bash
# vim: dict+=/usr/share/beakerlib/dictionary.vim cpt=.,w,b,u,t,i,k
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# runtest.sh of /tools/dist-git/Regression/
# Description: Test dist-git functionality.
# Author: clime <clime@redhat.com>
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# Copyright (c) 2016 Red Hat, Inc.
#
# This program is free software: you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation, either version 2 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
# PURPOSE. See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/.
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
while [[ $# > 0 ]]
do
key="$1"
case $key in
--nosetup|-x)
nosetup=YES
;;
--runonly|-r)
shift
runonly=$1
;;
*)
echo "Unknown option $key."
exit 1
;;
esac
shift
done
# unnecessary on actual beaker machines but good for local docker testing
if ! rpm -qa | grep -E '^rhts.*' &> /dev/null || ! rpm -qa | grep -E '.*beaker.*' &> /dev/null; then
releasever=`cat /etc/redhat-release | awk '{print $3}'`
sudo dnf -y --repofrompath=beakerrepo,http://beaker-project.org/yum/client/Fedora$releasever/ \
--enablerepo=beakerrepo install rhts-test-env beakerlib
fi
# include Beaker environment
. /usr/bin/rhts-environment.sh || exit 1
. /usr/share/beakerlib/beakerlib.sh || exit 1
PACKAGE="dist-git"
export TESTPATH="$( builtin cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
rlJournalStart
rlPhaseStartSetup
if [[ ! $nosetup ]]; then
./setup.sh
fi
rlPhaseEnd
rlPhaseStartTest Tests
for t in $TESTPATH/tests/*; do
if [[ ! $runonly ]] || echo $runonly | grep `basename $t` &> /dev/null; then
$t/run.sh
fi
done
rlPhaseEnd
rlPhaseStartCleanup
rlPhaseEnd
rlJournalPrintText
rlJournalEnd

56
beaker-tests/setup.sh Executable file
View file

@ -0,0 +1,56 @@
#!/bin/bash
export SCRIPTPATH="$( builtin cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
export LANG=en_US.utf8
# primarily install git for the setup below
dnf -y install git
if [[ `pwd` =~ ^/mnt/tests.*$ ]]; then
echo "Setting up native beaker environment."
git clone https://pagure.io/dist-git.git
export DISTGITROOTDIR=$SCRIPTPATH/dist-git
else
echo "Setting up from source tree."
export DISTGITROOTDIR=$SCRIPTPATH/../
fi
# install files from 'files'
cp -rT $SCRIPTPATH/files /
# install stuff needed for the test
dnf -y install vagrant
dnf -y install vagrant-libvirt
dnf -y install jq
dnf -y --best install fedpkg
dnf -y downgrade fedpkg --allowerasing # FIXME: the dist-git Vagrantfile is not compatible with fedpkg-1.26 due to it not sending server cert
# enable libvirtd for Vagrant (distgit)
systemctl enable libvirtd && systemctl start libvirtd
systemctl start virtlogd.socket # this is currently needed in f25 for vagrant to work with libvirtd
cd $DISTGITROOTDIR
vagrant up distgit
IPADDR=`vagrant ssh -c "ifconfig eth0 | grep -E 'inet\s' | sed 's/\s*inet\s*\([0-9.]*\).*/\1/'"`
echo "$IPADDR pkgs.example.org" >> /etc/hosts
if ! [ -f ~/.ssh/id_rsa ]; then
mkdir -p ~/.ssh && chmod 700 ~/.ssh
ssh-keygen -f ~/.ssh/id_rsa -N '' -q
fi
PUBKEY=`cat ~/.ssh/id_rsa.pub`
vagrant ssh -c "echo $PUBKEY > /tmp/id_rsa.pub.remote"
vagrant ssh -c '
sudo mkdir -p /home/clime/.ssh
sudo touch /home/clime/.ssh/authorized_keys
sudo mv /tmp/id_rsa.pub.remote /home/clime/.ssh/authorized_keys
sudo chown -R clime:clime /home/clime/.ssh
sudo chmod 700 /home/clime/.ssh
sudo chmod 600 /home/clime/.ssh/authorized_keys
' distgit
cd $SCRIPTPATH

BIN
beaker-tests/tests/basic-test/prunerepo-1.1-1.fc23.src.rpm Normal file
View file

Binary file not shown.

56
beaker-tests/tests/basic-test/run.sh Executable file
View file

@ -0,0 +1,56 @@
#!/bin/bash
. /usr/bin/rhts-environment.sh || exit 1
. /usr/share/beakerlib/beakerlib.sh || exit 1
export TESTPATH="$( builtin cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
function pkgs_cmd {
ssh clime@pkgs.example.org $1
}
rlJournalStart
rlPhaseStartSetup BasicTest
pkgs_cmd 'git config --global user.email "clime@redhat.com"' # todo: add this to setup?
pkgs_cmd 'git config --global user.name "clime"' # todo: add this to setup?
pkgs_cmd '/usr/share/dist-git/setup_git_package prunerepo'
rlPhaseEnd
rlPhaseStartTest BasicTest
CWD=`pwd`
cd $TESTPATH
# clone repo using fedpkg
rlRun "fedpkg clone /var/lib/dist-git/git/repositories/prunerepo"
cd prunerepo
# upload into lookaside and working tree update
rlRun "fedpkg import --skip-diffs ../prunerepo-1.1-1.fc23.src.rpm"
# test of presence of the uploaded file
rlRun 'wget http://pkgs.example.org/repo/pkgs/prunerepo/prunerepo-1.1.tar.gz/md5/c5af09c7fb2c05e556898c93c62b1e35/prunerepo-1.1.tar.gz'
# commit of spec and updated sources and push into the git repo
rlRun "git add -A && git commit -m 'test commit'"
rlRun "git push"
# get srpm file using fedpkg
rlRun 'fedpkg --dist f25 srpm'
cd ..
# test git-daemon and git-http-backend by read access
rlRun "git clone git://pkgs.example.org/prunerepo.git prunerepo-copy"
rlRun "git clone http://pkgs.example.org/git/prunerepo prunerepo-copy2"
cd $CWD
rlPhaseEnd
rlPhaseStartCleanup BasicTest
rm -rf $TESTPATH/prunerepo $TESTPATH/prunerepo-copy* $TESTPATH/prunerepo-1.1.tar.gz
pkgs_cmd 'rm -rf /var/lib/dist-git/git/repositories/prunerepo.git'
pkgs_cmd 'sudo rm -rf /var/lib/dist-git/cache/lookaside/pkgs/prunerepo'
rlPhaseEnd
rlJournalEnd &> /dev/null

12
beaker-tests/tests/test-template/run.sh Executable file
View file

@ -0,0 +1,12 @@
#!/bin/bash
. /usr/bin/rhts-environment.sh || exit 1
. /usr/share/beakerlib/beakerlib.sh || exit 1
export TESTPATH="$( builtin cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
rlJournalStart
rlPhaseStartTest TestTemplate
# write your test here
rlPhaseEnd
rlJournalEnd &> /dev/null

76
changes.txt
View file

@ -1,76 +0,0 @@
This file describes the changes with files and paths that has been made during
the separation of this project from the fedora-infra-ansible git.
Renamed files and their use of the main configutation file:
================================================================================
genacls.sh ................. dist_git_sync.sh
genacls.pkgdb .............. pkgdb_gen_gitolite_conf.py <-- Config driven
pkgdb_sync_branches.py ..... pkgdb_sync_git_branches.py <-- Config driven
setup_git_package .......... git_package.sh <-- Needs arguments*
mkbranch ................... git_branch.sh <-- Needs arguments*
make-cgit-pkgs-list.sh ..... gen_cgit_list.sh
* Scripts that needs some arguments to operate with the correct configuration.
Those scripts should be called by the pkgdb_sync_git_branches.py only.
In case we need an admin user to use them, a utility would be made for this
purpouse. Such utility would read the config itself and execute those scripts
with the correct arguments. Example usage would be something like:
$ dist-git add package copr-frontend
$ dist-git add branch epel-7
Files and paths:
================================================================================
configs
├── cron
│   ├── cgit_pkg_list.cron ................ /etc/cron.d/dist-git/
│   └── dist_git_sync.cron ................ /etc/cron.d/dist-git/
├── dist-git
│   └── dist-git.conf ..................... /etc/dist-git/
├── gitolite
│   └── gitolite.rc ....................... /etc/dist-git/
├── httpd
│   ├── dist-git
│   │   ├── git-smart-http.conf ........... /etc/httpd/conf.d/dist-git/
│   │   ├── lookaside.conf ................ /etc/httpd/conf.d/dist-git/
│   │   └── lookaside-upload.conf ......... /etc/httpd/conf.d/dist-git/
│   ├── dist-git.conf ..................... /etc/httpd/conf.d/
│   └── ssl.conf .......................... /etc/httpd/conf.d/
└── systemd
└── dist-git@.service ................. systemd service directory
└── dist-git.socket ................... systemd service directory
scripts
├── dist-git
│   ├── cgit_pkg_list.sh .................. /usr/share/dist-git/
│   ├── dist_git_sync.sh .................. /usr/share/dist-git/
│   ├── git_branch.sh ..................... /usr/share/dist-git/
│   ├── git_package.sh .................... /usr/share/dist-git/
│   ├── pkgdb_gen_gitolite_conf.py ........ /usr/share/dist-git/
│   └── pkgdb_sync_git_branches.py ........ /usr/share/dist-git/
└── httpd
└── upload.cgi ........................ /var/lib/dist-git/web/
places:
git repositories ...................... /var/lib/dist-git/git/rpms/
lookaside cache files ................. /var/lib/dist-git/cache/lookaside/pkgs/
symbolic links:
/etc/dist-git/gitolite.rc ............. /var/lib/dist-git/git/.gitolite.rc
/var/lib/dist-git/gitolite/ ........... /var/lib/dist-git/git/.gitolite/
/var/lib/dist-git/git/rpms/ ........... /var/lib/dist-git/git/repositories/
Excluded files:
================================================================================
cgitrc
clean-lock.cron
fedmsg-genacls-config.py
pkgdb2-clone
redirect.conf
upload_cgi.pp -> SELinux policies will be included in a subpackage
upload_cgi.te -> SELinux policies will be included in a subpackage
updatecrl.sh

3
configs/cron/cgit_pkg_list.cron
View file

@ -1,3 +0,0 @@
# This would allow to refresh the cgit repos every 10 minutes
#
# */10 * * * * root /usr/share/dist-git/cgit_pkg_list.sh

3
configs/cron/dist_git_sync.cron
View file

@ -1,3 +0,0 @@
# Sync the repositories with a Package Database
#
# 02 10 * * * root /usr/share/dist-git/dist_git_sync.sh

66
configs/dist-git/dist-git.conf
View file

@ -1,64 +1,2 @@
# What to do AFTER changing this config:
# 1. /etc/gitolite/admins <- usernames of gitolite admins
# 2. /etc/cron.d/dist-git/dist_git_sync.cron <- enable auto-sync
# /cgit_pkg_list.cron <- enable cgit auto-sync
[acls]
# user groups on the server associated with dist-git
# example:
# user_groups=cvsadmin,fedora-arm,fedora-sparc,fedora-ia64,fedora-s390,fedora-ppc,provenpackager
user_groups=cvsadmin
# user groups with write access to all repositories
# example:
# admin_groups=cvsadmin,fedora-arm,fedora-s390,fedora-ppc
admin_groups=cvsadmin
# comma separated list of active branches to create ACLs for
# example:
# active_branches=olpc2,olpc3,el4,el5,el6,el7,epel7,f11,f12,f13,f14,f15,f16,f17,f18,f19,f20,f21,f22,master
active_branches=
# a "regex"ish list of the reserved branches
# example:
# reserved_branches=f[0-9][0-9],epel[0-9],epel[0-9][0-9],el[0-9],olpc[0-9]
reserved_branches=f[0-9][0-9],epel[0-9],epel[0-9][0-9],el[0-9],olpc[0-9]
# url of package database - acls (data in json format)
# You can leave it empty if you don't want to use it.
# example:
# pkgdb_acls_url=https://admin.fedoraproject.org/pkgdb/api/vcs?format=json
pkgdb_acls_url=
# url of package database - groups (data in json format)
# You can leave it empty if you don't want to use it.
# example:
# pkgdb_groups_url=https://admin.fedoraproject.org/pkgdb/api/groups?format=json
pkgdb_groups_url=
[notifications]
# email notifications:
# example:
# email_domain=fedoraproject.org
email_domain=example.com
# List of email adresses to be notified about changes
# $PACKAGE can be used as a package name
# example:
# pkg_owner_emails=$PACKAGE-owner@fedoraproject.org,scm-commits@lists.fedoraproject.org
pkg_owner_emails=$PACKAGE-nobody@example.com
[git]
# the default author of git branches
# example:
# default_branch_author=Fedora Release Engineering <rel-eng@lists.fedoraproject.org>
default_branch_author=Jerry Random <nobody@example.com>
GITROOT_DIR=/var/lib/dist-git/git
CACHE_DIR=/var/lib/dist-git/cache

196
configs/gitolite/gitolite.rc
View file

@ -1,196 +0,0 @@
# configuration variables for gitolite
# This file is in perl syntax. But you do NOT need to know perl to edit it --
# just mind the commas, use single quotes unless you know what you're doing,
# and make sure the brackets and braces stay matched up!
# (Tip: perl allows a comma after the last item in a list also!)
# HELP for commands can be had by running the command with "-h".
# HELP for all the other FEATURES can be found in the documentation (look for
# "list of non-core programs shipped with gitolite" in the master index) or
# directly in the corresponding source file.
%RC = (
# ------------------------------------------------------------------
# default umask gives you perms of '0700'; see the rc file docs for
# how/why you might change this
UMASK => 0002,
# look for "git-config" in the documentation
GIT_CONFIG_KEYS => '',
# comment out if you don't need all the extra detail in the logfile
LOG_EXTRA => 1,
# syslog options
# 1. leave this section as is for normal gitolite logging
# 2. uncomment this line to log only to syslog:
LOG_DEST => 'syslog',
# 3. uncomment this line to log to syslog and the normal gitolite log:
# LOG_DEST => 'syslog,normal',
# roles. add more roles (like MANAGER, TESTER, ...) here.
# WARNING: if you make changes to this hash, you MUST run 'gitolite
# compile' afterward, and possibly also 'gitolite trigger POST_COMPILE'
ROLES => {
READERS => 1,
WRITERS => 1,
},
# enable caching (currently only Redis). PLEASE RTFM BEFORE USING!!!
# CACHE => 'Redis',
# ------------------------------------------------------------------
# rc variables used by various features
# the 'info' command prints this as additional info, if it is set
# SITE_INFO => 'Please see http://blahblah/gitolite for more help',
# the CpuTime feature uses these
# display user, system, and elapsed times to user after each git operation
# DISPLAY_CPU_TIME => 1,
# display a warning if total CPU times (u, s, cu, cs) crosses this limit
# CPU_TIME_WARN_LIMIT => 0.1,
# the Mirroring feature needs this
# HOSTNAME => "foo",
# TTL for redis cache; PLEASE SEE DOCUMENTATION BEFORE UNCOMMENTING!
# CACHE_TTL => 600,
# ------------------------------------------------------------------
# suggested locations for site-local gitolite code (see cust.html)
# this one is managed directly on the server
LOCAL_CODE => "/var/lib/dist-git/gitolite/local",
# or you can use this, which lets you put everything in a subdirectory
# called "local" in your gitolite-admin repo. For a SECURITY WARNING
# on this, see http://gitolite.com/gitolite/non-core.html#pushcode
# LOCAL_CODE => "$rc{GL_ADMIN_BASE}/local",
# ------------------------------------------------------------------
# List of commands and features to enable
ENABLE => [
# COMMANDS
# These are the commands enabled by default
'help',
'desc',
'info',
'perms',
'writable',
# Uncomment or add new commands here.
# 'create',
# 'fork',
# 'mirror',
# 'readme',
# 'sskm',
# 'D',
# These FEATURES are enabled by default.
# essential (unless you're using smart-http mode)
'ssh-authkeys',
# creates git-config enties from gitolite.conf file entries like 'config foo.bar = baz'
'git-config',
# creates git-daemon-export-ok files; if you don't use git-daemon, comment this out
# 'daemon',
# creates projects.list file; if you don't use gitweb, comment this out
# 'gitweb',
# These FEATURES are disabled by default; uncomment to enable. If you
# need to add new ones, ask on the mailing list :-)
# user-visible behaviour
# prevent wild repos auto-create on fetch/clone
# 'no-create-on-read',
# no auto-create at all (don't forget to enable the 'create' command!)
'no-auto-create',
# access a repo by another (possibly legacy) name
# 'Alias',
# give some users direct shell access. See documentation in
# sts.html for details on the following two choices.
# "Shell $ENV{HOME}/.gitolite.shell-users",
# 'Shell alice bob',
"Shell /etc/gitolite/admins",
# set default roles from lines like 'option default.roles-1 = ...', etc.
# 'set-default-roles',
# show more detailed messages on deny
# 'expand-deny-messages',
# show a message of the day
# 'Motd',
# system admin stuff
# enable mirroring (don't forget to set the HOSTNAME too!)
# 'Mirroring',
# allow people to submit pub files with more than one key in them
# 'ssh-authkeys-split',
# selective read control hack
# 'partial-copy',
# manage local, gitolite-controlled, copies of read-only upstream repos
# 'upstream',
# updates 'description' file instead of 'gitweb.description' config item
'cgit',
# allow repo-specific hooks to be added
'repo-specific-hooks',
# performance, logging, monitoring...
# be nice
# 'renice 10',
# log CPU times (user, system, cumulative user, cumulative system)
# 'CpuTime',
# syntactic_sugar for gitolite.conf and included files
# allow backslash-escaped continuation lines in gitolite.conf
# 'continuation-lines',
# create implicit user groups from directory names in keydir/
# 'keysubdirs-as-groups',
# allow simple line-oriented macros
# 'macros',
# Kindergarten mode
# disallow various things that sensible people shouldn't be doing anyway
# 'Kindergarten',
],
);
# ------------------------------------------------------------------------------
# per perl rules, this should be the last line in such a file:
1;
# Local variables:
# mode: perl
# End:
# vim: set syn=perl:

9
configs/httpd/dist-git/git-smart-http.conf
View file

@ -1,3 +1,10 @@
SetEnv GIT_PROJECT_ROOT /var/lib/dist-git/git/rpms
SetEnv GIT_PROJECT_ROOT /var/lib/dist-git/git/repositories
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
<Directory "/usr/libexec/git-core*">
Options ExecCGI Indexes
Order allow,deny
Allow from all
Require all granted
</Directory>

112
configs/httpd/dist-git/lookaside-upload.conf.example
View file

@ -1,5 +1,3 @@
Alias /repo/ /var/lib/dist-git/cache/lookaside/
# default SSL configuration...
Listen 443
@ -18,95 +16,55 @@ SSLCryptoDevice builtin
ScriptAlias /repo/pkgs/upload.cgi /var/lib/dist-git/web/upload.cgi
Alias /repo/ /var/lib/dist-git/cache/lookaside/
#ServerName pkgs.fedoraproject.org
#ServerAdmin webmaster@fedoraproject.org
ServerName pkgs.example.org
ServerAdmin webmaster@fedoraproject.org
SSLEngine on
# Server Certificate:
# Point SSLCertificateFile at a PEM encoded certificate. If
# the certificate is encrypted, then you will be prompted for a
# pass phrase. Note that a kill -HUP will prompt again. A new
# certificate can be generated using the genkey(1) command.
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateFile /etc/pki/tls/certs/pkgs.example.org.pem
SSLCertificateKeyFile /etc/pki/tls/certs/pkgs.example.org.pem
SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt
#SSLCARevocationFile /etc/pki/tls/certs/crl.pem
# Server Private Key:
# If the key is not combined with the certificate, use this
# directive to point at the key file. Keep in mind that if
# you've both a RSA and a DSA private key you can configure
# both in parallel (to also allow the use of DSA ciphers, etc.)
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
SSLProtocol -All +TLSv1 +TLSv1.1 +TLSv1.2
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK
# Certificate Authority (CA):
# Set the CA certificate verification path where to find CA
# certificates for client authentication or alternatively one
# huge file containing all of them (file must be PEM encoded)
SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt
# Certificate Revocation Lists (CRL):
# Such a file is simply the concatenation of the various
# PEM-encoded CRL files, in order of preference.
# These are used to revoke the client certificate on
# Client Authentication.
SSLCARevocationFile /etc/pki/tls/crl.pem
SSLCipherSuite RSA:!EXPORT:!DH:!LOW:!NULL:+MEDIUM:+HIGH
# Must be 'optional' everywhere in order to have POST operations work to upload.cgi
# Must be 'optional' everywhere in order to have POST operations work to upload.cgi
SSLVerifyClient optional
# Must be here for POST operations to upload.cgi
# Must be here for POST operations to upload.cgi
SSLOptions +OptRenegotiate
ErrorLog logs/ssl_error_log
CustomLog logs/ssl_access_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%{SSL_CLIENT_S_DN_OU}x\" %{SSL_CLIENT_S_DN_CN}x %{SSL_CLIENT_S_DN_emailAddress}x \"%r\" %b"
<Directory /repo/pkgs/>
SSLVerifyClient optional
SSLVerifyDepth 1
SSLOptions +StrictRequire +StdEnvVars +OptRenegotiate
# Access Control:
# With SSLRequire you can do per-directory access control based
# on arbitrary complex boolean expressions containing server
# variable checks and other lookup directives. The syntax is a
# mixture between C and Perl. See the mod_ssl documentation
# for more details.
#
# Following example means:
# require that the access comes from internal or that
# the client auth cert was created by us and signed by us
SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
and %{SSL_CLIENT_S_DN_O} eq "Fedora Project" \
and %{SSL_CLIENT_S_DN_OU} eq "Fedora User Cert" \
and %{SSL_CLIENT_I_DN_O} eq "Fedora Project" \
and %{SSL_CLIENT_I_DN_OU} eq "Fedora Project CA" )
</Directory>
<Directory /repo/pkgs/>
SSLVerifyClient optional
SSLVerifyDepth 1
SSLOptions +StrictRequire +StdEnvVars +OptRenegotiate
# require that the client auth cert was created by us and signed by us
SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
and %{SSL_CLIENT_S_DN_O} eq "Example Project" \
and %{SSL_CLIENT_S_DN_OU} eq "Example User Cert" \
and %{SSL_CLIENT_I_DN_O} eq "Example Project" \
and %{SSL_CLIENT_I_DN_OU} eq "Example Project CA" )
</Directory>
<Location /repo/pkgs/upload.cgi>
SSLRequireSSL
<Location /repo/pkgs/upload.cgi>
SSLRequireSSL
Options +ExecCGI
Require all granted
Options +ExecCGI
Require all granted
SSLVerifyClient optional
SSLVerifyDepth 1
SSLOptions +StrictRequire +StdEnvVars +OptRenegotiate
# Access Control:
# With SSLRequire you can do per-directory access control based
# on arbitrary complex boolean expressions containing server
# variable checks and other lookup directives. The syntax is a
# mixture between C and Perl. See the mod_ssl documentation
# for more details.
#
# Following example means:
# require that the access comes from internal or that
# the client auth cert was created by us and signed by us
SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
and %{SSL_CLIENT_S_DN_O} eq "Fedora Project" \
and %{SSL_CLIENT_S_DN_OU} eq "Fedora User Cert" \
and %{SSL_CLIENT_I_DN_O} eq "Fedora Project" \
and %{SSL_CLIENT_I_DN_OU} eq "Fedora Project CA" )
SSLVerifyClient optional
SSLVerifyDepth 1
SSLOptions +StrictRequire +StdEnvVars +OptRenegotiate
# require that the access comes from internal or that
# the client auth cert was created by us and signed by us
SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
and %{SSL_CLIENT_S_DN_O} eq "Example Project" \
and %{SSL_CLIENT_S_DN_OU} eq "Example User Cert" \
and %{SSL_CLIENT_I_DN_O} eq "Example Project" \
and %{SSL_CLIENT_I_DN_OU} eq "Example Project CA" )
</Location>
</VirtualHost>

1
configs/httpd/dist-git/lookaside.conf
View file

@ -1,4 +1,5 @@
Alias /lookaside /var/lib/dist-git/cache/lookaside
Alias /repo /var/lib/dist-git/cache/lookaside
<Directory /var/lib/dist-git/cache/lookaside>
Options Indexes FollowSymLinks
AllowOverride None

16
configs/httpd/ssl.conf.example
View file

@ -1,16 +0,0 @@
#
# This is the Apache server configuration file providing SSL support.
# It contains the configuration directives to instruct the server how to
# serve pages over an https connection. For detailing information about these
# directives see <URL:http://httpd.apache.org/docs-2.0/mod/mod_ssl.html>
#
# For the moment, see <URL:http://www.modssl.org/docs/> for this info.
# The documents are still being prepared from material donated by the
# modssl project.
#
# Do NOT simply read the instructions in here without understanding
# what they do. They're here only as hints or reminders. If you are unsure
# consult the online docs. You have been warned.
#
LoadModule ssl_module modules/mod_ssl.so

2
configs/systemd/dist-git@.service
View file

@ -5,5 +5,5 @@ Wants=dist-git.socket
[Service]
User=nobody
ExecStart=/usr/libexec/git-core/git-daemon --base-path=/var/lib/dist-git/git/rpms --export-all --user-path=public_git --syslog --inetd --verbose
ExecStart=/usr/libexec/git-core/git-daemon --base-path=/var/lib/dist-git/git/repositories --export-all --user-path=public_git --syslog --inetd --verbose
StandardInput=socket

138
dist-git.spec
View file

@ -1,7 +1,7 @@
%global selinux_variants mls targeted
%global selinux_policyver %(%{__sed} -e 's,.*selinux-policy-\\([^/]*\\)/.*,\\1,' /usr/share/selinux/devel/policyhelp || echo 0.0.0)
%global modulename dist_git
%global installdir /var/lib/dist-git
Name: dist-git
Version: 0.13
@ -10,8 +10,8 @@ Summary: Package source version control system
Group: Applications/Productivity
# upload.cgi uses GPLv1 and pkgdb_sync_git_branches.py uses GPLv2+
License: MIT and GPLv1 and GPLv2+
# upload.cgi uses GPLv1
License: MIT and GPLv1
URL: https://github.com/release-engineering/dist-git
# Source is created by
# git clone https://github.com/release-engineering/dist-git.git
@ -23,13 +23,12 @@ BuildArch: noarch
BuildRequires: systemd
Requires: httpd
Requires: gitolite3
Requires: perl(Sys::Syslog)
Requires: git
Requires: git-daemon
Requires: python-requests
Requires: mod_ssl
Requires: fedmsg
Requires: cronie
Requires(pre): shadow-utils
%description
@ -85,70 +84,38 @@ cd -
getent group packager > /dev/null || \
groupadd -r packager
getent group gen-acls > /dev/null || \
groupadd -r gen-acls
getent passwd gen-acls > /dev/null || \
useradd -r -g gen-acls -G packager -s /bin/bash \
-d %{_sharedstatedir}/dist-git/git gen-acls
%install
# ------------------------------------------------------------------------------
# /usr/share/ .... static files
# /usr/local/bin ........... scripts
# ------------------------------------------------------------------------------
install -d %{buildroot}%{_datadir}/dist-git
install -d %{buildroot}%{_datadir}/dist-git/
cp -a scripts/dist-git/* %{buildroot}%{_datadir}/dist-git/
# ------------------------------------------------------------------------------
# /etc/ .......... config files
# ------------------------------------------------------------------------------
install -d %{buildroot}%{_sysconfdir}/dist-git
cp -a configs/dist-git/dist-git.conf %{buildroot}%{_sysconfdir}/dist-git/
install -d %{buildroot}%{_sysconfdir}/httpd/conf.d/dist-git
install -d %{buildroot}%{_sysconfdir}/cron.d/dist-git
mkdir -p %{buildroot}%{_unitdir}
cp -a configs/dist-git/dist-git.conf %{buildroot}%{_sysconfdir}/dist-git/
cp -a configs/gitolite/gitolite.rc %{buildroot}%{_sysconfdir}/dist-git/
cp -a configs/httpd/dist-git.conf %{buildroot}%{_sysconfdir}/httpd/conf.d/
cp -a configs/httpd/ssl.conf.example %{buildroot}%{_sysconfdir}/httpd/conf.d/
cp -a configs/httpd/dist-git.conf %{buildroot}%{_sysconfdir}/httpd/conf.d/
cp -a configs/httpd/dist-git/* %{buildroot}%{_sysconfdir}/httpd/conf.d/dist-git/
cp -a configs/cron/* %{buildroot}%{_sysconfdir}/cron.d/dist-git/
cp -a configs/systemd/* %{buildroot}%{_unitdir}/
# ------------------------------------------------------------------------------
# /var/lib/ ...... dynamic persistent files
# ------------------------------------------------------------------------------
install -d %{buildroot}%{_sharedstatedir}/dist-git
install -d %{buildroot}%{_sharedstatedir}/dist-git/git
install -d %{buildroot}%{_sharedstatedir}/dist-git/git/rpms
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite/conf
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite/logs
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite/local
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite/local/VREF
install -d %{buildroot}%{_sharedstatedir}/dist-git/cache
install -d %{buildroot}%{_sharedstatedir}/dist-git/cache/lookaside
install -d %{buildroot}%{_sharedstatedir}/dist-git/cache/lookaside/pkgs
install -d %{buildroot}%{_sharedstatedir}/dist-git/web
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite/hooks
install -d %{buildroot}%{_sharedstatedir}/dist-git/gitolite/hooks/common
install -d %{buildroot}%{installdir}
install -d %{buildroot}%{installdir}/git
install -d %{buildroot}%{installdir}/git/repositories
install -d %{buildroot}%{installdir}/cache
install -d %{buildroot}%{installdir}/cache/lookaside
install -d %{buildroot}%{installdir}/cache/lookaside/pkgs
install -d %{buildroot}%{installdir}/web
cp -a scripts/httpd/upload.cgi %{buildroot}%{_sharedstatedir}/dist-git/web/
cp -a scripts/git/hooks/update-block-push-origin \
%{buildroot}%{_sharedstatedir}/dist-git/gitolite/local/VREF/update-block-push-origin
ln -f -s %{_sysconfdir}/dist-git/gitolite.rc \
%{buildroot}%{_sharedstatedir}/dist-git/git/.gitolite.rc
ln -f -s %{_sharedstatedir}/dist-git/gitolite \
%{buildroot}%{_sharedstatedir}/dist-git/git/.gitolite
ln -f -s %{_sharedstatedir}/dist-git/git/rpms \
%{buildroot}%{_sharedstatedir}/dist-git/git/repositories
cp -a scripts/httpd/upload.cgi %{buildroot}%{installdir}/web/
# ------------------------------------------------------------------------------
# SELinux
@ -164,15 +131,18 @@ cd -
/usr/sbin/hardlink -cv %{buildroot}%{_datadir}/selinux
%post selinux
for selinuxvariant in %{selinux_variants}
do
/usr/sbin/semodule -s ${selinuxvariant} -i \
%{_datadir}/selinux/${selinuxvariant}/%{modulename}.pp &> /dev/null || :
done
%{_sbindir}/restorecon -Rv %{_sharedstatedir}/dist-git || :
%{_sbindir}/restorecon -v %{installdir}/cache || :
%{_sbindir}/restorecon -v %{installdir}/cache/lookaside || :
%{_sbindir}/restorecon -v %{installdir}/cache/lookaside/pkgs || :
%{_sbindir}/restorecon -v %{installdir}/git || :
%{_sbindir}/restorecon -v %{installdir}/git/repositories || :
%{_sbindir}/restorecon -Rv %{installdir}/web/ || :
%postun selinux
if [ $1 -eq 0 ] ; then
@ -190,25 +160,15 @@ fi
%license LICENSE
%doc README.md
# ------------------------------------------------------------------------------
# /usr/share/ .... static files
# ------------------------------------------------------------------------------
%dir %{_datadir}/dist-git
%attr (755, -, -) %{_datadir}/dist-git/*
# ------------------------------------------------------------------------------
# /etc/ .......... config files
# ------------------------------------------------------------------------------
%dir %{_sysconfdir}/dist-git
%config(noreplace) %{_sysconfdir}/dist-git/dist-git.conf
%config(noreplace) %{_sysconfdir}/dist-git/gitolite.rc
%config(noreplace) %{_sysconfdir}/httpd/conf.d/dist-git.conf
%config %{_sysconfdir}/httpd/conf.d/ssl.conf.example
%dir %{_sysconfdir}/httpd/conf.d/dist-git
%config(noreplace) %{_sysconfdir}/httpd/conf.d/dist-git/*
%dir %{_sysconfdir}/cron.d/dist-git
%config(noreplace) %{_sysconfdir}/cron.d/dist-git/cgit_pkg_list.cron
%config(noreplace) %{_sysconfdir}/cron.d/dist-git/dist_git_sync.cron
%config(noreplace) %{_sysconfdir}/httpd/conf.d/dist-git.conf
%{_unitdir}/dist-git@.service
%{_unitdir}/dist-git.socket
@ -218,38 +178,21 @@ fi
# non-standard-dir-perm:
# - git repositories and their contents must have w permission for their creators
%dir %{_sharedstatedir}/dist-git
%dir %{_sharedstatedir}/dist-git/git
%attr (2775, -, packager) %{_sharedstatedir}/dist-git/git/rpms
%dir %{_sharedstatedir}/dist-git/gitolite
%attr (755, gen-acls, gen-acls) %{_sharedstatedir}/dist-git/gitolite/conf
# non-standard-dir-perm:
# - write access needed into log directory for gitolite
%attr (775, gen-acls, packager) %{_sharedstatedir}/dist-git/gitolite/logs
%dir %{_sharedstatedir}/dist-git/gitolite/local
# non-standard-dir-perm:
# - write access needed for gitolite admin groups
%attr (775, gen-acls, packager) %{_sharedstatedir}/dist-git/gitolite/local/VREF
# non-standard-executable-perm:
# - write access needed for gitolite admin groups
# - exec permission needed for execution by git (it's a git hook script)
%attr (775, gen-acls, packager) %{_sharedstatedir}/dist-git/gitolite/local/VREF/update-block-push-origin
# non-standard-dir-perm:
# - write access needed for gitolite admin groups
%attr (770, -, packager) %{_sharedstatedir}/dist-git/gitolite/hooks
# script-without-shebang:
# zero-length:
# - initial empty file required by gitolite with the correct perms
%dir %{_sharedstatedir}/dist-git/gitolite/hooks/common
%ghost %attr (775, gen-acls, packager) %{_sharedstatedir}/dist-git/gitolite/hooks/common/update
%dir %{_sharedstatedir}/dist-git/web
%attr (755, apache, apache) %{_sharedstatedir}/dist-git/web/upload.cgi
%dir %{_sharedstatedir}/dist-git/cache
%dir %{_sharedstatedir}/dist-git/cache/lookaside
%attr (775, apache, apache) %{_sharedstatedir}/dist-git/cache/lookaside/pkgs
%{_sharedstatedir}/dist-git/git/repositories
%{_sharedstatedir}/dist-git/git/.gitolite
%{_sharedstatedir}/dist-git/git/.gitolite.rc
%dir %{installdir}
%dir %{installdir}/git
%attr (2775, -, packager) %{installdir}/git/repositories
%dir %{installdir}/web
%attr (755, apache, apache) %{installdir}/web/upload.cgi
%dir %{installdir}/cache
%dir %{installdir}/cache/lookaside
%attr (2775, apache, apache) %{installdir}/cache/lookaside/pkgs
# ------------------------------------------------------------------------------
# /usr/share ...... executable files
# ------------------------------------------------------------------------------
%dir %{_datadir}/dist-git
%attr (775, -, -) %{_datadir}/dist-git/*
%files selinux
@ -264,12 +207,14 @@ fi
- change mv to cp + rm (asamalik@redhat.com)
- update config to not be Fedora specific (asamalik@redhat.com)
- Change: lookaside dir perms + cgit_pkg_list.sh (asamalik@redhat.com)
* Mon Jul 20 2015 Adam Samalik <asamalik@redhat.com> 0.12-1
- config update (asamalik@redhat.com)
- Upload files to new and old paths + remove email (asamalik@redhat.com)
* Tue May 05 2015 Adam Samalik <asamalik@redhat.com> 0.11-1
- SELinux subpackage
* Mon Apr 27 2015 Adam Samalik <asamalik@redhat.com> 0.10-1
- perl require and files update (asamalik@redhat.com)
@ -297,6 +242,7 @@ fi
* Fri Apr 10 2015 Adam Samalik <asamalik@redhat.com> 0.4-1
- spec and config fix
- systemd services
* Tue Mar 31 2015 Adam Samalik <asamalik@redhat.com> 0.3-1
- alpha package (asamalik@redhat.com)

6
scripts/dist-git/cgit_pkg_list.sh → scripts/dist-git/cgit_pkg_list Normal file → Executable file
View file

@ -6,7 +6,9 @@
# Looking for git repos.
#
destination=/var/lib/dist-git/git/pkgs-git-repos-list
source <(grep = /etc/dist-git/dist-git.conf)
destination=$GITROOT_DIR/pkgs-git-repos-list
if [ -n "$1" ]
then
@ -15,7 +17,7 @@ fi
newfile=`mktemp`
cd /var/lib/dist-git/git/rpms
cd $GITROOT_DIR/repositories
ls > $newfile
cp -fZ $newfile $destination
rm $newfile

18
scripts/dist-git/dist_git_sync.sh
View file

@ -1,18 +0,0 @@
#!/bin/sh
python /usr/share/dist-git/pkgdb_sync_git_branches.py
TEMPDIR=`mktemp -d -p /var/tmp genacls.XXXXX`
export GL_BINDIR=/usr/bin
cd $TEMPDIR
# Only replace the acls if genacls completes successfully
if /usr/share/dist-git/pkgdb_gen_gitolite_conf.py > gitolite.conf ; then
mv gitolite.conf /var/lib/dist-git/gitolite/conf/
chown gen-acls:gen-acls -R /var/lib/dist-git/gitolite/conf/
HOME=/var/lib/dist-git/git /usr/bin/gitolite compile
fi
cd /
rm -rf $TEMPDIR
chown root:packager /var/lib/dist-git/gitolite/conf/gitolite.conf-compiled.pm
chmod g+r /var/lib/dist-git/gitolite/conf/gitolite.conf-compiled.pm

147
scripts/dist-git/git_package.sh
View file

@ -1,147 +0,0 @@
#!/bin/bash
#
# Create a new repo.
# THIS HAS TO BE RUN ON THE GIT SERVER!
# WARNING:
# This file is maintained within ansible
# All local changes will be lost.
# Figure out the environment we're running in
GITROOT=/var/lib/dist-git/git/rpms
# check if a moron is driving me
if [ ! -d $GITROOT ] ; then
# we're not on the git server (this check is fragile)
echo "ERROR: This script has to be run on the git server."
echo "ERROR: Homer sez 'Duh'."
exit -9
fi
# Local variables
VERBOSE=0
TEST=
IGNORE=
GIT_SSH_URL="ssh://localhost"
Usage() {
cat <<EOF
Usage:
$0 [OPTIONS] <package_name>
Creates a new repo for <package_name>
Options:
-h,--help This help message
--email-domain DOMAIN Email domain for git hooks.maildomain
--pkg-owner-emails EMAILS Comma separated list of emails for
git hooks.mailinglist
EOF
}
# fail with no arguments
if [ $# -eq 0 ]; then
Usage
exit -1
fi
OPTS=$(getopt -o h -l help -l email-domain: -l pkg-owner-emails: -l default-branch-author: -- "$@")
if [ $? != 0 ]
then
exit 1
fi
eval set -- "$OPTS"
while true ; do
case "$1" in
-h | --help) Usage; exit 0;;
--email-domain) EMAIL_DOMAIN=$2; shift 2;;
--pkg-owner-emails) PKG_OWNER_EMAILS=$2; shift 2;;
--default-branch-author) AUTHOR=$2; shift 2;;
--) shift; break;;
esac
done
# fail when more or none packages are specified
if ! [ $# -eq 1 ]; then
Usage
exit -1
fi
PACKAGE=$1
if [ -z $EMAIL_DOMAIN ]; then
EMAIL_DOMAIN=fedoraproject.org
fi
if [ -z $PKG_OWNER_EMAILS ]; then
PKG_OWNER_EMAILS=$PACKAGE-owner@fedoraproject.org,scm-commits@lists.fedoraproject.org
fi
if [ -z $AUTHOR ]; then
AUTHOR="Fedora Release Engineering <rel-eng@lists.fedoraproject.org>"
fi
# Sanity checks before we start doing damage
[ $VERBOSE -gt 1 ] && echo "Checking package $PACKAGE..."
if [ -f $GITROOT/$PACKAGE.git/refs/heads/master ] ; then
echo "ERROR: Package module $PACKAGE already exists!" >&2
exit -1
fi
# A cleanup in case gitolite came by this repo
if [ -f $GITROOT/$PACKAGE.git/hooks/update ] ; then
echo "Gitolite already initialized this repo. Will remove its hooks"
rm -f $GITROOT/$PACKAGE.git/hooks/update
fi
# "global" permissions check
if [ ! -w $GITROOT ] ; then
echo "ERROR: You can not write to $GITROOT"
echo "ERROR: You can not create repos"
exit -1
fi
# Now start working on creating those branches
# Create a tmpdir to do some git work in
TMPDIR=$(mktemp -d /tmp/tmpXXXXXX)
# First create the master repo
mkdir -p $GITROOT/$PACKAGE.git
pushd $GITROOT/$PACKAGE.git >/dev/null
git init -q --shared --bare
echo "$PACKAGE" > description # This is used to figure out who to send mail to.
git config --add hooks.mailinglist $PKG_OWNER_EMAILS
git config --add hooks.maildomain $EMAIL_DOMAIN
popd >/dev/null
# Now clone that repo and create the .gitignore and sources file
git init -q $TMPDIR/$PACKAGE
pushd $TMPDIR/$PACKAGE >/dev/null
touch .gitignore sources
git add .
git commit -q -m 'Initial setup of the repo' --author "$AUTHOR"
git remote add origin $GITROOT/$PACKAGE.git
git push -q origin master
popd >/dev/null
# Place the gitolite update hook in place since we're not using our own
ln -s /var/lib/dist-git/gitolite/hooks/common/update $GITROOT/$PACKAGE.git/hooks/update
# Setup our post-receive hooks
mkdir -p $GITROOT/$PACKAGE.git/hooks/post-receive-chained.d
ln -s /usr/share/git-core/mail-hooks/gnome-post-receive-email \
$GITROOT/$PACKAGE.git/hooks/post-receive-chained.d/post-receive-email
ln -s /usr/share/git-core/post-receive-fedmsg \
$GITROOT/$PACKAGE.git/hooks/post-receive-chained.d/post-receive-fedmsg
# This one kicks off all the others in post-receive-chained.d
ln -s /usr/share/git-core/post-receive-chained \
$GITROOT/$PACKAGE.git/hooks/post-receive
rm -rf $TMPDIR
echo "Done."

38
scripts/dist-git/git_branch.sh → scripts/dist-git/mkbranch Normal file → Executable file
View file

@ -10,10 +10,12 @@
# Figure out the environment we're running in
RUNDIR=$(cd $(dirname $0) && pwd)
GITROOT=/var/lib/dist-git/git/rpms
source <(grep = /etc/dist-git/dist-git.conf)
REPODIR=$GITROOT_DIR/repositories
# check if a moron is driving me
if [ ! -d $GITROOT ] ; then
if [ ! -d $REPODIR ] ; then
# we're not on the git server (this check is fragile)
echo "ERROR: This script has to be run on the git server."
echo "ERROR: Homer sez 'Duh'."
@ -30,6 +32,7 @@ IGNORE=
BRANCH=""
PACKAGES=""
SRC_BRANCH="master"
AUTHOR="Fedora Release Engineering <rel-eng@lists.fedoraproject.org>"
Usage() {
cat <<EOF
@ -40,9 +43,8 @@ Usage:
The /master suffix on branch names is assumed.
Options:
/master suffix on other branches assumed
-n,--test Don't do nothing, only test
-i,--ignore Ignore erroneous modules
-i,--ignore Ignore erroneous modules
-h,--help This help message
-v,--verbose Increase verbosity
EOF
@ -74,11 +76,6 @@ while [ -n "$1" ] ; do
BRANCH=$1/master
;;
--default-branch-author )
shift
AUTHOR=$1
;;
* )
if [ -z "$BRANCH" ] ; then
BRANCH="$1"
@ -96,20 +93,16 @@ if [ -z "$BRANCH" -o -z "$PACKAGES" ] ; then
exit -1
fi
if [ -z $AUTHOR ]; then
AUTHOR="Fedora Release Engineering <rel-eng@lists.fedoraproject.org>"
fi
# Sanity checks before we start doing damage
NEWP=
for p in $PACKAGES ; do
[ $VERBOSE -gt 1 ] && echo "Checking package $p..."
if [ ! -d $GITROOT/$p.git ] ; then
if [ ! -d $REPODIR/$p.git ] ; then
echo "ERROR: Package module $p is invalid" >&2
[ "$IGNORE" = "yes" ] && continue || exit -1
fi
$(GIT_DIR=$GITROOT/$p.git git rev-parse -q --verify \
$(GIT_DIR=$REPODIR/$p.git git rev-parse -q --verify \
$BRANCH >/dev/null) && \
(echo "IGNORING: Package module $p already has a branch $BRANCH" >&2; \
[ "$IGNORE" = "yes" ] && continue || exit -1)
@ -127,8 +120,8 @@ if [ -n "$TEST" ] ; then
fi
# "global" permissions check
if [ ! -w $GITROOT ] ; then
echo "ERROR: You can not write to $GITROOT"
if [ ! -w $REPODIR ] ; then
echo "ERROR: You can not write to $REPODIR"
echo "ERROR: You can not perform branching operations"
exit -1
fi
@ -141,20 +134,13 @@ for NAME in $PACKAGES ; do
echo "Creating new module branch '$BRANCH' for '$NAME'..."
# permissions checks for this particular module
if [ ! -w $GITROOT/$NAME.git/refs/heads/ ] ; then
if [ ! -w $REPODIR/$NAME.git/refs/heads/ ] ; then
echo "ERROR: You can not write to $d"
echo "ERROR: $NAME can not be branched by you"
continue
fi
#### Replace the above with a gitolite permission check
#[ $VERBOSE -gt 0 ] && echo "Creating $BRANCH-split tag for $NAME/$SRC_BRANCH..."
# Is the above needed?
#cvs -Q rtag -f "$BRANCH-split" $TOPLEVEL/$NAME/$SRC_BRANCH || {
#echo "ERROR: Branch split tag for $NAME/$SRC_BRANCH could not be created" >&2
#exit -2
#}
[ $VERBOSE -gt 0 ] && echo "Creating $NAME $BRANCH from $NAME ..."
$(pushd $GITROOT/$NAME.git >/dev/null && \
$(pushd $REPODIR/$NAME.git >/dev/null && \
git branch --no-track $BRANCH `git rev-list --max-parents=0 master | head -1` && \
popd >/dev/null) || {
echo "ERROR: Branch $NAME $BRANCH could not be created" >&2

153
scripts/dist-git/mkbranch_branching Executable file
View file

@ -0,0 +1,153 @@
#!/bin/bash
#
# Create a new development branch for a module.
# THIS HAS TO BE RUN ON THE GIT SERVER!
# WARNING:
# This file is maintained within puppet?
# All local changes will be lost.
# Figure out the environment we're running in
RUNDIR=$(cd $(dirname $0) && pwd)
source <(grep = /etc/dist-git/dist-git.conf)
REPODIR=$GITROOT_DIR/repositories
# check if a moron is driving me
if [ ! -d $REPODIR ] ; then
# we're not on the git server (this check is fragile)
echo "ERROR: This script has to be run on the git server."
echo "ERROR: Homer sez 'Duh'."
exit -9
fi
# where are the packages kept
TOPLEVEL=rpms
# Local variables
VERBOSE=0
TEST=
IGNORE=
BRANCH=""
PACKAGES=""
SRC_BRANCH="master"
AUTHOR="Fedora Release Engineering <rel-eng@lists.fedoraproject.org>"
Usage() {
cat <<EOF
Usage:
$0 [ -s <src_branch>] <branch> <package_name>...
Creates a new branch <branch> for the list of <package_name>s.
The /master suffix on branch names is assumed.
Options:
-n,--test Don't do nothing, only test
-i,--ignore Ignore erroneous modules
-h,--help This help message
-v,--verbose Increase verbosity
EOF
}
# parse the arguments
while [ -n "$1" ] ; do
case "$1" in
-h | --help )
Usage
exit 0
;;
-v | --verbose )
VERBOSE=$(($VERBOSE + 1))
;;
-i | --ignore )
IGNORE="yes"
;;
-n | --test )
TEST="yes"
;;
-b | --branch )
shift
BRANCH=$1/master
;;
* )
if [ -z "$BRANCH" ] ; then
BRANCH="$1"
else
PACKAGES="$PACKAGES $1"
fi
;;
esac
shift
done
# check the arguments
if [ -z "$BRANCH" -o -z "$PACKAGES" ] ; then
Usage
exit -1
fi
# Sanity checks before we start doing damage
NEWP=
for p in $PACKAGES ; do
[ $VERBOSE -gt 1 ] && echo "Checking package $p..."
if [ ! -d $REPODIR/$p.git ] ; then
echo "ERROR: Package module $p is invalid" >&2
[ "$IGNORE" = "yes" ] && continue || exit -1
fi
$(GIT_DIR=$REPODIR/$p.git git rev-parse -q --verify \
$BRANCH >/dev/null) && \
(echo "IGNORING: Package module $p already has a branch $BRANCH" >&2; \
[ "$IGNORE" = "yes" ] && continue || exit -1)
NEWP="$NEWP $p"
done
PACKAGES="$(echo $NEWP)"
if [ -z "$PACKAGES" ] ; then
echo "NOOP: no valid packages found to process"
exit -1
fi
if [ -n "$TEST" ] ; then
echo "Branch $BRANCH valid for $PACKAGES"
exit 0
fi
# "global" permissions check
if [ ! -w $REPODIR ] ; then
echo "ERROR: You can not write to $REPODIR"
echo "ERROR: You can not perform branching operations"
exit -1
fi
# Now start working on creating those branches
# For every module, "create" the branch
for NAME in $PACKAGES ; do
echo
echo "Creating new module branch '$BRANCH' for '$NAME'..."
# permissions checks for this particular module
if [ ! -w $REPODIR/$NAME.git/refs/heads/ ] ; then
echo "ERROR: You can not write to $d"
echo "ERROR: $NAME can not be branched by you"
continue
fi
[ $VERBOSE -gt 0 ] && echo "Creating $NAME $BRANCH from $NAME ..."
$(pushd $REPODIR/$NAME.git >/dev/null && \
git branch --no-track $BRANCH `git rev-list master | head -1` && \
popd >/dev/null) || {
echo "ERROR: Branch $NAME $BRANCH could not be created" >&2
popd >/dev/null
exit -2
}
done
echo
echo "Done."

122
scripts/dist-git/pkgdb_gen_gitolite_conf.py
View file

@ -1,122 +0,0 @@
#!/usr/bin/python -t
#
# Create an /etc/gitolog/conf/getolog.conf file with acls for dist-git
#
# Takes no arguments!
#
import grp
import sys
import requests
from ConfigParser import ConfigParser
def _get_conf(cp, section, option, default):
if cp.has_section(section) and cp.has_option(section, option):
return cp.get(section, option)
return default
if __name__ == '__main__':
config = ConfigParser()
config.read("/etc/dist-git/dist-git.conf")
user_groups = _get_conf(config, "acls", "user_groups", "").split(",")
admin_groups = _get_conf(config, "acls", "admin_groups", "").split(",")
ACTIVE = _get_conf(config, "acls", "active_branches", "").split(",")
RESERVED = _get_conf(config, "acls", "reserved_branches", "").split(",")
pkgdb_acls_url = _get_conf(config, "acls", "pkgdb_acls_url", "")
pkgdb_groups_url = _get_conf(config, "acls", "pkgdb_groups_url", "")
# Read the ACL information from the packageDB
data = {"packageAcls":{}}
if pkgdb_acls_url:
data = requests.get(pkgdb_acls_url).json()
# Get a list of all the packages
acls = data['packageAcls']
pkglist = data['packageAcls'].keys()
pkglist.sort()
# sanity check
#if len(pkglist) < 2500:
# sys.exit(1)
# get the list of all groups
pkgdb_groups = {"groups":[]}
if pkgdb_groups_url:
pkgdb_groups = requests.get(pkgdb_groups_url).json()
# print out our user groups
for group in user_groups + pkgdb_groups["groups"]:
print "@{0} = {1}".format(group, " ".join(grp.getgrnam(group)[3]))
# Give a little space before moving onto the permissions
print ''
# print our default permissions
print 'repo @all'
print ' - VREF/update-block-push-origin = @all'
if admin_groups:
print ' RWC = @{}'.format(" @".join(admin_groups))
print ' R = @all'
#print ' RW private- = @all'
# dont' enable the above until we prevent building for real from private-
for pkg in pkglist:
branchAcls = {} # Check whether we need to set separate per branch acls
buffer = [] # Buffer the output per package
masters = [] # Folks that have commit to master
writers = [] # Anybody that has write access
# Examine each branch in the package
branches = acls[pkg].keys()
branches.sort()
for branch in branches:
if not branch in ACTIVE:
continue
if 'packager' in acls[pkg][branch]['commit']['groups']:
# If the packager group is defined, everyone has access
buffer.append(' RWC %s = @all' % (branch))
branchAcls.setdefault('@all', []).append((pkg, branch))
if branch == 'master':
masters.append('@all')
if '@all' not in writers:
writers.append('@all')
else:
# Extract the owners
committers = []
owners = acls[pkg][branch]['commit']['people']
owners.sort()
for owner in owners:
committers.append(owner)
for group in acls[pkg][branch]['commit']['groups']:
committers.append('@%s' % group)
if branch == 'master':
masters.extend(committers)
# add all the committers to the top writers list
for committer in committers:
if not committer in writers:
writers.append(committer)
# Print the committers to the acl for this package-branch
committers = ' '.join(committers)
buffer.append(' RWC %s = %s' %
(branch, committers))
branchAcls.setdefault(committers, []).append((pkg, branch))
print
print 'repo %s' % pkg
#if len(branchAcls.keys()) == 1:
# acl = branchAcls.keys()[0]
# print ' RW = %s' % acl
#else:
print '\n'.join(buffer)
for reserved in RESERVED:
print ' - %s = @all' % reserved
print ' RWC refs/tags/ = %s' % ' '.join(writers)
if masters:
print ' RWC = %s' % ' '.join(masters)
sys.exit(0)

269
scripts/dist-git/pkgdb_sync_git_branches.py
View file

@ -1,269 +0,0 @@
#!/usr/bin/python -tt
# -*- coding: utf-8 -*-
"""
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
This script is able to query pkgdb and retrieve for all packages which active
branches should be there, browse all the git repos and find out which active
branches are missing.
It even goes one step further but actually adjusting the git repo by adding
the missing branches (or even the missing repo)
"""
import multiprocessing.pool
import os
import subprocess
import time
import pipes
import requests
from ConfigParser import ConfigParser
# Do some off-the-bat configuration of fedmsg.
# 1) since this is a one-off script and not a daemon, it needs to connect
# to the fedmsg-relay process running on another node (or noone will
# hear it)
# 2) its going to use the 'shell' certificate which only 'sysadmin' has
# read access to. Contrast that with the 'scm' certificate which
# everyone in the 'packager' group has access to.
def _get_conf(cp, section, option, default):
if cp.has_section(section) and cp.has_option(section, option):
return cp.get(section, option)
return default
config = ConfigParser()
config.read("/etc/dist-git/dist-git.conf")
PKGDB_URL = _get_conf(config, "acls", "pkgdb_acls_url", "")
EMAIL_DOMAIN = _get_conf(config, "notifications", "email_domain", "fedoraproject.org")
PKG_OWNER_EMAILS = _get_conf(config, "notifications", "pkg_owner_emails",
"$PACKAGE-owner@fedoraproject.org,scm-commits@lists.fedoraproject.org")
DEFAULT_BRANCH_AUTHOR = _get_conf(config, "git", "default_branch_author",
"Fedora Release Engineering <rel-eng@lists.fedoraproject.org>")
GIT_FOLDER = '/var/lib/dist-git/git/rpms/'
MKBRANCH = '/usr/share/dist-git/git_branch.sh'
SETUP_PACKAGE = '/usr/share/dist-git/git_package.sh'
THREADS = 20
VERBOSE = False
class InternalError(Exception):
pass
class ProcessError(InternalError):
pass
def _invoke(program, args, cwd=None):
'''Run a command and raise an exception if an error occurred.
:arg program: The program to invoke
:args: List of arguments to pass to the program
raises ProcessError if there's a problem.
'''
cmdLine = [program]
cmdLine.extend(args)
if VERBOSE:
print ' '.join(cmdLine)
print ' in', cwd
program = subprocess.Popen(
cmdLine, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=cwd)
stdout, stderr = program.communicate()
if program.returncode != 0:
e = ProcessError()
e.returnCode = program.returncode
e.cmd = ' '.join(cmdLine)
e.cwd = cwd
e.message = 'Error, "%s" (in %r) returned %s\n stdout: %s\n stderr: %s' % (
e.cmd, e.cwd, e.returnCode, stdout, stderr)
print e.message
raise e
return stdout.strip()
def _create_branch(pkgname, branch, existing_branches):
'''Create a specific branch for a package.
:arg pkgname: Name of the package to branch
:arg branch: Name of the branch to create
:arg existing_branches: A list of the branches that already exist locally.
'''
branch = branch.replace('*', '').strip()
if branch == 'master':
print 'ERROR: Proudly refusing to create master branch. Invalid repo?'
print 'INFO: Please check %s repo' % pkgname
return
if branch in existing_branches:
print 'ERROR: Refusing to create a branch %s that exists' % branch
return
try:
_invoke(MKBRANCH, ["--default-branch-author", pipes.quote(DEFAULT_BRANCH_AUTHOR),
branch, pkgname])
except ProcessError, e:
if e.returnCode == 255:
# This is a warning, not an error
return
raise
def pkgdb_pkg_branch():
""" Queries pkgdb information about VCS and return a dictionnary of
which branches are available for which packages.
:return: a dict[pkg_name] = [pkg_branches]
:rtype: dict
"""
data = {"packageAcls": {}}
if PKGDB_URL:
data = requests.get(PKGDB_URL).json()
output = {}
for pkg in data['packageAcls']:
if pkg in output:
if VERBOSE:
print 'Strange package: %s, it is present twice in the ' \
'pkgdb output' % pkg
output[pkg].updated(data['packageAcls'][pkg].keys())
else:
output[pkg] = set(data['packageAcls'][pkg].keys())
return output
def get_git_branch(pkg):
""" For the specified package name, check the local git and return the
list of branches found.
"""
git_folder = os.path.join(GIT_FOLDER, '%s.git' % pkg)
if not os.path.exists(git_folder):
if VERBOSE:
print 'Could not find %s' % git_folder
return set()
branches = [
lclbranch.replace('*', '').strip()
for lclbranch in _invoke('git', ['branch'], cwd=git_folder).split('\n')
]
return set(branches)
def branch_package(pkgname, requested_branches, existing_branches):
'''Create all the branches that are listed in the pkgdb for a package.
:arg pkgname: The package to create branches for
:arg requested_branches: The branches to creates
:arg existing_branches: A list of existing local branches
'''
if VERBOSE:
print 'Fixing package %s for branches %s' % (pkgname, requested_branches)
# Create the devel branch if necessary
exists = os.path.exists(os.path.join(GIT_FOLDER, '%s.git' % pkgname))
if not exists or 'master' not in existing_branches:
emails = PKG_OWNER_EMAILS.replace("$PACKAGE", pkgname)
_invoke(SETUP_PACKAGE, ["--pkg-owner-emails", pipes.quote(emails),
"--email-domain", pipes.quote(EMAIL_DOMAIN),
"--default-branch-author", pipes.quote(DEFAULT_BRANCH_AUTHOR),
pkgname])
if 'master' in requested_branches:
requested_branches.remove('master') # SETUP_PACKAGE creates master
# Create all the required branches for the package
# Use the translated branch name until pkgdb falls inline
for branch in requested_branches:
_create_branch(pkgname, branch, existing_branches)
def main():
""" For each package found via pkgdb, check the local git for its
branches and fix inconsistencies.
"""
local_pkgs = set(os.listdir(GIT_FOLDER))
local_pkgs = set([it.replace('.git', '') for it in local_pkgs])
if VERBOSE:
print "Found %i local packages" % len(local_pkgs)
pkgdb_info = pkgdb_pkg_branch()
pkgdb_pkgs = set(pkgdb_info.keys())
if VERBOSE:
print "Found %i pkgdb packages" % len(pkgdb_pkgs)
## Commented out as we keep the git of retired packages while they won't
## show up in the information retrieved from pkgdb.
#if (local_pkgs - pkgdb_pkgs):
#print 'Some packages are present locally but not on pkgdb:'
#print ', '.join(sorted(local_pkgs - pkgdb_pkgs))
if (pkgdb_pkgs - local_pkgs):
print 'Some packages are present in pkgdb but not locally:'
print ', '.join(sorted(pkgdb_pkgs - local_pkgs))
if VERBOSE:
print "Finding the lists of local branches for local repos."
start = time.time()
if THREADS == 1:
git_branch_lookup = map(get_git_branch, sorted(pkgdb_info))
else:
threadpool = multiprocessing.pool.ThreadPool(processes=THREADS)
git_branch_lookup = threadpool.map(get_git_branch, sorted(pkgdb_info))
# Zip that list of results up into a lookup dict.
git_branch_lookup = dict(zip(sorted(pkgdb_info), git_branch_lookup))
if VERBOSE:
print "Found all local git branches in %0.2fs" % (time.time() - start)
tofix = set()
for pkg in sorted(pkgdb_info):
pkgdb_branches = pkgdb_info[pkg]
git_branches = git_branch_lookup[pkg]
diff = (pkgdb_branches - git_branches)
if diff:
print '%s missing: %s' % (pkg, ','.join(sorted(diff)))
tofix.add(pkg)
branch_package(pkg, diff, git_branches)
if tofix:
print 'Packages fixed (%s): %s' % (
len(tofix), ', '.join(sorted(tofix)))
else:
if VERBOSE:
print 'Didn\'t find any packages to fix.'
if __name__ == '__main__':
import sys
sys.exit(main())

117
scripts/dist-git/setup_git_package Executable file
View file

@ -0,0 +1,117 @@
#!/bin/bash
#
# Create a new repo.
# THIS HAS TO BE RUN ON THE GIT SERVER!
# WARNING:
# This file is maintained within ansible
# All local changes will be lost.
# Figure out the environment we're running in
source <(grep = /etc/dist-git/dist-git.conf)
REPODIR=$GITROOT_DIR/repositories
# check if a moron is driving me
if [ ! -d $REPODIR ] ; then
# we're not on the git server (this check is fragile)
echo "ERROR: This script has to be run on the git server."
echo "ERROR: Homer sez 'Duh'."
exit -9
fi
# Local variables
VERBOSE=0
TEST=
IGNORE=
AUTHOR="Fedora Release Engineering <rel-eng@lists.fedoraproject.org>"
GIT_SSH_URL="ssh://localhost"
Usage() {
cat <<EOF
Usage:
$0 <package_name>
Creates a new repo for <package_name>
Options:
-h,--help This help message
EOF
}
while [[ $# > 0 ]]
do
key="$1"
case $key in
--help|-h)
Usage
exit 0
;;
*)
PACKAGE="$1"
;;
esac
shift
done
# check the arguments
if [ -z "$PACKAGE" ] ; then
Usage
exit -1
fi
# Sanity checks before we start doing damage
[ $VERBOSE -gt 1 ] && echo "Checking package $PACKAGE..."
if [ -f $REPODIR/$PACKAGE.git/refs/heads/master ] ; then
echo "ERROR: Package module $PACKAGE already exists!" >&2
exit -1
fi
# "global" permissions check
if [ ! -w $REPODIR ] ; then
echo "ERROR: You can not write to $REPODIR"
echo "ERROR: You can not create repos"
exit -1
fi
# Now start working on creating those branches
# Create a tmpdir to do some git work in
TMPDIR=$(mktemp -d /tmp/tmpXXXXXX)
# First create the master repo
mkdir -p $REPODIR/$PACKAGE.git
pushd $REPODIR/$PACKAGE.git >/dev/null
git init -q --shared --bare
echo "$PACKAGE" > description # This is used to figure out who to send mail to.
git config --add hooks.mailinglist "$PACKAGE-owner@fedoraproject.org,scm-commits@lists.fedoraproject.org"
git config --add hooks.maildomain fedoraproject.org
popd >/dev/null
# Now clone that repo and create the .gitignore and sources file
git init -q $TMPDIR/$PACKAGE
pushd $TMPDIR/$PACKAGE >/dev/null
touch .gitignore sources
git add .
git commit -q -m 'Initial setup of the repo' --author "$AUTHOR"
git remote add origin $REPODIR/$PACKAGE.git
git push -q origin master
popd >/dev/null
# Place the gitolite update hook in place since we're not using our own
ln -s /etc/gitolite/hooks/common/update $REPODIR/$PACKAGE.git/hooks/update
# Setup our post-receive hooks
mkdir -p $REPODIR/$PACKAGE.git/hooks/post-receive-chained.d
ln -s /usr/share/git-core/mail-hooks/gnome-post-receive-email \
$REPODIR/$PACKAGE.git/hooks/post-receive-chained.d/post-receive-email
ln -s /usr/share/git-core/post-receive-fedmsg \
$REPODIR/$PACKAGE.git/hooks/post-receive-chained.d/post-receive-fedmsg
ln -s /usr/share/git-core/post-receive-alternativearch \
$REPODIR/$PACKAGE.git/hooks/post-receive-chained.d/post-receive-alternativearch
# This one kicks off all the others in post-receive-chained.d
ln -s /usr/share/git-core/post-receive-chained \
$REPODIR/$PACKAGE.git/hooks/post-receive
rm -rf $TMPDIR
echo "Done."

15
scripts/git/hooks/update-block-push-origin
View file

@ -1,15 +0,0 @@
#!/bin/sh
#
# Block pushes to branches if their name starts with `origin/`
# https://fedorahosted.org/rel-eng/ticket/4071
refname="${1}"
echo "${refname}" | grep -qE '^refs/heads/origin/'
if [ $? == 0 ]; then
echo "Can't push a branch named $refname"
exit 1
fi
exit 0

197
scripts/httpd/upload.cgi
View file

@ -6,34 +6,23 @@
#
# License: GPL
import cgi
import errno
import grp
import hashlib
import os
import sys
import cgi
import tempfile
import grp
import pwd
import syslog
import smtplib
import errno
import fedmsg
import fedmsg.config
from ConfigParser import ConfigParser
from email import Header, Utils
try:
from email.mime.text import MIMEText
except ImportError:
from email.MIMEText import MIMEText
import hashlib
# Reading buffer size
BUFFER_SIZE = 4096
# We check modules exist from this dircetory
GITREPO = '/var/lib/dist-git/git/rpms'
GITREPO = '/var/lib/dist-git/git/repositories'
# Lookaside cache directory
CACHE_DIR = '/var/lib/dist-git/cache/lookaside/pkgs'
@ -41,21 +30,35 @@ CACHE_DIR = '/var/lib/dist-git/cache/lookaside/pkgs'
# Fedora Packager Group
PACKAGER_GROUP = 'packager'
# dist git configuration file
CONFIG_FILE = "/etc/dist-git/dist-git.conf"
def send_error(text):
def send_error(text, status='500 Internal Server Error'):
"""Send an error back to the client
This ensures that the client will get a proper error, including the HTTP
status code, so that it can handle problems appropriately.
Args:
text (str): The error message to send the client
status (str, optional): The HTTP status code to return to the client.
"""
print 'Status: %s' % status
print 'Content-type: text/plain'
print
print text
sys.exit(1)
sys.exit(0)
def check_form(form, var):
ret = form.getvalue(var, None)
if ret is None:
send_error('Required field "%s" is not present.' % var)
send_error('Required field "%s" is not present.' % var,
status='400 Bad Request')
if isinstance(ret, list):
send_error('Multiple values given for "%s". Aborting.' % var)
send_error('Multiple values given for "%s". Aborting.' % var,
status='400 Bad Request')
return ret
def check_auth(username):
authenticated = False
try:
@ -65,22 +68,45 @@ def check_auth(username):
pass
return authenticated
def hardlink(src, dest, username):
makedirs(os.path.dirname(dest), username)
try:
os.link(src, dest)
except OSError as e:
if e.errno != errno.EEXIST:
send_error(str(e))
# The file already existed at the dest path, hardlink over it
os.unlink(dest)
os.link(src, dest)
sys.stderr.write("[username=%s] ln %s %s\n" % (username, src, dest))
def makedirs(dir_, username, mode=02755):
try:
os.makedirs(dir_, mode=mode)
sys.stderr.write('[username=%s] mkdir %s\n' % (username, dir_))
except OSError as e:
if e.errno != errno.EEXIST:
send_error(str(e))
def main():
config = ConfigParser()
config.read(CONFIG_FILE)
EMAIL_DOMAIN = _get_conf(config, "notifications", "email_domain", "")
PKG_OWNER_EMAILS = _get_conf(config, "notifications", "pkg_owner_emails", "")
os.umask(002)
username = os.environ.get('SSL_CLIENT_S_DN_CN', None)
gssname = os.environ.get('GSS_NAME', None)
if gssname and '@' in gssname and not username:
username = gssname.partition('@')[0]
if not check_auth(username):
print 'Status: 403 Forbidden'
print 'Content-type: text/plain'
print
print 'You must connect with a valid certificate and be in the %s group to upload.' % PACKAGER_GROUP
sys.exit(0)
send_error('You must connect with a valid certificate and be in the '
'%s group to upload.' % PACKAGER_GROUP,
status='403 Forbidden')
print 'Content-Type: text/plain'
print
@ -91,17 +117,18 @@ def main():
name = check_form(form, 'name')
# Search for the file hash, start with stronger hash functions
if form.has_key('sha512sum'):
if 'sha512sum' in form:
checksum = check_form(form, 'sha512sum')
hash_type = "sha512"
elif form.has_key('md5sum'):
elif 'md5sum' in form:
# Fallback on md5, as it's what we currently use
checksum = check_form(form, 'md5sum')
hash_type = "md5"
else:
send_error('Required checksum is not present.')
send_error('Required checksum is not present.',
status='400 Bad Request')
action = None
upload_file = None
@ -110,38 +137,47 @@ def main():
# Is this a submission or a test?
# in a test, we don't get a file, just a filename.
# In a submission, we don;t get a filename, just the file.
if form.has_key('filename'):
if 'filename' in form:
action = 'check'
filename = check_form(form, 'filename')
filename = os.path.basename(filename)
print >> sys.stderr, '[username=%s] Checking file status: NAME=%s FILENAME=%s %sSUM=%s' % (username, name, filename, hash_type.upper(), checksum)
sys.stderr.write('[username=%s] Checking file status: NAME=%s '
'FILENAME=%s %sSUM=%s\n' % (username, name, filename,
hash_type.upper(),
checksum))
else:
action = 'upload'
if form.has_key('file'):
if 'file' in form:
upload_file = form['file']
if not upload_file.file:
send_error('No file given for upload. Aborting.')
send_error('No file given for upload. Aborting.',
status='400 Bad Request')
filename = os.path.basename(upload_file.filename)
else:
send_error('Required field "file" is not present.')
print >> sys.stderr, '[username=%s] Processing upload request: NAME=%s FILENAME=%s %sSUM=%s' % (username, name, filename, hash_type.upper(), checksum)
send_error('Required field "file" is not present.',
status='400 Bad Request')
sys.stderr.write('[username=%s] Processing upload request: '
'NAME=%s FILENAME=%s %sSUM=%s\n' % (
username, name, filename, hash_type.upper(),
checksum))
module_dir = os.path.join(CACHE_DIR, name)
hash_dir = os.path.join(module_dir, filename, hash_type, checksum)
msgpath = os.path.join(name, module_dir, filename, hash_type, checksum, filename)
unwanted_prefix = '/var/lib/dist-git/cache/lookaside/pkgs/'
if msgpath.startswith(unwanted_prefix):
msgpath = msgpath[len(unwanted_prefix):]
msgpath = os.path.join(name, filename, hash_type, checksum, filename)
# first test if the module really exists
git_dir = os.path.join(GITREPO, '%s.git' % name)
git_dir = os.path.join(GITREPO, '%s.git' % name)
if not os.path.isdir(git_dir):
print >> sys.stderr, '[username=%s] Unknown module: %s' % (username, name)
send_error('Module "%s" does not exist!' % name)
sys.stderr.write('[username=%s] Unknown module: %s' % (username, name))
send_error('Module "%s" does not exist!' % name,
status='404 Not Found')
# try to see if we already have this file...
dest_file = os.path.join(hash_dir, filename)
old_dir = os.path.join(module_dir, filename, checksum)
old_path = os.path.join(old_dir, filename)
if os.path.exists(dest_file):
if action == 'check':
print 'Available'
@ -152,12 +188,17 @@ def main():
print 'File: %s Size: %d' % (dest_file, dest_file_stat.st_size)
sys.exit(0)
elif action == 'check':
print 'Missing'
if os.path.exists(old_path):
# The file had been uploaded at the old path
hardlink(old_path, dest_file, username)
print 'Available'
else:
print 'Missing'
sys.exit(0)
# check that all directories are in place
if not os.path.isdir(module_dir):
os.makedirs(module_dir, 02775)
makedirs(module_dir, username)
# grab a temporary filename and dump our file in there
tempfile.tempdir = module_dir
@ -180,41 +221,24 @@ def main():
check_checksum = m.hexdigest()
if checksum != check_checksum:
os.unlink(tmpfile)
send_error("%s check failed. Received %s instead of %s." % (hash_type.upper(), check_checksum, checksum))
send_error("%s check failed. Received %s instead of %s." %
(hash_type.upper(), check_checksum, checksum),
status='400 Bad Request')
# wow, even the checksum matches. make sure full path is valid now
if not os.path.isdir(hash_dir):
os.makedirs(hash_dir, 02775)
print >> sys.stderr, '[username=%s] mkdir %s' % (username, hash_dir)
makedirs(hash_dir, username)
os.rename(tmpfile, dest_file)
os.chmod(dest_file, 0644)
print >> sys.stderr, '[username=%s] Stored %s (%d bytes)' % (username, dest_file, filesize)
print 'File %s size %d %s %s stored OK' % (filename, filesize, hash_type.upper(), checksum)
sys.stderr.write('[username=%s] Stored %s (%d bytes)' % (username,
dest_file,
filesize))
print 'File %s size %d %s %s stored OK' % (filename, filesize,
hash_type.upper(), checksum)
# Add the file to the old path, where fedpkg is currently looking for it
if hash_type == "md5":
old_dir = os.path.join(module_dir, filename, checksum)
old_path = os.path.join(old_dir, filename)
try:
os.makedirs(old_dir)
except OSError as e:
if e.errno != errno.EEXIST:
raise e
try:
os.link(dest_file, old_path)
except OSError as e:
if e.errno != errno.EEXIST:
raise e
# The file already existed at the old path, hardlink over it
os.unlink(old_path)
os.link(old_path)
hardlink(dest_file, old_path, username)
# Emit a fedmsg message. Load the config to talk to the fedmsg-relay.
try:
@ -224,11 +248,18 @@ def main():
fedmsg.init(name="relay_inbound", cert_prefix="lookaside", **config)
topic = "lookaside.new"
msg = dict(name=name, md5sum=checksum, filename=filename.split('/')[-1],
agent=username, path=msgpath)
msg = dict(name=name, md5sum=checksum,
filename=filename.split('/')[-1], agent=username,
path=msgpath)
fedmsg.publish(modname="git", topic=topic, msg=msg)
except Exception as e:
print "Error with fedmsg", str(e)
if __name__ == '__main__':
main()
try:
main()
except Exception as e:
import traceback
sys.stderr.write('%s\n' % traceback.format_exc())
send_error(str(e))

2
selinux/dist_git.fc
View file

@ -1,4 +1,4 @@
/var/lib/dist-git/git(/.*)? gen_context(system_u:object_r:git_user_content_t,s0)
/var/lib/dist-git/git/pkgs-git-repos-list gen_context(system_u:object_r:git_content_t,s0)
/var/lib/dist-git/cache(/.*)? gen_context(system_u:object_r:git_rw_content_t,s0)
/var/lib/dist-git/web/upload.cgi gen_context(system_u:object_r:git_script_exec_t,s0)

15
selinux/dist_git.te
View file

@ -1,9 +1,12 @@
policy_module(dist_git,1.0.0)
policy_module(dist_git,1.0.1)
require {
type httpd_git_script_t;
type git_script_tmp_t;
type git_system_t;
type git_user_content_t;
type httpd_t;
}
files_tmp_file(git_script_tmp_t);
@ -24,3 +27,13 @@ term_getattr_all_ptys(httpd_git_script_t);
term_getattr_all_ttys(httpd_git_script_t);
# Do not audit attempts to get the attributes of generic pty devices.
term_dontaudit_getattr_generic_ptys(httpd_git_script_t);
# For git-daemon
allow git_system_t git_user_content_t:dir { search getattr open read };
allow git_system_t git_user_content_t:file { read open getattr };
allow git_system_t git_user_content_t:lnk_file { read open getattr };
# For git-http-backend
allow httpd_t git_user_content_t:dir { search getattr open read };
allow httpd_t git_user_content_t:file { read open getattr };
allow httpd_t git_user_content_t:lnk_file { read open getattr };