Commit graph

84 commits

Author SHA1 Message Date
Michel Alexandre Salim
ff25e84718
lookaside_upload_sig: fix checksum reporting
Instead of calling the checksum the 'sha1sum', actually report the
hash parameter used

Signed-off-by: Michel Alexandre Salim <salimma@centosproject.org>
2023-03-03 11:22:40 -06:00
Fabian Arrotin
ff65ce6b4a Fixed lookaside_upload_sig for missing and wrong variables
Signed-off-by: Fabian Arrotin <arrfab@centos.org>
2022-05-04 14:18:06 +02:00
Adam Piasecki
792d9ae960 Modified comment section 2022-04-22 09:06:50 +01:00
Adam Piasecki
050ddddbeb Removed the branch element from the script and forced use of SHA512 algorithm 2022-04-21 21:56:51 +01:00
Adam Piasecki
bb01e710ed Modified lookaside_upload_sig file based on received feedback 2022-04-05 17:15:17 +01:00
Adam Piasecki
2918905b9b Implemented suggested changes to lookaside_upload_sig 2022-04-04 19:04:16 +01:00
Adam Piasecki
03f8bc4c7e Created simple lookaside_upload_sig 2022-04-04 11:39:26 +01:00
Pierre-Yves Chibon
5480ad2988 Add a little debugging info to the get_sources script
This will tell which part of the script is being considered when it is
retrieving the sources from the lookaside cache.

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2022-03-03 09:56:28 +01:00
Pierre-Yves Chibon
04de770669 Adjust get_sources.sh to allow empty sources file
There are situations in which one will want to use a `sources` file to
indicate that they wish to use the flat dist-git layout.
However, until now we did not allow empty `sources` file in flat dist-git
layout.
With this change we allow empty `sources` file and we will just echo
something in the logs saying that this file is empty and bail.

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2022-03-03 09:53:43 +01:00
Pierre-Yves Chibon
014e3890be Add support for a different lookaside structure
There is a will to offer SIGs the possibility to use a similar lookaside
structure as the one used by CentOS Stream and Fedora which is not tied
to the name of the archive but to its hash.
The idea though is to offer the new structure as opt-in and thus keep
the old structure working.
We thus need to adjust the get_sources.sh script to support both
structure.
This commit makes it so, it allows both exploded-srpm and flat dist-git
structures to use either the old or new lookaside cache structure. The
way this is achieved is simply to first call the URL corresponding to
the old lookaside structure. If that call returns a http code in the 200
range, then the script stops, otherwise, the script will call the URL
corresponding to the new lookaside structure.

This commit also makes consistent the different curl calls and add
--retry 5 on all of them (which does not work for 404 replies, so
using the new structure will not results in 6 requests to the old one
before moving on, but just 1).

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2022-03-03 09:52:58 +01:00
Fabian Arrotin
ae9a7e62f9 Backported alphacc's fix for el7/cbs and bash 4.2 (ca63b53c8b)
Signed-off-by: Fabian Arrotin <arrfab@centos.org>
2022-02-17 13:25:45 +01:00
Fabian Arrotin
8ffd801904 Removed unused terms/infra/services in README
Signed-off-by: Fabian Arrotin <arrfab@centos.org>
2022-02-17 13:22:23 +01:00
Pierre-Yves Chibon
518994fd59 Make the get_sources shell script support flat layout
The idea of this commit is to make get_sources.sh support both the
'traditional' git layout used on dist-git for CentOS Linux where the
spec files are stored in a SPECS folder and the patches in a SOURCES
folder with the sha of the tarball being in a '.<pkg_name>.metadata'
file as well as the 'flat' git layout that have adopted Fedora as well
as CentOS-Stream (9+) where the spec files and patches are all stored
at the top level directory of the repository and the sha of the tarball
is present in a 'sources' file.

This commit re-uses code from the fedpkg-minimal project which is
license under the GPL (so the License field of the spec file for
centpkg-minimal may need to be adjusted to MIT and GPL).

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2022-02-01 10:59:12 +01:00
Alfredo Moralejo
510f1aa0bc Use http code to check object in lookaside cache
Currently, it only checks the size of the http object returned but it
may happen that the size of the 404 error page match the size of the
tarball. You may think this is a corner case we'll never hit... well, we
hit it, :)

This patch it's adding a check based on the http code, so to give it as
already uploaded we need it to return 200 code and match the actual
size.
2021-10-26 10:17:32 +02:00
Pablo Greco
b045956f51 Adapt the check before upload to the new format 2020-08-07 15:24:12 +00:00
Fabian Arrotin
e6e12ecbfd Cosmetic change for lookaside_upload and better curl return code check
Signed-off-by: Fabian Arrotin <arrfab@centos.org>
2020-05-29 18:00:58 +02:00
Fabian Arrotin
c43fda67c5 Added simple verification that source is reachable after upload
Signed-off-by: Fabian Arrotin <arrfab@centos.org>
2019-11-13 12:29:09 +01:00
Pat Riehecky
dc7af13b6c Fix mqtt message so the json isn't double escaped 2019-05-24 09:06:44 -05:00
Fabian Arrotin
8a81bc8622 Merge #4 2 Changes to make get_sources more portable 2019-05-14 05:54:44 +00:00
Johnny Hughes
96af14115c correct the escape sequence inside code tags for mqtt/mosquitto_sub/README.md 2019-04-17 09:21:16 -05:00
Johnny Hughes
92c530fe42 add a readme for some simple mqtt queries using jq and mosquitto_sub 2019-04-17 09:12:37 -05:00
Brian Stinson
b2f5375fcf add a grep to be sure we only check the origin remote (no forks) 2019-04-16 13:10:24 -05:00
Brian Stinson
7707c61e83 A WHICH! BURN IT 2019-04-16 13:10:19 -05:00
Brian Stinson
9857597bf6 add some quotes to make shellcheck happier 2019-04-16 13:06:51 -05:00
Brian Stinson
fd37ab5810 use command instead of which. this is more portable and doesn\'t require another dep 2019-04-16 10:03:11 -05:00
Brian Stinson
3382beadcc Get the proper branch name from the remote instead of the local branch
name.

This allows you to clone a commit directly by its hash, and not be
required to set up a tracking branch (with a matching name) to its
upstream.
2019-04-16 09:59:13 -05:00
Johnny Hughes
e575c2e598 set baseurl to git.centos.org 2019-04-12 08:00:45 -05:00
Johnny Hughes
489f5a7024 Merge #2 Provide sample MQTT scripts for the new infrastructure 2019-04-11 18:23:17 +00:00
Fabian Arrotin
eeff201b83 Merge #3 Check file existence before upload 2019-04-11 17:33:50 +00:00
Johnny Hughes
1fbac76eac remove gitblit specific info about centos.git.repolist.py 2019-04-11 11:59:32 -05:00
Pablo Greco
353eaf8879 Check file existence before upload 2019-04-11 16:36:20 +00:00
Pat Riehecky
0b0750a5f7 Provide sample MQTT scripts for the new infrastructure 2019-04-11 11:21:10 -05:00
Pat Riehecky
059392a377 Update centos.git.repolist.py to use pagure api 2019-04-11 10:03:10 -05:00
Fabian Arrotin
9023a882e1 Initial upload tool for git.stg test, more to be included in centpkg later
Signed-off-by: Fabian Arrotin <arrfab@centos.org>
2018-10-25 17:32:30 +02:00
Johnny Hughes
4ec6d6633e update centos.git.repolist.py for git update, submitted by Pat Riehecky 2017-06-26 10:26:57 -05:00
Colin Walters
1ad34919a1 get_sources: Print the URL we're fetching
Just for better debugging.
2016-12-14 09:46:35 -06:00
Colin Walters
552ca48e93 get_sources: Follow redirects
This is general best practice, and would have avoided the
need for 696c29f5c9
(I tested with that commit reverted)
2016-12-14 09:46:35 -06:00
Tuomo Soini
bd83a7926b get_sources: download source again if file checksum has changed
This functionality requires use of option -c.

Signed-off-by: Tuomo Soini <tis@foobar.fi>
2016-11-15 08:47:05 -06:00
Johnny Hughes
696c29f5c9 fix for git.c.o redirect 2016-07-06 14:11:13 -05:00
Pat Riehecky
4a0580184e Some packages seem to want to know where 'topdir' is 2014-12-01 10:27:25 -06:00
Johnny Hughes
3a0ccf9afb update logic for SCLs 2014-11-12 11:31:51 -06:00
Johnny Hughes
e9e1669c14 update readme to require scl-utils-build to use return_disttag.sh 2014-07-22 14:11:34 -05:00
Pat Riehecky
a68f527d1c SCL macros can prevent rpmbuild from examining some spec files
---
 into_srpm.sh      |    8 ++++++++
 return_disttag.sh |    6 ++++++
 2 files changed, 14 insertions(+), 0 deletions(-)
2014-07-22 10:03:07 -05:00
Pat Riehecky
7520e4b130 Send these errors to stderr 2014-07-22 10:01:28 -05:00
Johnny Hughes
dea6c03fe6 added one shift to the -d option 2014-07-10 15:02:33 -05:00
Pat Riehecky
733e0e2cd7 Added option to set explicit %dist per request 2014-07-10 14:12:38 -05:00
From: Pat Riehecky
db2f8e733d Now (optionally) supports software collections 2014-07-10 10:47:06 -05:00
From: Pat Riehecky
c429fbdd94 Added switch to preserve timestamps on creation of SRPMs for ls -ltr 2014-07-10 10:46:03 -05:00
From: Pat Riehecky
01a821d933 return_disttag now works with SCL packages 2014-07-09 18:21:48 -05:00
Tyler Parsons
d96c00fa0f Allow get_sources.sh hash verification to work with hashes other than sha1 2014-06-27 13:06:10 -05:00