mirror of
https://github.com/release-engineering/repo-autoindex.git
synced 2025-02-23 21:52:52 +00:00
eac74ec1e4
The Fetcher type was designed to return a 'str'. That wasn't a good idea because it implies that every fetched file must be loaded into memory completely. On certain large yum repos, decompressed primary XML can be hundreds of MB, and it's not appropriate to require loading that all into memory at once. Make it support a file-like object (stream of bytes). Since the SAX XML parser supports reading from a stream, this makes it possible to avoid loading everything into memory at once. A test of repo-autoindex CLI against /content/dist/rhel/server/7/7Server/x86_64/os showed major improvement: - before: ~1200MiB - after: ~80MiB Note that achieving the full improvement requires any downstream users of the library (e.g. exodus-gw) to update their Fetcher implementation as well, to stop returning a 'str'.
57 lines
1.9 KiB
Markdown
57 lines
1.9 KiB
Markdown
# repo-autoindex
|
|
|
|
Generate static HTML indexes of various repository types
|
|
|
|
|
|
|
|
[](https://release-engineering.github.io/repo-autoindex/)
|
|
[](https://pypi.org/project/repo-autoindex/)
|
|
|
|
## Overview
|
|
|
|
`repo-autoindex` provides a minimal CLI and Python library to generate static HTML indexes
|
|
for certain types of content, such as yum repositories.
|
|
|
|
```
|
|
pip install repo-autoindex
|
|
REPO_URL=$(curl -s 'https://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-f36&arch=x86_64' | egrep '^http' | head -n1)
|
|
repo-autoindex $REPO_URL
|
|
xdg-open index.html
|
|
```
|
|
|
|
See [the manual](https://release-engineering.github.io/repo-autoindex/) for more
|
|
information about the usage of `repo-autoindex`.
|
|
|
|
## Changelog
|
|
|
|
### v1.2.0 - 2023-09-22
|
|
|
|
- Support streamed fetching to reduce memory usage when fetching large files.
|
|
|
|
### v1.1.2 - 2023-09-18
|
|
|
|
- Add `py.typed` to make package PEP 561 compliant / enable downstream type-checking.
|
|
|
|
### v1.1.1 - 2023-04-12
|
|
|
|
- Fix handling of kickstart repositories with no checksums in treeinfo.
|
|
|
|
### v1.1.0 - 2023-04-04
|
|
|
|
- Added limited support for kickstart repositories.
|
|
|
|
### v1.0.2 - 2022-10-21
|
|
|
|
- Reduced memory usage when handling large yum repositories.
|
|
|
|
### v1.0.1 - 2022-08-15
|
|
|
|
- Use correct SPDX license identifier in package metadata.
|
|
|
|
### v1.0.0 - 2022-08-15
|
|
|
|
- Initial stable release.
|
|
|
|
## License
|
|
|
|
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
|