Commit graph

16 commits

Author SHA1 Message Date
Rohan McGovern
117cabb0b7 Use SAX instead of pulldom for primary.xml parsing [RHELDST-14338]
Redo the parsing of packages from primary.xml to use SAX; previously it
was using pulldom. The motivation for the change is to reduce memory usage.

When parsing a larger yum repo such as that contained within
rhel-8-for-ppc64le-appstream-kickstart__8_DOT_4, the observed memory
usage from repo-autoindex command was:

- pulldom: ~700MB
- SAX:     ~85MB

This does not affect the output of the indexing process, and is covered
by existing tests.
2022-10-20 09:51:30 +10:00
Rohan McGovern
af99a34e39 Fix 'error: Unused "type: ignore" comment' with mypy 0.981 2022-10-05 10:38:32 +10:00
Rohan McGovern
293f5887b7 Implement error handling
Ultimately, all errors are propagated in some way, but it's important to
differentiate between "the content was invalid" vs "failed to fetch the
content".
2022-08-09 08:51:06 +10:00
Rohan McGovern
90b746caee Add PULP_MANIFEST support 2022-08-08 13:45:50 +10:00
Rohan McGovern
254e0f7cd9 Add some documentation 2022-08-08 12:46:07 +10:00
Rohan McGovern
52ec5f195b Add a test running entire command 2022-08-02 16:45:24 +10:00
Rohan McGovern
c4fd70e240 Add a test for text elision 2022-08-02 16:41:46 +10:00
Rohan McGovern
fbd3cb37ec Add a test which renders a yum repo 2022-07-07 14:04:18 +10:00
Rohan McGovern
787ba01a0e Rearrange sources to keep API separate, add a real test 2022-07-07 13:20:43 +10:00
Rohan McGovern
3a0285d433 Add github workflows for testing 2022-06-29 16:25:51 +10:00
Rohan McGovern
119f0ea9b6 Make it all pass mypy 2022-06-29 16:19:05 +10:00
Rohan McGovern
05c6bd14be Add some basic tests & CI setup 2022-06-29 15:52:00 +10:00
Rohan McGovern
8d5a624f18 Tweak default time/size for nonexistent data 2022-06-29 15:44:46 +10:00
Rohan McGovern
3062bdad26 Declare HTML lang and charset 2022-06-20 09:47:09 +10:00
Rohan McGovern
975e9f9f9e Declare HTML doctype
This is important due to certain behavior of localstack S3.
2022-06-20 09:33:52 +10:00
Rohan McGovern
5c4e5354b2 Initial implementation
Basically working for yum repos.
2022-06-17 10:31:31 +10:00