u-boot/test/py/tests
Tom Rini 8162f35a10 Merge patch series "test: Improvements to ut command and test-suite running"
Simon Glass <sjg@chromium.org> says:

The current method of running unit tests relies on subcommands of the
ut command. Only the code in each subcommand knows how to find the tests
related to that subcomand.

This is not ideal and we now have quite a few subcommands which do
nothing but locate the relevant tests in a linker list, then call a
common function to run them.

This series adds a list of test suites, so that these subcommands can be
removed.

An issue with 'ut all' is that it doesn't record how many tests failed
overall, so it is necessary to examine copious amounts of output to look
for failures. This series adds a new 'total' feature allow recording the
total number of failed tests.

To help with 'ut all' a new pytest is created which runs it (as well as
'ut info') and makes sure that all is well. Due to the 'ut all' failures
this does not pass, so the test is disabled for now. It is here because
it provides security against misnaming a test suite and causing it not
to run.

Future work may:
- get 'ut all' passing
- enable test_suite() in CL, to ensure that 'ut all' keeps passing
- record duration of each suite
- allow running the tests in random order to tease out dependencies
- tweak the output to remove common prefixes
- getting rid of bootstd, optee and seame 'ut' subcommands

Link: https://lore.kernel.org/r/20250120212613.516664-1-sjg@chromium.org
2025-01-24 14:35:37 -06:00
..
bootstd bootstd: Remove prepared images 2024-11-26 11:13:39 -06:00
test_android cmd: bcb: introduce 'ab_dump' command to print BCB block content 2024-10-24 09:45:55 +02:00
test_cat tests: Fix exception when cleaning up skipped test 2023-07-17 15:39:55 -04:00
test_efi_bootmgr global: Use proper project name U-Boot 2023-06-12 13:24:31 +02:00
test_efi_capsule sandbox: switch to dynamic UUIDs 2024-09-12 17:35:37 +02:00
test_efi_secboot efi_loader: Fix EFI_VARIABLE_APPEND_WRITE hash check 2024-06-10 15:01:44 +02:00
test_eficonfig cmd/eficonfig: capitalize 'enter description' 2024-10-30 21:44:39 +01:00
test_fs test_fs: Rename mount dir to scratch 2024-11-26 11:13:39 -06:00
test_semihosting test: unit test for semihosting 2023-08-14 17:55:53 -04:00
test_xxd tests: Fix exception when cleaning up skipped test 2023-07-17 15:39:55 -04:00
vboot test: py: vboot: add test for global image signature 2022-03-31 14:12:23 -04:00
fit_util.py vbe: Add a test for VBE device tree fixups 2022-10-17 21:17:13 -06:00
fs_helper.py test_fs: Allow running unprivileged 2024-11-26 11:13:39 -06:00
source.its test: Add test for source command 2022-12-31 13:35:19 -05:00
test_000_version.py SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
test_bind.py test: Make test_bind_unbind_with_uclass() single-threaded 2022-09-12 18:06:36 -04:00
test_bootmenu.py test: restore timeout after bootmenu unit test 2022-05-28 10:59:27 +02:00
test_bootstage.py test/py/tests/test_bootstage.py: Combine stash/unstash tests 2024-09-24 10:10:18 -06:00
test_button.py cmd/button: return button status 2020-10-14 11:16:34 -04:00
test_cleanup_build.py test: Skip cleanup test if not built out of tree 2024-01-18 20:23:37 -05:00
test_dfu.py test: Use single quote consistently 2019-01-15 15:28:41 -05:00
test_dm.py test: dm: Show failing driver name 2024-07-04 09:25:21 -06:00
test_efi_fit.py efi_loader: Rename and move CMD_BOOTEFI_HELLO_COMPILE 2024-10-09 22:04:56 -06:00
test_efi_loader.py test: efi_loader: Fix dependency for http test 2024-10-27 20:11:30 -06:00
test_efi_selftest.py test: don't call restart_uboot in EFI watchdog test 2024-10-09 22:04:56 -06:00
test_env.py fs: ext4: use fs_ls_generic 2024-11-01 13:37:58 -06:00
test_event_dump.py acpi: Allocate and write ACPI tables 2024-10-27 17:24:13 -06:00
test_extension.py test: fix test_extension.py 2025-01-05 02:30:47 +01:00
test_fit.py bootm: Fix flags used for bootargs string substitution 2023-12-20 10:46:54 -05:00
test_fit_auto_signed.py mkimage: fit: Support signed configurations in 'auto' FITs 2023-01-27 12:51:27 -05:00
test_fit_ecdsa.py test: Update FIT tests to run in parallel 2022-09-12 18:06:36 -04:00
test_fit_hashes.py test: Update FIT tests to run in parallel 2022-09-12 18:06:36 -04:00
test_fpga.py test/py/tests/test_fpga.py: Correct check for legacy image format support 2024-07-11 13:15:37 -06:00
test_gpio.py test/py: gpio: Add gpio pins generic test 2024-01-29 10:31:24 -05:00
test_gpt.py cmd: gpt: Add command to swap partition order 2023-09-11 16:27:49 -04:00
test_handoff.py test: Fix the boardspec for the SPL handoff test 2020-01-02 10:27:23 -05:00
test_help.py test: Check help output 2024-07-04 09:25:21 -06:00
test_i2c.py test/py: i2c: Add tests for i2c command 2024-01-16 17:05:29 -05:00
test_kconfig.py test: Add some tests for kconfig.h 2022-08-10 13:38:30 -04:00
test_log.py log: Allow tests to pass with CONFIG_LOGF_FUNC_PAD set 2024-07-04 09:25:21 -06:00
test_lsblk.py test: test_lsblk: Mark as sandbox specific 2022-01-14 12:26:42 -05:00
test_md.py SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
test_mdio.py test/py: mdio: Add tests for mdio command 2024-01-16 12:00:05 -05:00
test_memtest.py test/py: memtest: Add tests for mtest command 2024-01-16 17:05:29 -05:00
test_mii.py test/py: mii: Add tests for mii command 2024-01-16 17:05:29 -05:00
test_mmc.py test/py: mmc: Add support for different mmc modes 2024-11-17 13:14:22 -06:00
test_mmc_rd.py test/py: mmc: Fix 'mmc info' testcase 2020-03-27 15:42:04 -04:00
test_mmc_wr.py test/py: Automated conversion to Python 3 2019-10-30 17:48:47 -04:00
test_net.py test/py/tests: Update some network dependencies 2024-07-04 09:26:30 -06:00
test_net_boot.py test/py: net_boot: fix comment 2024-09-24 13:41:21 -06:00
test_of_migrate.py dm: test: Add a test for the various migration combinations 2023-02-14 09:43:27 -07:00
test_ofplatdata.py dm: treewide: Complete migration to new driver model schema 2023-02-14 09:43:27 -07:00
test_optee_rpmb.py test: py: add optee_rpmb tests 2024-04-18 16:04:48 +03:00
test_part.py test/py: test_part: Correct the test case name 2022-05-26 10:27:14 -04:00
test_pinmux.py test: Fix test_pinmux to run in parallel 2022-09-12 18:06:36 -04:00
test_pstore.py test: Add PStore command tests 2020-10-14 11:16:34 -04:00
test_pstore_data_console.hex test: Add PStore command tests 2020-10-14 11:16:34 -04:00
test_pstore_data_panic1.hex test: Add PStore command tests 2020-10-14 11:16:34 -04:00
test_pstore_data_panic2.hex test: Add PStore command tests 2020-10-14 11:16:34 -04:00
test_qfw.py test: qemu: add qfw sandbox driver, dm tests, qemu tests 2021-04-12 17:45:39 -04:00
test_reset.py test/py: reset: Add a test for reset command 2024-03-19 18:40:47 -04:00
test_sandbox_exit.py test: unit test for exception command 2020-12-13 07:58:17 -07:00
test_sandbox_opts.py CI, pytest: Add a test for sandbox without LTO 2023-11-07 14:49:40 -05:00
test_saveenv.py test/py: saveenv: Add a test for saveenv command 2024-03-19 18:40:47 -04:00
test_scp03.py global: Use proper project name U-Boot 2023-06-12 13:24:31 +02:00
test_scsi.py disk: simplify print_part_header() 2024-04-12 08:53:31 -06:00
test_sf.py SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
test_shell_basics.py test: py: test_shell_run() with CONFIG_HUSH_PARSER=n 2020-08-07 22:31:32 -04:00
test_sleep.py test: Fix skip check for sleep command test 2024-10-15 12:30:07 -06:00
test_smbios.py cmd: update smbios cmd 2025-01-14 14:29:29 -06:00
test_source.py cmd: source: Support specifying config name 2022-12-31 13:35:19 -05:00
test_spi.py test/py: spi: Rephrase the warning/error messages 2024-11-15 13:11:29 -06:00
test_spl.py test: Move stat-printing into its own function 2025-01-24 14:34:41 -06:00
test_stackprotector.py test/py: test_stackprotector: Disable for ASAN 2022-06-23 12:58:18 -04:00
test_suite.py test: Disable test_suite 2025-01-24 14:34:41 -06:00
test_tpm2.py tpm-v2: allow algorithm name to be configured for pcr_read and pcr_extend 2024-05-27 09:00:27 +03:00
test_trace.py test: test_trace.py: test 'trace wipe' 2024-12-31 19:00:46 -06:00
test_ums.py test: Use single quote consistently 2019-01-15 15:28:41 -05:00
test_unknown_cmd.py SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
test_upl.py Merge patch series "test: Improvements to ut command and test-suite running" 2025-01-24 14:35:37 -06:00
test_usb.py test/py: usb: Distinguish b/w ext2/ext4 partitions 2024-11-17 13:14:18 -06:00
test_ut.py test: Move stat-printing into its own function 2025-01-24 14:34:41 -06:00
test_vbe.py test: Move stat-printing into its own function 2025-01-24 14:34:41 -06:00
test_vbe_vpl.py vbe: Use the correct image filename in the test 2023-04-03 06:53:53 +12:00
test_vboot.py test: vboot: Using variable 'old_dtb' before assignment 2023-12-13 18:39:06 -05:00
test_vpl.py test: Move stat-printing into its own function 2025-01-24 14:34:41 -06:00
test_zynq_secure.py test/py: secure: Add secure tests for Zynq & ZynqMP 2024-01-29 10:31:24 -05:00
test_zynqmp_rpu.py test/py: zynqmp_rpu: Fix tcminit mode value 2024-11-29 13:21:17 +01:00
test_zynqmp_secure.py test/py: secure: Add secure tests for Zynq & ZynqMP 2024-01-29 10:31:24 -05:00
vboot_evil.py test: Correct pylint errors 2022-03-02 10:28:12 -05:00
vboot_forge.py fit: Don't allow verification of images with @ nodes 2021-02-15 19:17:25 -05:00