There is no need to mount the filesystem on the host side.
All filesystem tools offer some way to fill the fs without mounting.
So, create the content on the host side, create and fill the fs
without mounting.
No more sudo or guestmount needed.
This new approach works because the tests don't care about user IDs
and no device files are needed.
If user IDs start to matter it's still possible to use wrapper
tools like fakeroot in future while filling the fs.
Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Since net_test_wget() is testing a command and is in test/cmd it should
be in the 'cmd' test suite, not 'lib'.
Saving and restoring the values of the environment variables that the
test manipulates is necessary to avoid a regression when running the
whole ut test suite. A minimal reproducer is:
$ ./u-boot -T -c "ut cmd net_test_wget; ut dm dm_test_eth_act" | \
grep -E "(Test:|Failures:)"
Reported-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
We don't need the fallback anymore. Remove the code which uses these
files.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Richard Weinberger <richard@nod.at>
We don't need the fallback anymore. As a first step to removing it,
drop the try...except clauses and unindent the code.
This produces a large diff but there are no other code changes.
Signed-off-by: Simon Glass <sjg@chromium.org>
Like for test_fs, no need to mess with loop mounts.
Signed-off-by: Richard Weinberger <richard@nod.at>
Tweaks to reduce diff (keep mnt variable):
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a parameter to indicate the size of the image to build.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Richard Weinberger <richard@nod.at>
Since no mounting happens anymore, rename the "mnt"
directory to "scratch" and the related variables.
Signed-off-by: Richard Weinberger <richard@nod.at>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
There is no need to mount the filesystem on the host side.
All filesystem tools offer some way to fill the fs without mounting.
So, create the content on the host side, create and fill the fs
without mounting.
No more sudo or guestmount needed.
This new approach works because the tests don't care about user IDs
and no device files are needed.
If user IDs start to matter it's still possible to use wrapper
tools like fakeroot in future while filling the fs.
Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Rename actual android bootmethod test to specify it's for boot image
version 4.
Add a unit test for testing the Android bootmethod with boot image
version 2.
This requires another mmc image (mmc8) to contain the following
partitions:
- misc: contains the Bootloader Control Block (BCB)
- boot_a: contains a fake generic kernel image
we can test this with:
$ ./test/py/test.py --bd sandbox --build -k test_ut # build the mmc8.img
$ ./test/py/test.py --bd sandbox --build -k bootflow_android
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Signed-off-by: Guillaume La Roque <glaroque@baylibre.com>
Link: https://lore.kernel.org/r/20241126-adnroidv2-v4-5-11636106dc69@baylibre.com
Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Janne Grunau <j@jannau.net> says:
Starting with v2024.10 dev_iommu_dma_unmap calls during device removal
trigger a NULL pointer dereference in the Apple dart iommu driver. The
iommu device is removed before its user. The sparsely used DM_FLAG_VITAL
flag is intended to describe this dependency. Add it to the driver.
Adding this flag is unfortunately not enough since the boot routines
except the arm one simply remove all drivers. Add and use a new function
which calls
dm_remove_devioce_flags(DM_REMOVE_ACTIVE_ALL | DM_REMOVE_NON_VITAL);
dm_remove_devices_flags(DM_REMOVE_ACTIVE_ALL);
to ensure this order dependency is head consistently.
Link: https://lore.kernel.org/r/20241123-iommu_apple_dart_ordering-v2-0-cc2ade6dde97@jannau.net
This replaces dm_remove_devices_flags() calls in all boot
implementations to ensure non vital devices are consistently removed
first. All boot implementation except arch/arm/lib/bootm.c currently
just call dm_remove_devices_flags(DM_REMOVE_ACTIVE_ALL). This can result
in crashes when dependencies between devices exists. The driver model's
design document describes DM_FLAG_VITAL as "indicates that the device is
'vital' to the operation of other devices". Device removal at boot
should follow this.
Instead of adding dm_remove_devices_flags() with (DM_REMOVE_ACTIVE_ALL |
DM_REMOVE_NON_VITAL) everywhere add dm_remove_devices_active() which
does this.
Fixes a NULL pointer deref in the apple dart IOMMU driver during EFI
boot. The xhci-pci (driver which depends on the IOMMU to work) removes
its mapping on removal. This explodes when the IOMMU device was removed
first.
dm_remove_devices_flags() is kept since it is used for testing of
device_remove() calls in dm.
Signed-off-by: Janne Grunau <j@jannau.net>
Provide a unit test for the hextoull() function.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
While at the base level, this conversion looks equivalent, we now see
both of these tests failing (due to exceeding their allowed margin for
being too slow) in Azure with a very high frequency.
This reverts commit 88db4fc5fe.
Signed-off-by: Tom Rini <trini@konsulko.com>
Tested-by: Andrew Goodbody <andrew.goodbody@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
We make fewer calls to dm_test_restore() since
commit fbdac8155c ("test: Expand implementation of ut_list_has_dm_tests()")
Because of this some valid test combinations are now broken:
$ ./test/py/test.py --bd sandbox --build -k test_ut
$ ./test/py/test.py --bd sandbox --build -k "bootflow_android or bootflow_cros"
Shows:
Expected ' 2 cros ready mmc 4 mmc5.bootdev.part_4 ',
got ' 2 cros ready mmc 2 mmc5.bootdev.part_2 '
Here prep_mmc_bootdev() is called twice and it will bind bootmeth_cros twice.
Since bootmeth_cros is bound twice, 'bootflow scan' will find 2x the
expected bootflows.
Before
commit fbdac8155c ("test: Expand implementation of ut_list_has_dm_tests()")
this did not happen because a cleanup was called each time.
Add UTF_DM and UTF_SCAN_FDT flags to both tests to make sure that the
bootmeths are unbound after the test finishes.
Fixes: fbdac8155c ("test: Expand implementation of ut_list_has_dm_tests()")
Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
ut_check_console_line() does include an assert.
Pass the result to ut_assertok().
Addresses-Coverity-ID: 514958 Error handling issues
Fixes: 7dfafcd65e ("test: unit test for hash command")
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Currently, MMC test runs on default mmc modes, adding a provision to
support multiple mmc modes through user defined parameters.
Signed-off-by: Love Kumar <love.kumar@amd.com>
'usb part' command shows the partition maps and shows the partition type
by displaying number such as 0c, 83 etc. Observed that ext2 and ext4
partitions shows the same number, i.e, 83, so, using the fstype command
to distiniguish between ext2 and ext4 partitions.
Signed-off-by: Love Kumar <love.kumar@amd.com>
'mmc part' command shows the partition maps and shows the partition type
by displaying number such as 0c, 83 etc. Observed that ext2 and ext4
partitions shows the same number, i.e, 83, so, using the fstype command
to distiniguish between ext2 and ext4 partitions.
Signed-off-by: Love Kumar <love.kumar@amd.com>
Some of the bootm tests rely on state that is assumed to be
correct but is changed by the tests. This means that running
'ut bootm' twice will result in failures on the second run
as the state left by the first run is not what the tests
expect.
Fix this by ensuring the state is as expected by explicitly
setting that state.
Signed-off-by: Andrew Goodbody <andrew.goodbody@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
We are about to add a large number of new entries. Update the prefix to
be a little shorter.
For SMBIOS items, use SYSID_SM_ (for System Management) which is enough
to distinguish it. For now at least, it seems that most items will be
for SMBIOS.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Raymond Mao <raymond.mao@linaro.org>
Rasmus Villemoes <ravi@prevas.dk> says:
Doing bringup of a board, part of my bootstrap logic is in U-Boot. So
when tweaking that logic, I was bitten by a previous completed
bootstrap having left a copy of the environment on the device, which
was imported and thus overrided the new logic.
So I thought, "ok, I'll just make sure to put 'env default -a' as the
first part of the bootstrap logic so I'm not bitten again". Alas, my
logic also relies on certain variables that are set by C code
(e.g. for detecting board variant), and doing 'env default -a' also
eliminates those.
Looking around, the hashtab code already supports a flag that does
exactly what I need, and exposing that is (morally) a one-liner.
Link: https://lore.kernel.org/r/20241030213404.2894247-1-ravi@prevas.dk
Check that the new -k flag works as expected.
This also adds a test of the -a flag, which was previously missing,
and as the comment says, perhaps for a good reason. At least now we
have a test for it in combination with -k (and -f, because the ethaddr
variables otherwise cause complaining).
Signed-off-by: Rasmus Villemoes <ravi@prevas.dk>
Instead of testing the same expected behaviour for both
non_default_varX, test that when var1 is not in the default env but is
mentioned in the "env default" cmdline, it is removed, while var2 is
untouched.
Signed-off-by: Rasmus Villemoes <ravi@prevas.dk>
In the message string " %s[%d]\t[0x%llx-0x%llx], 0x%08llx bytes flags: "
a comma is missing before flags.
To avoid increasing the code size replace '0x%' by '%#'.
Printing the size with leading zeros but not the addresses does not really
make sense. Remove the leading zeros from the size output.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
[trini: Fix test/cmd/bdinfo.c for these changes]
Signed-off-by: Tom Rini <trini@konsulko.com>
For printing size_t we must use %zd and not %ld to avoid
a -Wformat error on 32-bit systems.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Availability of %ls in printf() depends on having
CONFIG_EFI_LOADER or CONFIG_EFI_APP.
Respect this when testing.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
We want to completely initialize the mbr and embr buffers. This requires
passing the buffer size and not the size of a pointer to the buffer.
Addresses-Coverity-ID: 510454 Wrong sizeof argument
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Heinrich Schuchardt <heinrich.schuchardt@canonical.com> says:
Since commit 348ea87850 ("cmd: hash: fix param count check") the hash
command cannot be used without the optional variable name parameter if
CONFIG_HASH_VERIFY=y. 'hash sha1 $loadaddr $filesize' returns
CMD_RET_USAGE.
The minimum number of arguments is four no matter if verification is
enabled or not.
Fix the parameter check.
Provide a unit test.
Link: https://lore.kernel.org/r/20241102100836.103005-1-heinrich.schuchardt@canonical.com
Simon Glass <sjg@chromium.org> says:
Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.
By way of background, about 4 years ago I wrong a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.
It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:
- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices
It lacks a 'lab check' feature and a few other things, but these can be
remedied.
On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.
I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.
Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:
ub-int <target>
Build and boot on a target, starting an interactive session
ub-cli <target>
Build and boot on a target, ensure U-Boot starts and provide an interactive
session from there
ub-smoke <target>
Smoke test U-Boot to check that it boots to a prompt on a target
ub-bisect <target>
Bisect a git tree to locate a failure on a particular target
ub-pyt <target> <testspec>
Run U-Boot pytests on a target
Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.
The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.
I hope you find this interesting and take a look!
[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] https://github.com/labgrid-project/labgrid/pull/1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html
Link: https://lore.kernel.org/r/20241112141326.643128-1-sjg@chromium.org
[trini: Move the sjg-lab job to prior to world build, to fix pipeline
status]
Signed-off-by: Tom Rini <trini@konsulko.com>
Use an 'r' string to avoid a warning:
test/py/tests/test_spi.py:698: DeprecationWarning: invalid escape
sequence '\s'
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Love Kumar <love.kumar@amd.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The Beagleplay board uses an SoC from the TI K3 family. This has both a
Cortex-R core and a Cortex-A core and the R core needs to come up before
the A core. In both cases we have U-Boot SPL then U-Boot proper being
used.
In practice this means we need two entirely separate builds to produce
an image.
Handle this in test.py by adding more parameters.
Signed-off-by: Simon Glass <sjg@chromium.org>
This can take a while and involve multiple steps (e.g. turning the board
back off). Add a section for it and show the output.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Send the Labgrid quit characters to ask it to exit gracefully. This
typically allows it to power off the board being used. Only do this when
labgrid is being used (detected with an env var).
If that doesn't work, try the less graceful approach.
The normal approach for pytest is to simply kill the child process. This
makes Labgrid exit immediately. Thus it does not get a chance to execute
the 'off' part of strategy (which may power it off) and release the
device.
Without this, every board disconnect leaves the board in a bad state,
requiring separate steps to recover the board, then power it off.
The action is conditional on since USE_LABGRID_SJG being set, so only
affects operation if the Labgrid-sjg integration is being used.
Signed-off-by: Simon Glass <sjg@chromium.org>
There is a very annoying bug at present where the terminal echos part
of the first command sent to the board. This happens because the
terminal is still set to echo for a period until Labgrid starts up and
can change this.
Fix this by disabling echo (and other terminal features) as soon as the
spawn happens.
Signed-off-by: Simon Glass <sjg@chromium.org>
We expect commands to be echoed and this should happen quite quickly,
since U-Boot is sitting at the prompt waiting for a command.
Reduce the timeout for this situation. Try to produce a more useful
error message when something goes wrong. Also handle the case where the
connection has gone away since the last command was issued.
Signed-off-by: Simon Glass <sjg@chromium.org>
There is quite a bit of code in pytest to try to start up U-Boot on a
board, with timeouts, expects, etc.
This is tedious to maintain and is peripheral to the test system's
purpose. It seems better to put this logic in the lab itself, where is
can provide such support.
With Labgrid we can use the UbootStrategy class to get the board into a
useful state, however it needs to do it. Then it can report to pytest
by writing a suitable string along with the U-Boot version it detected.
Add support for detecting 'lab mode' and simply assume that all is well
in that case. Collect the version string when Labgrid says it is ready.
This is only used with the Labgrid-sjg integration. When Labgrid starts
the UbootStrategy it checks if U_BOOT_SOURCE_DIR is set. If so it emits
a string '{lab mode}' that tells test.py to simply wait for an
indication that the board is ready. All banner-checking is skipped. The
indication comes in the form of another string 'Lab: Board is ready'
which Labgrid sends once the board is sitting at a prompt ready to run
tests. Then test.py emits 'U-Boot is ready' and continues with testing.
Note that Labgrid has the same kind of "check for a string" logic that
is in test.py, except it's not caring about the correct number / order
of banner prints. This checking could be added, however. If something
fails, the complete output is shown, so it is possible to see what went
wrong.
Signed-off-by: Simon Glass <sjg@chromium.org>
In Labgrid there is the concept of a 'role', which is similar to the
U-Boot board ID in U-Boot's pytest subsystem.
The role indicates both the target and information about the U-Boot
build to use. It can also provide any amount of other configuration.
The information is obtained using the 'labgrid-client query' operation.
Using this role, all required configuration for the board is stored
within the Labgrid environment, with pytest simply querying it. This
allows connecting to boards using an interactive console, something that
isn't possible without some kind of mapping. It also means that we don't
need to replicate the pytest functionality in tbot, since Labgrid can
handle the console and kick off builds as needed.
Make use of this in tests, so that only the role is required in gitlab
and other situations. The board type and other things can be queried
as needed.
Use a new 'u-boot-test-getrole' script to obtain the requested
information.
With this it is possible to run lab tests in gitlab with just a single
'ROLE' variable for each board.
Note that, without this feature:
- interactive use of boards with Labgrid-sjg would require repeating the
id/board in a separate configuration file
- Gitlab yaml file would need to specify both the id and board
This feature is entirely optional, however, with the code gracefully
falling back to using a separate ID and board.
Link: https://tbot.tools
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes we know that the board is already running the right software,
so provide an option to allow running of tests directly, without first
resetting the board.
This saves time when re-running a test where only the Python code is
changing.
Note that this feature is open to errors, since the user must know that
the board is in a fit state to execute tests. It is useful for repeated
iteration on a particular test, where it can save quite a bit of time.
Signed-off-by: Simon Glass <sjg@chromium.org>
When a board is finished with, the lab may want to power it off, or
perform some other function. Add a new script which is called when tests
are complete.
Reviewed-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
When Labgrid is used, it can get U-Boot ready for running tests. It
prints a message when it has done so.
Add logic to detect this message and accept it.
Note that this does not change pytest, which still (also) looks for the
U-Boot banner. This change merely makes it possible for pytest to
believe Labgrid when it says that the board is ready for use.
In several cases, the board starts up and Labgrid receives some initial
output, then pytest starts and misses some of that output, because it
came in while Labgrid had the console open. Then pytest fails because
it doesn't see the expected banners.
With this change, Labgrid handles getting U-Boot to a prompt, in a
fully reliable manner. Then pytest starts up and can simply start
running its tests.
But, again, this does not prevent pytest from handling a banner if one
is provided (e.g. if not using the Labgrid integration).
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
When mentioning a test name, add single quotes to make it easier to see.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
This should show the test name, not the selected name, since the user
may be running all tests, in which case 'select_name' is NULL
Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
Rather than returning various error codes, use assertions to check that
the test passes.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
