For most boards, the device-tree compiler is built in-tree, ignoring the
system version. Add a special option to skip this build. This can be
useful when the system dtc is up-to-date, as it speeds up the build.
Signed-off-by: Simon Glass <sjg@chromium.org>
When building for sandbox, self.cross is empty.
In MakeEnvironment(), CROSS_COMPILE is defined to be self.cross (with
or without a full path), optionally prefixed by the toolchain wrapper
defined in ~/.buildman. This is fine when self.cross is not empty, but
it doesn't make sense when it is:
- Either there is no wrapper and we end up with an empty CROSS_COMPILE
which is the same as not defining it (the host compiler will be used),
- Or there is a wrapper and CROSS_COMPILE will contain only the wrapper
which obviously is not a valid compiler, hence an error.
Test case:
$ sudo apt install ccache
$ grep -q toolchain-wrapper ~/.buildman || \
printf "[toolchain-wrapper]\nwrapper = ccache\n" >>~/.buildman
$ make mrproper
$ ./tools/buildman/buildman sandbox_noinst
$ ./tools/buildman/buildman sandbox_noinst
Building current source for 1 boards (1 thread, 24 jobs per thread)
sandbox: + sandbox_noinst
+arch/sandbox/lib/reloc_sandbox_efi.c:10:15: error: operator '==' has no left operand
+ 10 | #if HOST_ARCH == HOST_ARCH_X86_64
+ | ^~
[...]
The GetEnvArgs function is modified too, since the VAR_CROSS_COMPILE
case has the same issue.
In tools/buildman/test.py, testGetEnvArgs is extended and
testMakeEnvironment is added. They check the 'arm' and 'sandbox'
toolchains, with and without a wrapper.
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
The Python virtualenv tool sets up a few things in the environment,
putting its path first in the PATH environment variable and setting up
a sys.prefix different from the sys.base_prefix value.
At present buildman puts the toolchain path first in PATH so that it can
be found easily during the build. For sandbox this causes problems since
/usr/bin/gcc (for example) results in '/usr/bin' being prepended to the
PATH variable. As a result, the venv is partially disabled.
The result is that sandbox builds within a venv ignore the venv, e.g.
when looking for packages.
Correct this by detecting the venv and adding the toolchain path after
the venv path.
Signed-off-by: Simon Glass <sjg@chromium.org>
Newer versions of filelock use time.monotonic() instead of time.time().
Update the test the handle this.
It would be better if filelock had support for writing unit tests which
use locking.
Signed-off-by: Simon Glass <sjg@chromium.org>
There are operations in buildman that result in running the cross-tools
(such as performing size checks) and now that we have not modified PATH
to know where our tools are, these operations fail.
This reverts commit 6c0a3cf75f.
Signed-off-by: Tom Rini <trini@konsulko.com>
The feature to set the toolchain path does not seem to be needed. It
causes problems with venv (see [1]). Let's remove it.
Add some tests while we are here.
It does not look like any docs changes are needed for this.
[1] https://patchwork.ozlabs.org/project/uboot/patch/20240621131423.2363294-6-sjg@chromium.org/
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
Buildman uses all available CPUs by default, so running more than one or
two concurrent processes is not normally useful.
However in some CI cases we want to be able to run several jobs at once
to save time. For example, in a lab situation we may want to run a test
on 20 boards at a time, since only the build step actually takes much
CPU.
Add an option which allows such a limit. When buildman starts up, it
waits until the number of running processes goes below the limit, then
claims a spot in the list. The list is maintained with a temporary file.
Note that the temp file is user-specific, since it is hard to create a
locked temporary file which can be accessed by any user. In most cases,
only one user is running jobs on a machine, so this should not matter.
Signed-off-by: Simon Glass <sjg@chromium.org>
The methods `unittest.assertEquals()` and
`unittest.assertRegexpMatches()` are marked deprecated[1].
In Python 3.12 these aliases have been removed, so do a sed to replace
them with their new names.
[1] https://docs.python.org/3.11/library/unittest.html#deprecated-aliases
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
CC: Simon Glass <sjg@chromium.org>
CC: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use a separate file for the Boards class so that its name matches the
module name.
Fix up the function names to match the pylint style and fix some other
warnings.
Signed-off-by: Simon Glass <sjg@chromium.org>
We want to create a module called 'boards' so avoid use of this variable
name in this module. Change the global to be capitalised, as required by
Python style.
Signed-off-by: Simon Glass <sjg@chromium.org>
We have a module called 'board'. Sometimes buildman uses 'brd' as an
instance variable but sometimes it uses 'board', which is confusing and
can mess with the module handling. Update the code to use 'brd'
consistently, making it easier for tools to determine when the module
is being referenced.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fix documentation path in deprecated warning message about device
driver.
Signed-off-by: Johannes Krottmayer <krjdev@gmail.com>
Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
This method has the same name as its class which is confusing. It is also
annoying when searching the code.
It builds a string with a colour, so rename it to build().
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the only straightforward way to write tests that need a
slightly different configuration is to create a new board with its own
configuration. This is cumbersome.
It would be useful if buildman could adjust the configuration of a build
on the fly. In preparation for this, add a utility library which can
modify a .config file according to various parameters passed to it.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present even if only a single thread is in use, buildman still uses
threading.
For some debugging it is helpful to do everything in the main process.
Allow -T0 to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
This current fails with an error. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: 7664b03ffc ("buildman: Remove _of_#_ from results directory paths")
Currently, the following scenario will rebuild the first commit even
though it is not really necessary - the commit sha or the position in the
patchset did not change:
$ git am <local-patch-0001>
$ tools/buildman/buildman -P -E -W -b master mx6
<do some more development work>
$ git am <local-patch-0002>
$ tools/buildman/buildman -P -E -W -b master mx6 <- will rebuild the first
commit as well, even
though nothing has
changed about it.
This is due to the fact that previous results directories get removed
when the number of commits change. By removing the _of_#_ part of the
directory path, the commits will be rebuilt only if the commit sha or the
position in the patchset changes. Also, update the testcase to reflect this
change.
Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Now that we are using absolute paths we can remove some of the sys.path
mangling that appears in the tools.
We only need to add the path to 'tools/' so that everything can find
modules relative to that directory.
The special paths for finding pylibfdt remain.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move buildman to use absolute imports. Also adjust moveconfig.py too since
it uses some buildman modules and cannot work without this.
Signed-off-by: Simon Glass <sjg@chromium.org>
These are becoming more common now. They cause boards to show warnings
which can be mistaking for compiler warnings.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
Unfortunately the plague of device-tree warnings has not lifted. These
warnings infiltrate almost every build, adding noise and confusion.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
When buildman finishes it leaves the last summary line visible, which
shows the number of successful builds, builds with warnings and builds
with errors.
It is useful also to see how many builds were done in total along with
the time taken. Show these on a separate line before buildman finishes.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the board names shown with -l are separated by commas. This
makes it hard to double-click to select a particular board. Also it is not
possible to select all boards and paste them as arguments to a subsequent
buildman run, since buildman requires spaces to separate the list on the
command line, not commas.
Change the output format to use spaces instead of commas.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is quite hard to see the list of board for each error line since the
colour is the same as the actual error line. Show the board list in
magenta so that it is easier to distinguish them.
There is no point in checking the colour of the overall line, since there
are now multiple colours. So drop those tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present warnings are shown in yellow in the summary (-s) but magenta in
the detail listing (-e). Use yellow in both.
Signed-off-by: Simon Glass <sjg@chromium.org>
We want to add a few more tests similar to testOutput(). Split its logic
into a function which runs buildman to get the output and another which
checks the output. This will make it easier to reuse the code.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than having a few tests handle this themselves, create the
temporary directory in the setUp() method and remove it in tearDown().
This will make it easier to add more tests.
Only testOutput and testGit() actually need it, but it doesn't add to the
test time noticeably to do this for all tests in this file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than using the absolute array index, use an interator to work
through the expected output lines. This is easier to follow.
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman should output the right colours for each error/warning line. Some
of these checks are missing. Add them.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman removes any directory it doesn't intend to write
output into. This is overly expansive since if the output directory
happens to be somewhere with existing files, they may be removed. Using
an existing directory for buildman is not a good practice, but since the
result might be catastrophic, it is best to guard against it.
A previous commit[1] fixed this by refusing to write to a subdirectory
of the current directory, assumed to have U-Boot source code. But we can
do better by only removing directories that look like the ones buildman
creates.
Update the code to do this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
[1] 409fc029c4 tools: buildman: Don't use the working dir as build dir
Sometimes it is useful for external tools to use buildman to provide the
toolchain information. Add an -a option which shows the value to use for
the ARCH environment variable, and -A which does the same for
CROSS_COMPILE
Signed-off-by: Simon Glass <sjg@chromium.org>
In the 'Make' function, the codes tries to create a directory
if current stage is 'build'. But the directory isn't used at
all anywhere.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present we should boards with warnings in the same way as those with
errors. This is not ideal. Add a new 'warn' state and show these listed
in yellow to match the actual warning lines printing with -e.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present messages from the device-tree compiler like this:
arch/arm/dts/socfpga_arria10_socdk_sdmmc.dtb: Warning
(avoid_unnecessary_addr_size): /clocks: unnecessary
#address-cells/#size-cells without "ranges" or child "reg" property
are detected as errors since they don't match the gcc warning regex. Add a
new one for dtc to fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
