For most boards, the device-tree compiler is built in-tree, ignoring the
system version. Add a special option to skip this build. This can be
useful when the system dtc is up-to-date, as it speeds up the build.
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman retries a failed build when processing a branch, but does not
do this when building current source. It is useful to do this retry in
both cases, so add the logic for it.
Signed-off-by: Simon Glass <sjg@chromium.org>
There are operations in buildman that result in running the cross-tools
(such as performing size checks) and now that we have not modified PATH
to know where our tools are, these operations fail.
This reverts commit 6c0a3cf75f.
Signed-off-by: Tom Rini <trini@konsulko.com>
The feature to set the toolchain path does not seem to be needed. It
causes problems with venv (see [1]). Let's remove it.
Add some tests while we are here.
It does not look like any docs changes are needed for this.
[1] https://patchwork.ozlabs.org/project/uboot/patch/20240621131423.2363294-6-sjg@chromium.org/
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
When a file is removed by a commit (e.g. include/common.h yay!) it can
cause incremental build failures since one of the dependency files from
a previous build may mention the file.
Add an option to run 'make mrproper' automatically when a build fails.
This can be used to automatically resolve the problem, without always
adding the large overhead of 'make mrproper' to every build.
Signed-off-by: Simon Glass <sjg@chromium.org>
When this flag is enabled, 'make mrproper' is always used when
reconfiguring, so there is no point in doing it again.
Update this.
Signed-off-by: Simon Glass <sjg@chromium.org>
We cannot be sure that the new config is consistent, particularly when
changing a major item like CONFIG_CMDLINE. Use 'make oldconfig' to
check that and avoid any such problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
Make a list of common output extensions and use it to ensure that the -k
option preserves all of these.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
This is probably a little more efficient and it allows passing the object
to another function to write data. Convert config_out to use a string I/O
device.
Signed-off-by: Simon Glass <sjg@chromium.org>
Move more of the argument-building code into this function. Fix a missing
assignment for out_rel_dir too.
Rename the function since it now builds all the arguments.
Signed-off-by: Simon Glass <sjg@chromium.org>
This variable has a different meaning in the outer scope. Use a different
name to avoid confusion, or bugs.
Signed-off-by: Simon Glass <sjg@chromium.org>
The host compiler is not failing on warnings at present, when the
-E flag is used in buildman. Add the required flag to fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com>
The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is quite a useful thing to use when building since it avoids small
size changes between commits. Add a -r flag for it.
Also undefine CONFIG_LOCALVERSION_AUTO since this appends the git hash
to the version string, causing every build to be slightly different.
Signed-off-by: Simon Glass <sjg@chromium.org>
This cuts down build performance considerably and is not always needed,
when checking for build errors, etc.
Add a flag to disable it.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is sometimes useful to see the exact 'make' command used by buildman
for a commit. Add an output file for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a new flag to buildman so that we will in turn pass
BINMAN_ALLOW_MISSING=1 to 'make'. Make use of this flag in CI.
Allow the settings file to control this.
Cc: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman should consider a build as a success (with warnings) if missing
blobs have been dealt with by binman, even though buildman itself returns
and error code overall. This is how other warnings are dealt with.
We cannot easily access the 103 exit code, so detect the problem in the
output.
With this change, missing blobs result in an exit code of 101, although
they still indicate failure.
Signed-off-by: Simon Glass <sjg@chromium.org>
We have a module called 'board'. Sometimes buildman uses 'brd' as an
instance variable but sometimes it uses 'board', which is confusing and
can mess with the module handling. Update the code to use 'brd'
consistently, making it easier for tools to determine when the module
is being referenced.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a -a option to specify changes to the config before the build
commences. For example
buildman -a ~CONFIG_CMDLINE
disables CONFIG_CMDLINE before doing the build.
This makes it easier to try things out as well as to write tests without
creating a new board or manually manging the .config file.
Signed-off-by: Simon Glass <sjg@chromium.org>
If a thread crashes it is helpful to try the operation again with
threading disabled. Add a hint about that.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman does not write any output (to the 'out' and 'err)
files if the build terminates with a fatal error. This is to avoid adding
lots of spam to the logs.
However there are times when this is actually useful, such as when the
build fails for an obscure reason such as a Kconfig loop.
Update the logic to always write the output, so that the user gets a clue
as to what is happening.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we sometimes see problems in gitlab where the environment has
0x80 characters or sequences which are not valid UTF-8.
Avoid this by using bytes for the environment, both internal to buildman
and when writing out the 'env' file. Add a test to make sure this works
as expected.
Reported-by: Marek Vasut <marex@denx.de>
Fixes: e5fc79ea71 ("buildman: Write the environment out to an 'env' file")
Signed-off-by: Simon Glass <sjg@chromium.org>
There have been at least a few cases where an exception has occurred in a
thread and resulted in buildman hanging: running out of disk space and
getting a unicode error.
Handle these by collecting a list of exceptions, printing them out and
reporting failure if any are found. Add a test for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
The environment may contain some unicode characters. At least that is what
seemed to happen on one commit:
Building current source for 1 boards (0 threads, 64 jobs per thread)
0 0 0 /1 -1 (starting)
Traceback (most recent call last):
File ".../tools/buildman/buildman", line 64, in <module>
ret_code = control.DoBuildman(options, args)
File "tools/buildman/control.py", line 372, in DoBuildman
options.keep_outputs, options.verbose)
File ".../tools/buildman/builder.py", line 1704, in BuildBoards
results = self._single_builder.RunJob(job)
File ".../tools/buildman/builderthread.py", line 526, in RunJob
self._WriteResult(result, job.keep_outputs, job.work_in_output)
File ".../tools//buildman/builderthread.py", line 349, in _WriteResult
print('%s="%s"' % (var, env[var]), file=fd)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
311-312: ordinal not in range(128)
The problem defies repetition with any change at all to buildman. But
let's set an encoding in any case.
Signed-off-by: Simon Glass <sjg@chromium.org>
