Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.
We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.
Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.
The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.
Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
The existing DSU errata workarounds hijack the errata framework's inner
workings to register with it. However, that is undesirable as any change
to the framework may end up missing these workarounds. So convert the
checks and workarounds to macros and have them included with the
standard wrappers.
The only problem with this is the is_scu_present_in_dsu weak function.
Fortunately, it is only needed for 2 of the errata and only on 3 cores.
So drop it, assuming the default behaviour and have the callers handle
the exception.
Change-Id: Iefa36325804ea093e938f867b9a6f49a6984b8ae
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.
Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
Cortex-A510 erratum 2080326 is a Cat B erratum that applies
to all revisions <= r0p2 and is fixed in r0p3.
The workaround sequence helps perform a DSB after each TLBI
instruction and can be applied only for version r0p2 and has
minimal performance impact.
The workaround is not applicable for versions < r0p2.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1873361/latest
Change-Id: Ib9bce8b711c25a79f7b2f891ae6f8b366fc80ddd
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive
It is important to note that the errata workaround sequences remain
unchanged and preserve their git blame.
Note: cortex_a510.S is applicable and being used only by arm_fpga platform.
However, to test the ported changes, below steps were carried out on the
fvp and the obtained results has been verified.
Testing was conducted by:
* Building for release with all errata flags enabled and running script
in change 19136 to compare output of objdump for each errata.
* Testing via script was not complete, as it directed to verify the
check and the workaround functions of few erratas manually.
* Manual comparison of disassembly of converted functions with non-
converted functions
aarch64-none-elf-objdump -D <trusted-firmware-a with conversion>/build/../release/bl31/bl31.elf
vs
aarch64-none-elf-objdump -D <trusted-firmware-a clean repo>/build/fvp/release/bl31/bl31.elf
* Manual comparison of disassembly of both both files(bl31.elf)
ensured, the ported changes were identical and hence verified.
* Build for release with all errata flags enabled and run default
tftf tests.
CROSS_COMPILE=aarch64-none-elf- \
make PLAT=fvp \
ARCH=aarch64 \
DEBUG=0 \
HW_ASSISTED_COHERENCY=1 \
USE_COHERENT_MEM=0 \
CTX_INCLUDE_AARCH32_REGS=0 \
ERRATA_A510_1922240=1 \
ERRATA_A510_2288014=1 \
ERRATA_A510_2042739=1 \
ERRATA_A510_2041909=1 \
ERRATA_A510_2250311=1 \
ERRATA_A510_2218950=1 \
ERRATA_A510_2172148=1 \
ERRATA_A510_2347730=1 \
ERRATA_A510_2371937=1 \
ERRATA_A510_2666669=1 \
ERRATA_A510_2684597=1 \
ERRATA_DSU_2313941=1 \
BL33=/home/jaychi01/tf_a/tf-a-tests/build/fvp/release/tftf.bin \
fip all -j12
* Build for debug with all errata enabled and step through ArmDS
at reset to ensure that if Errata are applicable then the
workaround functions are entered precisely.
Change-Id: Icf7aa25c0b3b30f5e2ad6db83953f7f4f0b201d9
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
The "tsb csync" instruction is part of the Armv8.4 architecture
extension, and is not supported by many older assemblers.
We already cater for this in lib/extensions/trbe/trbe.c, where we use
the equivalent "hint #18" encoding for this, but use the new mnemonic
in the Cortex-A510 CPU support code.
Replace "tsb csync" with the hint encoding there as well, to support
building with older binutils versions.
Change-Id: Idf39f5c6c4dbf72802c3c120047b8bc499145e3b
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Cortex-A510 erratum 2684597 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3, r1p0, r1p1 and r1p2. It is fixed in r1p3. The
workaround is to execute a TSB CSYNC and DSB before executing WFI for
power down.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1873361/latesthttps://developer.arm.com/documentation/SDEN1873351/latest
Change-Id: Ic0b24b600bc013eb59c797401fbdc9bda8058d6d
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
Cortex-A510 erratum 2347730 is a Cat B erratum that affects
revisions r0p0, r0p1, r0p2, r0p3, r1p0 and r1p1. It is
fixed in r1p2. The workaround is to set CPUACTLR_EL1[17]
to 1, which will disable specific microarchitectural clock
gating behaviour.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1873351/latesthttps://developer.arm.com/documentation/SDEN1873361/latest
Signed-off-by: Akram Ahmad <Akram.Ahmad@arm.com>
Change-Id: I115386284c2d91bd61515142f971e2e72de43e68
Cortex-A510 erratum 2371937 is a Cat B erratum that applies
to revisions r0p0, r0p1, r0p2, r0p3, r1p0, and r1p1. It is
fixed in r1p2. The workaround is to set the ATOM field of
CPUECTLR_EL1 (bits [40:38]) to 0b010, which will force all
cacheable atomic operations to be executed near.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1873351/latesthttps://developer.arm.com/documentation/SDEN1873361/latest
Signed-off-by: Akram Ahmad <Akram.Ahmad@arm.com>
Change-Id: Ia219a609a3397e39631de65831ecff8a3cd1227e
DSU-110 erratum 2313941 is a Cat B erratum and applies to revisions
r0p0, r1p0, r2p0, r2p1, r3p0, r3p1 and is still open.
The workaround sets IMP_CLUSTERACTLR_EL1[16:15] bits to 0b11 to disable
clock gating of the SCLK domain. This will increase the idle power
consumption.
This patch applies the fix for Cortex-X2/A510/A710 and Neoverse N2.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1781796/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I54d948b23e8e01aaf1898ed9fe4e2255dd209318
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Cortex-A510 erratum 2172148 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3 and r1p0, and is fixed in r1p1.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I1784d643ca3d1d448340cd421facb5f229df1d22
Cortex-A510 erratum 2218950 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3 and r1p0, and is fixed in r1p1.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Icb6e369946f8978a08cf8ed5e4452782efb0a77a
Cortex-A510 erratum 2250311 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3 and r1p0 and is fixed in r1p1.
This erratum workaround is a bit different because it interacts with a
feature supported in TFA. The typical method of implementing an errata
workaround will not work in this case as the MPMM feature would just be
re-enabled by context management at every core power on after being
disabled by the errata framework. So in addition to disabling MPMM, this
workaround also sets a flag in the MPMM runtime framework indicating
that the feature should not be enabled even if ENABLE_MPMM=1.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I7805756e65ec90b6ef8af47e200617c9e07a3a7e
Cortex-A510 erratum 2041909 is a Cat B erratum that applies to revision
r0p2 and is fixed in r0p3. It is also present in r0p0 and r0p1 but there
is no workaround in these revisions.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I7b1498faa0c79488dee0d11d07f6e9f58144e298
Cortex-A510 erratum 2042739 is a Cat B erratum that applies to revisions
r0p0, r0p1 and r0p2 and is fixed in r0p3.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I1d2ebee3914396e1e298eb45bdab35ce9e194ad9
Cortex-A510 erratum 2288014 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3 and r1p0, and is fixed in r1p1.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I875519ff55be90244cc3d3a7e9f7abad0fc3c2b8
Cortex-A510 erratum 1922240 is a Cat B erratum that applies to revision
r0p0 and is fixed in r0p1.
Since no errata framework code existed for A510 prior to this patch, it
has been added as well. Also some general cleanup changes in the CPU lib
makefile.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2397239
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I8c427ef255cb4b38ed3e5c2c7444fcef957277e4
This patch renames the Matterhorn, Matterhorn ELP, and Klein CPUs to
Cortex A710, Cortex X2, and Cortex A510 respectively.
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I056d3114210db71c2840a24562b51caf2546e195
2021-05-28 13:53:23 -05:00
Renamed from lib/cpus/aarch64/cortex_klein.S (Browse further)
