MPMM is a core-specific microarchitectural feature. It has been present
in every Arm core since the Cortex-A510 and has been implemented in
exactly the same way. Despite that, it is enabled more like an
architectural feature with a top level enable flag. This utilised the
identical implementation.
This duality has left MPMM in an awkward place, where its enablement
should be generic, like an architectural feature, but since it is not,
it should also be core-specific if it ever changes. One choice to do
this has been through the device tree.
This has worked just fine so far, however, recent implementations expose
a weakness in that this is rather slow - the device tree has to be read,
there's a long call stack of functions with many branches, and system
registers are read. In the hot path of PSCI CPU powerdown, this has a
significant and measurable impact. Besides it being a rather large
amount of code that is difficult to understand.
Since MPMM is a microarchitectural feature, its correct placement is in
the reset function. The essence of the current enablement is to write
CPUPPMCR_EL3.MPMM_EN if CPUPPMCR_EL3.MPMMPINCTL == 0. Replacing the C
enablement with an assembly macro in each CPU's reset function achieves
the same effect with just a single close branch and a grand total of 6
instructions (versus the old 2 branches and 32 instructions).
Having done this, the device tree entry becomes redundant. Should a core
that doesn't support MPMM arise, this can cleanly be handled in the
reset function. As such, the whole ENABLE_MPMM_FCONF and platform hooks
mechanisms become obsolete and are removed.
Change-Id: I1d0475b21a1625bb3519f513ba109284f973ffdf
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.
We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.
Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.
The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.
Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
The existing DSU errata workarounds hijack the errata framework's inner
workings to register with it. However, that is undesirable as any change
to the framework may end up missing these workarounds. So convert the
checks and workarounds to macros and have them included with the
standard wrappers.
The only problem with this is the is_scu_present_in_dsu weak function.
Fortunately, it is only needed for 2 of the errata and only on 3 cores.
So drop it, assuming the default behaviour and have the callers handle
the exception.
Change-Id: Iefa36325804ea093e938f867b9a6f49a6984b8ae
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Cortex-X2 erratum 3701772 that applies to r0p0, r1p0, r2p0, r2p1
is still Open.
The workaround is for EL3 software that performs context save/restore
on a change of Security state to use a value of SCR_EL3.NS when
accessing ICH_VMCR_EL2 that reflects the Security state that owns the
data being saved or restored.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1775100/latest/
Change-Id: I2ffc5e7d7467f1bcff8b895fea52a1daa7d14495
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
Implements mitigation for CVE-2024-5660 that affects Cortex-X2
revisions r0p0, r1p0, r2p0, r2p1.
The workaround is to disable the hardware page aggregation at
EL3 by setting CPUECTLR_EL1[46] = 1'b1
Public Documentation:
https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-5660
Change-Id: If28804e154617a39d7d52c40b3a00a14a39df929
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.
Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.
Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.
Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.
Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
Cortex-X2 erratum 2778471 is a Cat B erratum that applies
to revisions r0p1, r1p0, r2p0 and r2p1 and is still open.
The workaround is to set CPUACTLR3_EL1[47] to 1.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1775100/latest
Change-Id: Ia95f0e276482283bf50e06c58c2bc5faab3f62c6
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Cortex-X2 erratum 2742423 is a Cat B erratum that applies to all
revisions <= r2p1 and is still open.
The workaround is to set CPUACTLR5_EL1[56:55] to 2'b01.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1775100/latest
Change-Id: I03897dc2a7f908937612c2b66ce7a043c1b7575d
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Update the revision and variant information in the
errata ABI file, cortex_X2.S file for erratum ID - 2058056
to match the revision and variant in the latest SDEN.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1775100/latest
Change-Id: I28ee39949d977c53d6f5243100f0c29bc3c0428c
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive
It is important to note that the errata workaround sequences remain
unchanged and preserve their git blame.
Testing was conducted by:
* Building for release with all errata flags enabled and running script
in change 19136 to compare output of objdump for each errata.
* Testing via script was not complete, as it directed to verify the
check and the workaround functions of few erratas manually.
* Manual comparison of disassembly of converted functions with non-
converted functions
aarch64-none-elf-objdump -D <trusted-firmware-a with conversion>/build/../release/bl31/bl31.elf
vs
aarch64-none-elf-objdump -D <trusted-firmware-a clean repo>/build/fvp/release/bl31/bl31.elf
* Manual comparison of disassembly of both both files(bl31.elf)
ensured,the ported changes were identical and hence verified.
* Build for release with all errata flags enabled and run default tftf
tests.
CROSS_COMPILE=aarch64-none-elf- \
make PLAT=fvp \
ARCH=aarch64 \
DEBUG=0 \
HW_ASSISTED_COHERENCY=1 \
USE_COHERENT_MEM=0 \
CTX_INCLUDE_AARCH32_REGS=0 \
ERRATA_X2_2002765=1 \
ERRATA_X2_2017096=1 \
ERRATA_X2_2058056=1 \
ERRATA_X2_2081180=1 \
ERRATA_X2_2083908=1 \
ERRATA_X2_2147715=1 \
ERRATA_X2_2216384=1 \
ERRATA_X2_2282622=1 \
ERRATA_X2_2371105=1 \
ERRATA_X2_2768515=1 \
WORKAROUND_CVE_2022_23960=1 \
ERRATA_DSU_2313941=1 \
BL33=/home/jaychi01/tf_a/tf-a-tests/build/fvp/release/tftf.bin \
fip all -j12
* Build for debug with all errata enabled and step through ArmDS
at reset to ensure that if Errata are applicable then the workaround
functions are entered precisely.
Change-Id: Icd2268cdf27f41240c92e3df23b5ad22f3ce3124
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Cortex-X2 erratum 2282622 is a Cat B erratum that applies to
all revisions <=r2p1 and is still open. The workaround is to set
CPUACTLR2_EL1[0] to 1 to force PLDW/PFRM ST to behave like
PLD/PRFM LD and not cause invalidations to other PE caches.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1775100/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I43956aa4898a8608eedc5d0dd1471172c641a0c6
Cortex-X2 erratum 2768515 is a Cat B erratum that applies to
all revisions <=r2p1 and is still open. The workaround is to
insert a dsb before the isb in the power down sequence.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1775100/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: Ib02688f7b6dc7f6ec305e68e8895174f6fd577a0
Cortex-X2 erratum 2371105 is a cat B erratum that applies to
revisions r0p0 - r2p0 and is fixed in r2p1. The workaround is to
set bit[40] of CPUACTLR2_EL1 to disable folding of demand requests
into older prefetches with L2 miss requests outstanding.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: Ib4f0caac36e1ecf049871acdea45526b394b7bad
DSU-110 erratum 2313941 is a Cat B erratum and applies to revisions
r0p0, r1p0, r2p0, r2p1, r3p0, r3p1 and is still open.
The workaround sets IMP_CLUSTERACTLR_EL1[16:15] bits to 0b11 to disable
clock gating of the SCLK domain. This will increase the idle power
consumption.
This patch applies the fix for Cortex-X2/A510/A710 and Neoverse N2.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1781796/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I54d948b23e8e01aaf1898ed9fe4e2255dd209318
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Cortex-X2 erratum 2147715 is a Cat B erratum that applies to revision
r2p0 and is fixed in r2p1. The workaround is to set CPUACTLR_EL1[22]=1,
which will cause the CFP instruction to invalidate all branch predictor
resources regardless of context.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I2d81867486d9130f2c36cd4554ca9a8f37254b57
Cortex-X2 erratum 2216384 is a Cat B erratum that applies to
revisions r0p0, r1p0 and r2p0 of CPU. It is fixed in r2p1.
The workaround is to set CPUACTLR5_EL1[17] to 1'b1 followed by
applying an instruction patching sequence.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I3c216161678887c06a28c59644e784e0c7d37bab
Cortex-X2 erratum 2081180 is a Cat B erratum present in r0p0, r1p0
and r2p0 of the Cortex-X2 processor core.
Cortex-X2 SDEN: https://developer.arm.com/documentation/SDEN1775100
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I64bed2fd5b7e12932d6de2ae668786e689885188
Cortex-X2 erratum 2017096 is a Cat B erratum that applies to
revisions r0p0, r1p0 & r2p0. The workaround is to set CPUECLTR_EL1[8]
to 1 which disables store issue prefetching.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I3b740aedc95c2394f6b8d1186014d2b2f640ae05
Cortex X2 erratum 2058056 is a Cat B erratum present in the X2 core.
It applies to revisions r0p0, r1p0, and r2p0 and is still open.
There are 2 ways this workaround can be accomplished, the first of
which involves executing a few additional instructions around MSR
writes to CPUECTLR when disabling the prefetcher. (see SDEN for
details)
However, this patch implements the 2nd possible workaround which sets
the prefetcher into its most conservative mode, since this workaround
is generic.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Idb20d9928c986616cd5bedf40bb29d46d384cfd3
Cortex X2 erratum 2002765 is a Cat B erratum present in the X2 core.
It applies to revisions r0p0, r1p0, and r2p0 and is still open.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I11576a03bfd8a6b1bd9ffef4430a097d763ca3cf
Cortex X2 erratum 2083908 is a Cat B erratum present in the Cortex
X2 core. It applies to revision r2p0 and is still open.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1775100
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Id9dca2b042bf48e75fb3013ab37d1c5925824728
This patch renames the Matterhorn, Matterhorn ELP, and Klein CPUs to
Cortex A710, Cortex X2, and Cortex A510 respectively.
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I056d3114210db71c2840a24562b51caf2546e195
