Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.
We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.
Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.
The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.
Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Implements mitigation for CVE-2024-5660 that affects Cortex-A78C
revisions r0p0, r0p1, r0p2.
The workaround is to disable the hardware page aggregation at
EL3 by setting CPUECTLR_EL1[46] = 1'b1.
Public Documentation:
https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-5660
Change-Id: Ieb8d7b122320d16bf8987a43dc683ca41227beb5
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.
Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.
Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.
Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.
Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
Cortex-A78C erratum 2683027 is a cat B erratum that applies to
revisions r0p1 - r0p2 and is still open. The workaround is to
execute a specific code sequence in EL3 during reset.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN-2004089/latest
Change-Id: I2bf9e675f48b62b4cd203100f7df40f4846aafa8
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Cortex-A78C erratum 2743232 is a Cat B erratum that applies
to revisions r0p1 and r0p2 and is still open.
The workaround is to set CPUACTLR5_EL1[56:55] to 2'b01.
SDEN Documentation:
https://developer.arm.com/documentation/SDEN-2004089/latest
Change-Id: Ic62579c2dd69b7a8cbbeaa936f45b2cc9436439a
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Testing:
- Manual comparison of disassembly with and without conversion.
- Using the test script in gerrit - 19136
- Building with errata and stepping through from ArmDS and running tftf.
Change-Id: Ib361cdfa43fc1c88d97e346d41b1cbf211c045d9
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
Cortex-A78C erratum 1827440 is a Cat B erratum that applies to
revision r0p0 and is fixed in r0p1.
The workaround is to set CPUACTLR2_EL1[2], which forces atomic store
operations to write-back memory to be performed in the L1 data cache.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1707916/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I41d8ef48f70216ec66bf2b0f4f03ea8d8c261ee7
Cortex-A78C erratum 1827430 is a Cat B erratum that applies to
revision r0p0 and is fixed in r0p1.
The workaround is to set the CPUECTLR_EL1[53] to 1, which disables
allocation of splintered pages in the L2 TLB.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1707916/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: Ie68771bdd3bddeff54d06b6a456dad4a7fc27426
Cortex-A78C erratum 2779484 is a Cat B erratum that applies to
revisions r0p1 and r0p2 and is still open.
The workaround is to set the CPUACTLR3_EL1[47] bit to 1. Setting this
bit might have a small impact on power and negligible impact on
performance.
SDEN documentation:
https://developer.arm.com/documentation/SDEN2004089/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I9a8c16a845c3ba6eb2f17a5119aa6ca09a0d27ed
Cortex-A78C erratum 2772121 is a Cat B erratum that applies to
all revisions <=r0p2 and is still open. The workaround is to
insert a dsb before the isb in the power down sequence.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1707916/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I0e190dabffc20c4d3b9b98d1abeb50f308b80bb9
Cortex-A78C erratum 2376749 is a Cat B erratum that applies
to revisions r0p1 and r0p2 of the A78C and is currently open.
The workaround is to set CPUACTLR2_EL1[0] to 1 to force
PLDW/PFRM ST to behave like PLD/PRFM LD and not cause
invalidations to other PE caches.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2004089/latest
Signed-off-by: Akram Ahmad <Akram.Ahmad@arm.com>
Change-Id: I3b29f4b7f167bf499d5d11ffef91a94861bd1383
Cortex-A78C erratum 2395411 is a Cat B erratum that affects
revisions r0p1 and r0p2, and is currently open. The workaround
is to set CPUACTLR2_EL1[40] to 1, which will disable folding
of demand requests into older prefetches with L2 miss requests
outstanding.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2004089/latest
Signed-off-by: Akram Ahmad <Akram.Ahmad@arm.com>
Change-Id: I4f0fb278ac20a2eb4dd7e4efd1b1246dd85e48c4
Cortex A78C erratum 2242638 is a Cat B erratum which applies to
revisions r0p1, r0p2 and is still open. The workaround is to apply
a CPU implementation specific specific patch sequence.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2004089/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I35d385245a04a39b87be71c1a42312f75e1152e5
Cortex-A78C erratum 2132064 is a cat B erratum that applies to revisions
r0p1 and r0p2 and is still open.
This patch implements workaround option 2 that places the data
prefetcher in the most conservative mode to greatly reduce prefetches
by writing the following bits to the value indicated:
ecltr[7:6], PF_MODE = 2'b11
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2004089/latest
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: Ica2561c1e257643c2482085447ef852fa62a1eb2
Implements the loop workaround for Cortex-A76AE, Cortex-A78AE and
Cortex-A78C.
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
Change-Id: I5c838f5b9d595ed3c461a7452bd465bd54acc548
