Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.
We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.
Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.
The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.
Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
The errata in this patch are declared as runtime, but are never called
explicitly. This means that they are never called! Convert them to reset
errata so that they are called at reset. Their SDENs entries have been
checked and confirm that this is how they should be implemented.
Also, drop the the MIDR check on the a57 erratum as it's not needed -
the erratum is already called from a cpu-specific function.
Change-Id: I22c3043ab454ce94b3c122c856e5804bc2ebb18b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Implements mitigation for CVE-2024-5660 that affects Neoverse-V1
revisions r0p0, r1p0, r1p1, r1p2.
The workaround is to disable the hardware page aggregation at
EL3 by setting CPUECTLR_EL1[46] = 1'b1.
Public Documentation:
https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-5660
Change-Id: Ia59452ea38c66b291790956d7f2880bfcd56d45f
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.
Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.
Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.
Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.
Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
Neoverse V1 erratum 2348377 is a Cat B erratum that applies to
all revisions <= r1p1 and is fixed in r1p2. The workaround is to
set CPUACTLR5_EL1[61] to 1.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401781/latest
Change-Id: Ica402494f78811c85e56a262e1f60b09915168fe
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Update the revision and variant information in the
errata ABI file, neoverse_v1.S file for erratum ID - 2294912
to match the revision and variant in the latest SDEN.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401781/latest
Change-Id: I38a0f53c3515860ba442b5c0872c8ab051fdda6f
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive
It is important to note that the errata workaround sequences remain
unchanged and preserve their git blame.
Testing was conducted by:
* Building for release with all errata flags enabled and running script
in change 19136 to compare output of objdump for each errata. Only
able to verify the check functions this way, rest had to manually
verified
* Manual comparison of disassembly of converted functions with non-
converted functions
aarch64-none-elf-objdump -D <trusted-firmware-a with conversion>/build/fvp/release/bl31/bl31.elf
vs
aarch64-none-elf-objdump -D <trusted-firmware-a clean repo>/build/fvp/release/bl31/bl31.elf
* Build for release with all errata flags enabled and run default tftf
tests
CROSS_COMPILE=aarch64-none-elf- make PLAT=fvp DEBUG=0 \
CTX_INCLUDE_AARCH32_REGS=0 HW_ASSISTED_COHERENCY=1 USE_COHERENT_MEM=0 \
BL33=./../tf-a-tests/build/fvp/release/tftf.bin \
ERRATA_V1_1618635=1 ERRATA_V1_1774420=1 ERRATA_V1_1791573=1 \
ERRATA_V1_1852267=1 ERRATA_V1_1925756=1 ERRATA_V1_1940577=1 \
ERRATA_V1_1966096=1 ERRATA_V1_2108267=1 ERRATA_V1_2139242=1 \
ERRATA_V1_2216392=1 ERRATA_V1_2294912=1 ERRATA_V1_2372203=1 \
ERRATA_V1_2743093=1 ERRATA_V1_2743233=1 ERRATA_V1_2779461=1 \
WORKAROUND_CVE_2022_23960=1 ERRATA_ABI_SUPPORT=1 all fip
* Build for debug with all errata enabled and step through ArmDS
at reset to ensure all functions are entered.
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: Ic5697b7cd2a508dee9978d89136fbe168f34626c
Neoverse V1 erratum 2743233 is a Cat B erratum that applies to
all revisions <= r1p2 and is still open.
The workaround sets CPUACTLR5_EL1[56:55] to 2'b01.
SDEN documentation: https://developer.arm.com/documentation/SDEN1401781/latest
Change-Id: If51a6f4293fa8b5b35c44edd564ebb715ba309a1
Signed-off-by: Sona Mathew <SonaRebecca.Mathew@arm.com>
Neoverse V1 erratum 2779461 is a Cat B erratum that applies to
all revisions <=r1p2 and is still open.
The workaround sets CPUACTLR3_EL1[47] bit to 1. Setting this
bit might have a small impact on power and negligible impact
on performance.
SDEN documentation:https://developer.arm.com/documentation/SDEN1401781/latest
Change-Id: I367cda1779684638063d7292fda20ca6734e6f10
Signed-off-by: Sona Mathew <SonaRebecca.Mathew@arm.com>
Neoverse V1 erratum 2743093 is a Cat B erratum that applies to
all revisions <=r1p2 and is still open. The workaround is to
insert a dsb before the isb in the power down sequence.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1401781/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I8ee7c16c14c4fd6ee35d20c855273ecfce0d1b32
The CPU version check was moved wrongly down in N2 and missing in V1.
The patch fixes the issues.
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: Icb6e5285d6cc97fbe416fe1f0b1ab7afbd8a8809
Neoverse-V1 erratum 1618635 is a Cat B erratum that applies to
revision r0p0. It is fixed in r1p0.
The workaround is done through the instruction patching
mechanism, which is performed by a write sequence of
IMPLEMENTATION DEFINED registers.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1401781/latest/
Signed-off-by: Juan Pablo Conde <juanpablo.conde@arm.com>
Change-Id: I53e406735cd3a2a930fdc72ebce3bbed97100168
Neoverse-V1 erratum 2372203 is a cat B erratum that applies to revisions
r0p0 - r1p1 and is still open. The workaround is to set bit[40] of
CPUACTLR2_EL1 to disable folding of demand requests into older
prefetches with L2 miss requests outstanding.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1401781/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: Ice8c2e5a0152972a35219c8245a2e07e646d0557
Neoverse-V1 erratum 2294912 is a cat B erratum that applies to revisions
r0p0 - r1p1 and is still open. The workaround is to set bit[0] of
CPUACTLR2_EL1 to force PLDW/PFRM ST to behave like PLD/PRFM LD and not
cause invalidations to other PE caches.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1401781/latest
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: Ia7afb4c42fe66b36fdf38a7d4281a0d168f68354
Neoverse V1 erratum 2216392 is a Cat B erratum present in the V1 core.
It applies to revisions r1p0 and r1p1 and is still open. The issue is
also present in r0p0 but there is no workaround in that revision.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1401781
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Ic2f90d79c75e8ffef01aac81eddf1bfd8b7164ab
Neoverse-V1 erratum 2108267 is a Cat B erratum that applies to
revisions r0p0, r1p0, and r1p1 of CPU. It is still open. The
workaround is to write the value 2'b11 to the PF_MODE bits in
the CPUECTLR_EL1 register which will place the data prefetcher
in the most conservative mode instead of disabling it.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN1401781/latest
Signed-off-by: nayanpatel-arm <nayankumar.patel@arm.com>
Change-Id: Iedcb84a7ad34af7083116818f49d7296f7d9bf94
Neoverse V1 erratum 2139242 is a Cat B erratum present in the V1
processor core. This issue is present in revisions r0p0, r1p0,
and r1p1, and it is still open.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I5c2e9beec72a64ac4131fb6dd76199821a934ebe
Neoverse V1 erratum 1966096 is a Cat B erratum present in the V1
processor core. This issue is present in revisions r0p0, r1p0,
and r1p1, but the workaround only applies to r1p0 and r1p1, it is still
open.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Ic0b9a931e38da8a7000648e221481e17c253563b
Neoverse V1 erratum 1925756 is a Cat B erratum present in r0p0, r1p0,
and r1p1 of the V1 processor core, and it is still open.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: I6500dc98da92a7c405b9ae09d794d666e8f4ae52
Neoverse V1 erratum 1852267 is a Cat B erratum present in r0p0 and
r1p0 of the V1 processor core. It is fixed in r1p1.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: Ide5e0bc09371fbc91c2385ffdff74e604beb2dbe
Neoverse V1 erratum 1774420 is a Cat B erratum present in r0p0 and
r1p0 of the V1 processor core. It is fixed in r1p1.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: I66e27b2518f73faeedd8615a1443a74b6a30f123
Neoverse V1 erratum 1940577 is a Cat B erratum, present in some
revisions of the V1 processor core. The workaround is to insert a
DMB ST before acquire atomic instructions without release semantics.
This issue is present in revisions r0p0 - r1p1 but this workaround
only applies to revisions r1p0 - r1p1, there is no workaround for older
versions.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I210ad7d8f31c81b6ac51b028dfbce75a725c11aa
Neoverse V1 erratum 1791573 is a Cat B erratum present in r0p0 and
r1p0 of the V1 processor core. It is fixed in r1p1.
SDEN can be found here:
https://documentation-service.arm.com/static/60d499080320e92fa40b4625
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Ic6f92da4d0b995bd04ca5b1673ffeedaebb71d10
