Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.
We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.
Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.
The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.
Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Implements mitigation for CVE-2024-5660 that affects Cortex-A78_AE
revisions r0p0, r0p1, r0p2, r0p3.
The workaround is to disable the hardware page aggregation at
EL3 by setting CPUECTLR_EL1[46] = 1'b1.
Public Documentation:
https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-5660
Change-Id: I33ac653fcb45f687fe9ace1c76a3eb2000459751
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.
Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
Update the revision and variant information in the
cortex_a78_ae.s and errata ABI file for erratum ID - 2376748
based on the latest SDEN.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1707912/latest
Change-Id: I082aac41adf717b0d5d59046a8933a3f5a3de94f
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
This patch fixes the mitigation for erratum 1941500 for the
Cortex-A78AE CPUs. The right fix is to set the bit 8, whereas
the current code clears it.
Reported-by: matthias.rosenfelder@nio.io
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: Ib7c3fddd567eeae6204756377e0f77a573c0a911
This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive
It is important to note that the errata workaround and checking
sequences remain unchanged and preserve their git blame.
At this point the binary output of all errata was checked with the
script from commit 19136. The reported discrepancies are immaterial.
All errata have been checked that they get invoked.
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: Ideb6397f4ac7c3c1d04549a57af43bfa7ef25c1d
So far we have the ENABLE_AMU build option to include AMU register
handling code for enabling and context switch. There is also an
ENABLE_FEAT_AMUv1 option, solely to protect the HAFGRTR_EL2 system
register handling. The latter needs some alignment with the new feature
scheme, but it conceptually overlaps with the ENABLE_AMU option.
Since there is no real need for two separate options, unify both into a
new ENABLE_FEAT_AMU name in a first step. This is mostly just renaming at
this point, a subsequent patch will make use of the new feature handling
scheme.
Change-Id: I97d8a55bdee2ed1e1509fa9f2b09fd0bdd82736e
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Cortex A78 AE erratum 2395408 is a Cat B erratum that applies
to revisions <= r0p1. It is still open.
This erratum states, "A translation table walk that matches an
existing L1 prefetch with a read request outstanding on CHI might
fold into the prefetch, which might lead to data corruption for
a future instruction fetch"
This erratum is avoided by setting CPUACTLR2_EL1[40] to 1 to
disable folding of demand requests into older prefetches with
L2 miss requests outstanding.
SDEN is available at https://developer.arm.com/documentation/SDEN-1707912
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: Ic17968987ca3c67fa7f64211bcde6dfcb35ed5d6
Cortex A78 AE erratum 2376748 is a Cat B erratum that applies
to revisions <= r0p1. It is still open.
The erratum states, "A PE executing a PLDW or PRFM PST instruction
that lies on a mispredicted branch path might cause a second PE
executing a store exclusive to the same cache line address to fail
continuously."
The erratum is avoided by setting CPUACTLR2_EL1[0] to 1 to force
PLDW/PFRM ST to behave like PLD/PRFM LD and not cause invalidations
to other PE caches. There might be a small performance degradation
to this workaround for certain workloads that share data.
SDEN is available at https://developer.arm.com/documentation/SDEN-1707912
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: I93bd392a870d4584f3e12c8e4626dbe5a3a40a4d
Implements the loop workaround for Cortex-A76AE, Cortex-A78AE and
Cortex-A78C.
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
Change-Id: I5c838f5b9d595ed3c461a7452bd465bd54acc548
Cortex A78 AE erratum 1941500 is a Cat B erratum that applies
to revisions <= r0p1. It is still open.
This erratum is avoided by by setting CPUECTLR_EL1[8] to 1.
There is a small performance cost (<0.5%) for setting this
bit.
SDEN is available at https://developer.arm.com/documentation/SDEN1707912/0900
Change-Id: I2d72666468b146714a0340ba114ccf0f5165b39c
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Cortex A78 AE erratum 1951502 is a Cat B erratum that applies to revisions
<= r0p1. It is still open. This erratum is avoided by inserting a DMB ST
before acquire atomic instructions without release semantics through a series
of writes to implementation defined system registers.
SDEN is available at https://developer.arm.com/documentation/SDEN1707912/0900
Change-Id: I812c5a37cdd03486df8af6046d9fa988f6a0a098
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
