Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.
We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.
Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.
The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.
Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
The errata in this patch are declared as runtime, but are never called
explicitly. This means that they are never called! Convert them to reset
errata so that they are called at reset. Their SDENs entries have been
checked and confirm that this is how they should be implemented.
Also, drop the the MIDR check on the a57 erratum as it's not needed -
the erratum is already called from a cpu-specific function.
Change-Id: I22c3043ab454ce94b3c122c856e5804bc2ebb18b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
* changes:
fix(cpus): clear CPUPWRCTLR_EL1.CORE_PWRDN_EN_BIT on reset
chore(docs): drop the "wfi" from `pwr_domain_pwr_down_wfi`
chore(psci): drop skip_wfi variable
feat(arm): convert arm platforms to expect a wakeup
fix(cpus): avoid SME related loss of context on powerdown
feat(psci): allow cores to wake up from powerdown
refactor: panic after calling psci_power_down_wfi()
refactor(cpus): undo errata mitigations
feat(cpus): add sysreg_bit_toggle
Cortex-X3 erratum 3701769 that applies to r0p0, r1p0, r1p1 and r1p2
is still Open.
The workaround is for EL3 software that performs context save/restore
on a change of Security state to use a value of SCR_EL3.NS when
accessing ICH_VMCR_EL2 that reflects the Security state that owns the
data being saved or restored.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-2055130/latest/
Change-Id: Ifd722e1bb8616ada2ad158297a7ca80b19a3370b
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
The workarounds introduced in the three patches starting at
888eafa00b assumed that any powerdown
request will be (forced to be) terminal. This assumption can no longer
be the case for new CPUs so there is a need to revisit these older
cores. Since we may wake up, we now need to respect the workaround's
recommendation that the workaround needs to be reverted on wakeup. So do
exactly that.
Introduce a new helper to toggle bits in assembly. This allows us to
call the workaround twice, with the first call setting the workaround
and second undoing it. This is also used for gelas' an travis' powerdown
routines. This is so the same function can be called again
Also fix the condition in the cpu helper macro as it was subtly wrong
Change-Id: Iff9e5251dc9d8670d085d88c070f78991955e7c3
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Introduce a new helper to toggle bits in assembly. This allows us to
call the workaround twice, with the first call setting the workaround
and second undoing it. This allows the (errata) workaround functions to
be used to both apply and undo the mitigation.
This is applied to functions where the undo part will be required in
follow-up patches.
Change-Id: I058bad58f5949b2d5fe058101410e33b6be1b8ba
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
This patch implements SMCCC_ARCH_WORKAROUND_4 and
allows discovery through SMCCC_ARCH_FEATURES.
This mechanism is enabled if CVE_2024_7881 [1] is enabled
by the platform. If CVE_2024_7881 mitigation
is implemented, the discovery call returns 0,
if not -1 (SMC_ARCH_CALL_NOT_SUPPORTED).
For more information about SMCCC_ARCH_WORKAROUND_4 [2], please
refer to the SMCCC Specification reference provided below.
[1]: https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-7881
[2]: https://developer.arm.com/documentation/den0028/latest
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I1b1ffaa1f806f07472fd79d5525f81764d99bc79
Implements mitigation for CVE-2024-5660 that affects Cortex-X3
revisions r0p0, r1p0, r1p1, r1p2.
The workaround is to disable the hardware page aggregation at
EL3 by setting CPUECTLR_EL1[46] = 1'b1.
Public Documentation:
https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-5660
Change-Id: Ibe90313948102ece3469f2cfe3faccc7f4beeabe
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.
Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.
Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.
Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.
Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
Cortex-X3 erratum 2372204 is a Cat B erratum that applies to revisions
r0p0 and r1p0. It is fixed in r1p1.
The workaround is to set bit[40] of CPUACTLR2_EL1 to disable folding
of demand requests into older prefetches with L2 miss requests
outstanding.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2055130/latest
Change-Id: Iad28f8625c84186fbd8049406d139d4f15c6e069
Signed-off-by: Bipin Ravi <biprav01@u203721.austin.arm.com>
Cortex X3 erratum 2641945 is a Cat B erratum that applies to all
revisions <= r1p0 and is fixed in r1p1.
The workaround is to disable the affected L1 data cache prefetcher
by setting CPUACTLR6_EL1[41] to 1. Doing so will incur a performance
penalty of ~1%. Contact Arm for an alternate workaround that impacts
power.
SDEN documentation:
https://developer.arm.com/documentation/2055130/latest
Change-Id: Ia6d6ac8a66936c63b8aa8d7698b937f42ba8f044
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Cortex-X3 erratum 2266875 is a Cat B erratum that applies to
all revisions <= r1p0 and is fixed in r1p1. The workaround is to
set CPUACTLR_EL1[22]=1 which will cause the CFP instruction to
invalidate all branch predictor resources regardless of context.
SDEN Documentation:
https://developer.arm.com/documentation/2055130/latest
Change-Id: I9c610777e222f57f520d223bb03fc5ad05af1077
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Cortex-X3 erratum 2302506 is a cat B erratum that applies to
revisions r0p0-r1p1 and is fixed in r1p2. The workaround is to
set bit[0] of CPUACTLR2 which will force PLDW/PFRM ST to behave
like PLD/PRFM LD and not cause invalidation to other PE caches.
There might be a small performance degradation to this workaround
for certain workloads that share data.
SDEN can be found here:
https://developer.arm.com/documentation/2055130/latest
Change-Id: I048b830867915b88afa36582c6da05734a56d22a
Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Cortex X3 erratum 2743088 is a Cat B erratum that applies to all
revisions <= r1p1 and is fixed in r1p2. The workaround is to add a DSB
instruction before the ISB of the powerdown code sequence specified in
the TRM.
SDEN documentation: https://developer.arm.com/documentation/2055130
Change-Id: I2c8577e3ca0781af8b1c3912e577d3bd77f92709
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
Cortex-X3 erratum 2779509 is a Cat B erratum that applies to
all revisions <= r1p1 and is fixed in r1p2. The workaround is
to set chicken bit CPUACTLR3_EL1[47], this might have a small
impact on power and has negligible impact on performance.
SDEN documentation:
https://developer.arm.com/documentation/2055130/latest
Change-Id: Id92dbae6f1f313b133ffaa018fbf9c078da55d75
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Cortex-X3 erratum 2070301 is a Cat B erratum that applies to all
revisions <= r1p2 and is still open.
The workaround is to write the value 4'b1001 to the PF_MODE bits
in the IMP_CPUECTLR2_EL1 register. This places the data prefetcher
in the most conservative mode instead of disabling it.
SDEN documentation:
https://developer.arm.com/documentation/2055130/latest
Change-Id: I337c4c7bb9221715aaf973a55d0154e1c7555768
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Cortex-X3 erratum 2742421 is a Cat B erratum that applies to
all revisions <= r1p1 and is fixed in r1p2. The workaround is to
set CPUACTLR5_EL1[56:55] to 2'b01.
SDEN documentation:
https://developer.arm.com/documentation/2055130/latest
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Change-Id: Idadd323e419739fe909b9b68ea2dbe857846666b
This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive
* Manual comparison of disassembly of converted functions with non-
converted functions.
aarch64-none-elf-objdump -D <TF-A with
conversion>/build/../release/bl31/bl31.elf
vs
aarch64-none-elf-objdump -D <TF-A clean
repo>/build/fvp/release/bl31/bl31.elf
* Build for debug with all errata enabled and step through ArmDS
at reset to ensure all functions are entered.
Change-Id: I62e030962edf4e8e8be2c19e7a3176e319468c50
Signed-off-by: Sona Mathew <SonaRebecca.Mathew@arm.com>
Errata report order is enforced to be in ascending order. To achieve
this with the errata framework this has to be done at the definition
level.
Change-Id: I168bf99be0cb0b046d6b641c855f9241991bb0bc
Signed-off-by: Sona Mathew <SonaRebecca.Mathew@arm.com>
Cortex-X3 erratum 2615812 is a Cat B erratum that applies to revisions
r0p0, r1p0, and r1p1, and is still open. The workaround is to disable
the use of the Full Retention power mode in the core (setting
WFI_RET_CTRL and WFE_RET_CTRL in CORTEX_X3_IMP_CPUPWRCTLR_EL1 to 0b000).
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2055130/latest
Change-Id: I5ad66df3e18fc85a6b23f6662239494ee001d82f
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
Cortex-X3 erratum 2313909 is a Cat B erratum that applies to revisions
r0p0 and r1p0, and is fixed in r1p1. The workaround is to set
CPUACTLR2_EL1[36] to 1 before the power down sequence that sets
CORE_PWRDN_EN. This allows the cpu to retry the power down and prevents
the deadlock. TF-A never clears this bit even if it wakes up from the
wfi in the sequence since it is not expected to do anything but retry to
power down after and the bit is cleared on reset.
SDEN can be found here:
https://developer.arm.com/documentation/SDEN2055130/latest
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I5935b4bcd1e6712477c0d6eab2acc96d7964a35d
The Cortex-X3 cpu port was developed before its public release when it
was known as Makalu ELP. Now that it's released we can use the official
product name.
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: Iebb90cf2f77330ed848a3d61c5f6928942189c5a
* ASM files are renamed to have public IP names in their filename.
* updated other files to include ASM filename changes.
Signed-off-by: Rupinderjit Singh <rupinderjit.singh@arm.com>
Change-Id: Ie899c512b11fd7c4312e3a808bb6b9d2376cdb8c
2022-08-31 18:31:29 +01:00
Renamed from lib/cpus/aarch64/cortex_makalu_elp_arm.S (Browse further)
