Commit graph

292 commits

Author SHA1 Message Date
Boyan Karatotev
0a580b5128 perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context
SVE and SME aren't enabled symmetrically for all worlds, but EL3 needs
to context switch them nonetheless. Previously, this had to happen by
writing the enable bits just before reading/writing the relevant
context. But since the introduction of root context, this need not be
the case. We can have these enables always be present for EL3 and save
on some work (and ISBs!) on every context switch.

We can also hoist ZCR_EL3 to a never changing register, as we set its
value to be identical for every world, which happens to be the one we
want for EL3 too.

Change-Id: I3d950e72049a298008205ba32f230d5a5c02f8b0
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-25 08:52:06 +00:00
Boyan Karatotev
83ec7e452c perf(amu): greatly simplify AMU context management
The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness in that this is rather slow. A large part of it is written in
assembly, making it opaque to the compiler for optimisations. The
future proofness requires reading registers that are effectively
`volatile`, making it even harder for the compiler, as well as adding
lots of implicit barriers, making it hard for the microarchitecutre to
optimise as well.

We can make a few assumptions, checked by a few well placed asserts, and
remove a lot of this burden. For a start, at the moment there are 4
group 0 counters with static assignments. Contexting them is a trivial
affair that doesn't need a loop. Similarly, there can only be up to 16
group 1 counters. Contexting them is a bit harder, but we can do with a
single branch with a falling through switch. If/when both of these
change, we have a pair of asserts and the feature detection mechanism to
guard us against pretending that we support something we don't.

We can drop contexting of the offset registers. They are fully
accessible by EL2 and as such are its responsibility to preserve on
powerdown.

Another small thing we can do, is pass the core_pos into the hook.
The caller already knows which core we're running on, we don't need to
call this non-trivial function again.

Finally, knowing this, we don't really need the auxiliary AMUs to be
described by the device tree. Linux doesn't care at the moment, and any
information we need for EL3 can be neatly placed in a simple array.

All of this, combined with lifting the actual saving out of assembly,
reduces the instructions to save the context from 180 to 40, including a
lot fewer branches. The code is also much shorter and easier to read.

Also propagate to aarch32 so that the two don't diverge too much.

Change-Id: Ib62e6e9ba5be7fb9fb8965c8eee148d5598a5361
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-25 08:50:46 +00:00
Boyan Karatotev
2590e819eb perf(mpmm): greatly simplify MPMM enablement
MPMM is a core-specific microarchitectural feature. It has been present
in every Arm core since the Cortex-A510 and has been implemented in
exactly the same way. Despite that, it is enabled more like an
architectural feature with a top level enable flag. This utilised the
identical implementation.

This duality has left MPMM in an awkward place, where its enablement
should be generic, like an architectural feature, but since it is not,
it should also be core-specific if it ever changes. One choice to do
this has been through the device tree.

This has worked just fine so far, however, recent implementations expose
a weakness in that this is rather slow - the device tree has to be read,
there's a long call stack of functions with many branches, and system
registers are read. In the hot path of PSCI CPU powerdown, this has a
significant and measurable impact. Besides it being a rather large
amount of code that is difficult to understand.

Since MPMM is a microarchitectural feature, its correct placement is in
the reset function. The essence of the current enablement is to write
CPUPPMCR_EL3.MPMM_EN if CPUPPMCR_EL3.MPMMPINCTL == 0. Replacing the C
enablement with an assembly macro in each CPU's reset function achieves
the same effect with just a single close branch and a grand total of 6
instructions (versus the old 2 branches and 32 instructions).

Having done this, the device tree entry becomes redundant. Should a core
that doesn't support MPMM arise, this can cleanly be handled in the
reset function. As such, the whole ENABLE_MPMM_FCONF and platform hooks
mechanisms become obsolete and are removed.

Change-Id: I1d0475b21a1625bb3519f513ba109284f973ffdf
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-25 08:50:45 +00:00
Manish V Badarkhe
a8a5d39d6e Merge changes from topic "bk/errata_speed" into integration
* changes:
  refactor(cpus): declare runtime errata correctly
  perf(cpus): make reset errata do fewer branches
  perf(cpus): inline the init_cpu_data_ptr function
  perf(cpus): inline the reset function
  perf(cpus): inline the cpu_get_rev_var call
  perf(cpus): inline cpu_rev_var checks
  refactor(cpus): register DSU errata with the errata framework's wrappers
  refactor(cpus): convert checker functions to standard helpers
  refactor(cpus): convert the Cortex-A65 to use the errata framework
  fix(cpus): declare reset errata correctly
2025-02-24 17:24:53 +01:00
Boyan Karatotev
b07c317f67 perf(cpus): inline the init_cpu_data_ptr function
Similar to the reset function inline, inline this too to not do a costly
branch with no extra cost.

Change-Id: I54cc399e570e9d0f373ae13c7224d32dbdfae1e5
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-24 09:36:11 +00:00
Boyan Karatotev
0d020822ae perf(cpus): inline the reset function
Similar to the cpu_rev_var and cpu_ger_rev_var functions, inline the
call_reset_handler handler. This way we skip the costly branch at no
extra cost as this is the only place where this is called.

While we're at it, drop the options for CPU_NO_RESET_FUNC. The only cpus
that need that are virtual cpus which can spare the tiny bit of
performance lost. The rest are real cores which can save on the check
for zero.

Now is a good time to put the assert for a missing cpu in the
get_cpu_ops_ptr function so that it's a bit better encapsulated.

Change-Id: Ia7c3dcd13b75e5d7c8bafad4698994ea65f42406
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-24 09:36:10 +00:00
Sona Mathew
41ae047352 fix(rmm): add support for BRBCR_EL2 register for feat_brbe
Currently BRBE is being disabled for Realm world in EL3 by
switching the SBRBE bit in mdcr_el3 register to 0b00.
The patch removes the switching of SBRBE bits, and adds
context switch of BRBCR_EL2 register.

Change-Id: I66ca13edefc37e40fa265fd438b0b66f7d09b4bb
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
2025-02-20 21:13:32 -06:00
Manish Pandey
fcb80d7d14 Merge changes I765a7fa0,Ic33f0b6d,I8d1a88c7,I381f96be,I698fa849, ... into integration
* changes:
  fix(cpus): clear CPUPWRCTLR_EL1.CORE_PWRDN_EN_BIT on reset
  chore(docs): drop the "wfi" from `pwr_domain_pwr_down_wfi`
  chore(psci): drop skip_wfi variable
  feat(arm): convert arm platforms to expect a wakeup
  fix(cpus): avoid SME related loss of context on powerdown
  feat(psci): allow cores to wake up from powerdown
  refactor: panic after calling psci_power_down_wfi()
  refactor(cpus): undo errata mitigations
  feat(cpus): add sysreg_bit_toggle
2025-02-11 16:52:18 +01:00
Igor Podgainõi
58fadd62be fix: add support for 128-bit sysregs to EL3 crash handler
The following changes have been made:
* Add new sysreg definitions and ASM macro is_feat_sysreg128_present_asm
* Add registers TTBR0_EL2 and VTTBR_EL2 to EL3 crash handler output
* Use MRRS instead of MRS for registers TTBR0_EL1, TTBR0_EL2, TTBR1_EL1,
  VTTBR_EL2 and PAR_EL1

Change-Id: I0e20b2c35251f3afba2df794c1f8bc0c46c197ff
Signed-off-by: Igor Podgainõi <igor.podgainoi@arm.com>
2025-02-05 21:19:15 +01:00
Boyan Karatotev
45c7328c0b fix(cpus): avoid SME related loss of context on powerdown
Travis' and Gelas' TRMs tell us to disable SME (set PSTATE.{ZA, SM} to
0) when we're attempting to power down. What they don't tell us is that
if this isn't done, the powerdown request will be rejected. On the
CPU_OFF path that's not a problem - we can force SVCR to 0 and be
certain the core will power off.

On the suspend to powerdown path, however, we cannot do this. The TRM
also tells us that the sequence could also be aborted on eg. GIC
interrupts. If this were to happen when we have overwritten SVCR to 0,
upon a return to the caller they would experience a loss of context. We
know that at least Linux may call into PSCI with SVCR != 0. One option
is to save the entire SME context which would be quite expensive just to
work around. Another option is to downgrade the request to a normal
suspend when SME was left on. This option is better as this is expected
to happen rarely enough to ignore the wasted power and we don't want to
burden the generic (correct) path with needless context management.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I698fa8490ebf51461f6aa8bba84f9827c5c46ad4
2025-02-03 14:29:47 +00:00
Arvind Ram Prakash
6b8df7b9e5 feat(mops): enable FEAT_MOPS in EL3 when INIT_UNUSED_NS_EL2=1
FEAT_MOPS, mandatory from Arm v8.8, is typically managed in EL2.
However, in configurations where NS_EL2 is not enabled,
EL3 must set the HCRX_EL2.MSCEn bit to 1 to enable the feature.

This patch ensures FEAT_MOPS is enabled by setting HCRX_EL2.MSCEn to 1.

Change-Id: Ic4960e0cc14a44279156b79ded50de475b3b21c5
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
2025-01-14 15:30:19 -06:00
Boyan Karatotev
79c0c7fac0 refactor(cm): clean up per-world context
In preparation for SMCCC_ARCH_FEATURE_AVAILABILITY, it is useful for
context to be directly related to the underlying system. Currently,
certain bits like SCR_EL3.APK are always set with the understanding that
they will only take effect if the feature is present.

However, that is problematic for SMCCC_ARCH_FEATURE_AVAILABILITY (an
SMCCC call to report which features firmware enables), as simply reading
the enable bit may contradict the ID register, like the APK bit above
for a system with no Pauth present.

This patch is to clean up these cases. Add a check for PAuth's presence
so that the APK bit remains unset if not present. Also move SPE and TRBE
enablement to only the NS context. They already only enable the features
for NS only and disable them for Secure and Realm worlds. This change
only makes these worlds' context read 0 for easy bitmasking.

There's only a single snag on SPE and TRBE. Currently, their fields have
the same values and any world asymmetry is handled by hardware. Since we
don't want to do that, the buffers' ownership will change if we just set
the fields to 0 for non-NS worlds. Doing that, however, exposes Secure
state to a potential denial of service attack - a malicious NS can
enable profiling and call an SMC. Then, the owning security state will
change and since no SPE/TRBE registers are contexted, Secure state will
start generating records. Always have NS world own the buffers to
prevent this.

Finally, get rid of manage_extensions_common() as it's just a level of
indirection to enable a single feature.

Change-Id: I487bd4c70ac3e2105583917a0e5499e0ee248ed9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-01-07 07:59:28 +00:00
Boyan Karatotev
fc7dca72ba refactor(cm): change owning security state when a feature is disabled
SPE and TRBE don't have an outright EL3 disable, there are only
constraints on what's allowed. Since we only enable them for NS at the
moment, we want NS to own the buffers even when the feature should be
"disabled" for a world. This means that when we're running in NS
everything is as normal but when running in S/RL then tracing is
prohibited (since the buffers are owned by NS). This allows us to fiddle
with context a bit more without having to context switch registers.

Change-Id: Ie1dc7c00e4cf9bcc746f02ae43633acca32d3758
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-01-06 07:38:23 +00:00
Chris Kay
bdcef87cf5 feat(aarch64): add DBGPRCR_EL1 register accessors
This is a small change adding accessor functions for the Debug Power
Control register (DBGPRCR_EL1) to the common architectural helpers.

Change-Id: I72261fbf0395d900347b46af320093ed946aa73d
Signed-off-by: Chris Kay <chris.kay@arm.com>
2025-01-02 11:23:55 +01:00
Boyan Karatotev
73d98e3759 fix(trbe): add a tsb before context switching
Just like for SPE, we need to synchronize TRBE samples before we change
the context to ensure everything goes where it was intended to. If that
is not done, the in-flight entries might use any piece of now incorrect
context as there are no implicit ordering requirements.

Prior to root context, the buffer drain hooks would have done that. But
now that must happen much earlier. So add a tsb to prepare_el3_entry as
well.

Annoyingly, the barrier can be reordered relative to other instructions
by default (rule RCKVWP). So add an isb after the psb/tsb to assure that
they are ordered, at least as far as context is concerned.

Then, drop the buffer draining hooks. Everything they need to do is
already done by now. There's a notable difference in that there are no
dsb-s now. Since EL3 does not access the buffers or the feature
specific context, we don't need to wait for them to finish.

Finally, drop a stray isb in the context saving macro. It is now
absorbed into root context, but was missed.

Change-Id: I30797a40ac7f91d0bb71ad271a1597e85092ccd5
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2024-12-16 15:14:30 +00:00
Boyan Karatotev
f808873372 fix(spe): add a psb before updating context and remove context saving
In the chapter about FEAT_SPE (D16.4 specifically) it is stated that
"Sampling is always disabled at EL3". That means that disabling sampling
(writing PMBLIMITR_EL1.E to 0) is redundant and can be removed. The only
reason we save/restore SPE context is because of that disable, so those
can be removed too.

There's the issue of draining the profiling buffer though. No new
samples will have been generated since entering EL3. However, old
samples might still be in-flight. Unless synchronised by a psb csync,
those might be affected by our extensive context mutation. Adding a psb
in prepare_el3_entry should cater for that. Note that prior to the
introduction of root context this was not a problem as context remained
unchanged and the hooks took care of the rest.

Then, the only time we care about the buffer actually making it to
memory is when we exit coherency. On HW_ASSISTED_COHERENCY systems we
don't have to do anything, it should be handled for us. Systems without
it need a dsb to wait for them to complete. There should be one already
in each cpu's powerdown hook which should work.

While on the topic of barriers, the esb barrier is no longer used.
Remove it.

Change-Id: I9736fc7d109702c63e7d403dc9e2a4272828afb2
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2024-12-16 15:14:30 +00:00
Arvind Ram Prakash
a57e18e433 feat(fpmr): disable FPMR trap
This patch enables support of FEAT_FPMR by enabling access
to FPMR register. It achieves it by setting the EnFPM bit of
SCR_EL3. This feature is currently enabled for NS world only.

Reference:
https://developer.arm.com/documentation/109697/2024_09/
Feature-descriptions/The-Armv9-5-architecture-extension?lang=en

Change-Id: I580c409b9b22f8ead0737502280fb9093a3d5dd2
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
2024-12-12 10:03:23 -06:00
Yann Gautier
b40bc36c20 Merge "build(bl31): support separated memory for RW DATA" into integration 2024-11-07 18:10:51 +01:00
Andre Przywara
19d52a83b7 feat(cpufeat): add ENABLE_FEAT_LS64_ACCDATA
Armv8.6 introduced the FEAT_LS64 extension, which provides a 64 *byte*
store instruction. A related instruction is ST64BV0, which will replace
the lowest 32 bits of the data with a value taken from the ACCDATA_EL1
system register (so that EL0 cannot alter them).
Using that ST64BV0 instruction and accessing the ACCDATA_EL1 system
register is guarded by two SCR_EL3 bits, which we should set to avoid a
trap into EL3, when lower ELs use one of those.

Add the required bits and pieces to make this feature usable:
- Add the ENABLE_FEAT_LS64_ACCDATA build option (defaulting to 0).
- Add the CPUID and SCR_EL3 bit definitions associated with FEAT_LS64.
- Add a feature check to check for the existing four variants of the
  LS64 feature and detect future extensions.
- Add code to save and restore the ACCDATA_EL1 register on
  secure/non-secure context switches.
- Enable the feature with runtime detection for FVP and Arm FPGA.

Please note that the *basic* FEAT_LS64 feature does not feature any trap
bits, it's only the addition of the ACCDATA_EL1 system register that
adds these traps and the SCR_EL3 bits.

Change-Id: Ie3e2ca2d9c4fbbd45c0cc6089accbb825579138a
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2024-11-06 16:52:12 +01:00
Ye Li
86acbbe2d8 build(bl31): support separated memory for RW DATA
Update linker file and init codes to allow using separated
memory region for RW DATA. Init codes will copy the RW DATA
from the image to the linked address.

On some NXP platforms, after the BL31 image has been verified,
the bl31 image space will be locked/protected as RO only, so
need to move the RW DATA and NOBITS out of the bl31 image.

Signed-off-by: Ye Li <ye.li@nxp.com>
Reviewed-by: Peng Fan <peng.fan@nxp.com>
Signed-off-by: Jacky Bai <ping.bai@nxp.com>
Change-Id: I361d9a715890961bf30790a3325f8085a40c0c39
2024-11-05 17:24:41 +08:00
Govindraj Raja
306551362c feat(d128): add support for FEAT_D128
This patch disables trapping to EL3 when the FEAT_D128
specific registers are accessed by setting the SCR_EL3.D128En bit.

If FEAT_D128 is implemented, then FEAT_SYSREG128 is implemented.
With FEAT_SYSREG128 certain system registers are treated as 128-bit,
so we should be context saving and restoring 128-bits instead of 64-bit
when FEAT_D128 is enabled.

FEAT_SYSREG128 adds support for MRRS and MSRR instruction which
helps us to read write to 128-bit system register.
Refer to Arm Architecture Manual for further details.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I1a53db5eac29e56c8fbdcd4961ede3abfcb2411a
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
2024-10-24 14:51:55 -05:00
Olivier Deprez
5ec9ade9dd Merge "fix(brbe): allow RME builds with BRBE" into integration 2024-10-21 12:12:17 +02:00
Manish Pandey
17ef5da7fd Merge "feat(context-mgmt): introduce EL3/root context" into integration 2024-10-18 14:44:19 +02:00
Boyan Karatotev
9890eab574 fix(brbe): allow RME builds with BRBE
It used to be the case that a FEAT_RME build could not be built with
FEAT_BRBE support. BRBE doesn't have a 3-world aware disable and
MDCR_EL3 was not context switched to allow for disabling in Realm world.

As of commit 123002f917 MDCR_EL3 is
context switched. Since the flag for BRBE support is
ENABLE_BRBE_FOR_NS, move brbe_enable() to only happen for NS world. The
other worlds will see BRBE disabled and branch recording prohibited.
This allows for a build with both RME and BRBE.

Note that EL2 BRBE registers are not context switched. Further work is
needed if non-NS support is required.

Change-Id: I82f0f08399dcd080902477dc9636bc4541685f89
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2024-10-18 11:30:34 +01:00
Jayanth Dodderi Chidanand
40e5f7a58f feat(context-mgmt): introduce EL3/root context
* This patch adds root context procedure to restore/configure
  the registers, which are of importance during EL3 execution.

* EL3/Root context is a simple restore operation that overwrites
  the following bits: (MDCR_EL3.SDD, SCR_EL3.{EA, SIF}, PMCR_EL0.DP
  PSTATE.DIT) while the execution is in EL3.

* It ensures EL3 world maintains its own settings distinct
  from other worlds (NS/Realm/SWd). With this in place, the EL3
  system register settings is no longer influenced by settings of
  incoming worlds. This allows the EL3/Root world to access features
  for its own execution at EL3 (eg: Pauth).

* It should be invoked at cold and warm boot entry paths and also
  at all the possible exception handlers routing to EL3 at runtime.
  Cold and warm boot paths are handled by including setup_el3_context
  function in  "el3_entrypoint_common"  macro, which gets invoked in
  both the entry paths.

* At runtime, el3_context is setup at the stage, while we get prepared
  to enter into EL3 via "prepare_el3_entry" routine.

Change-Id: I5c090978c54a53bc1c119d1bc5fa77cd8813cdc2
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2024-10-15 13:25:58 +01:00
Jayanth Dodderi Chidanand
4ec4e545c6 feat(sctlr2): add support for FEAT_SCTLR2
Arm v8.9 introduces FEAT_SCTLR2, adding SCTLR2_ELx registers.
Support this, context switching the registers and disabling
traps so lower ELs can access the new registers.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I0c4cba86917b6b065a7e8dd6af7daf64ee18dcda
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
2024-10-13 18:44:54 -05:00
Jayanth Dodderi Chidanand
6d0433f040 feat(the): add support for FEAT_THE
Arm v8.9 introduces FEAT_THE, adding Translation Hardening Extension
Read-Check-Write mask registers, RCWMASK_EL1 and RCWSMASK_EL1.
Support this, context switching the registers and disabling
traps so lower ELs can access the new registers.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I8775787f523639b39faf61d046ef482f73b2a562
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
2024-10-13 18:44:54 -05:00
Arvind Ram Prakash
1073bf3d8c refactor(errata-abi): move EXTRACT_PARTNUM to arch.h
This patch moves EXTRACT_PARTNUM from errata abi includes
to arch.h which is part of common includes

Change-Id: Id8bbaf21566f3145a75cfa0dafec6823ed2df3a9
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
2024-08-17 09:38:31 +01:00
Manish Pandey
b4f72cfa08 Merge "refactor(cpufeat): feat detect helpers inlining" into integration 2024-08-06 15:58:57 +02:00
Jayanth Dodderi Chidanand
59b7c0a03f feat(cm): add explicit context entries for ERRATA_SPECULATIVE_AT
* Currently, "ERRATA_SPECUALTIVE_AT" errata is enabled by default
  for few cores and they need context entries for saving and
  restoring EL1 regs "SCTLR_EL1 and TCR_EL1" registers at all times.

* This prevents the mechanism of decoupling EL1 and EL2 registers,
  as EL3 firmware shouldn't be handling both simultaneously.

* Depending on the build configuration either EL1 or EL2 context
  structures need to included, which would result in saving a good
  amount of context memory.

* In order to achieve this it's essential to have explicit context
  entries for registers supporting "ERRATA_SPECULATIVE_AT".

* This patch adds two context entries under "errata_speculative_at"
  structure to assist this errata and thereby allows decoupling
  EL1 and EL2 context structures.

Change-Id: Ia50626eea8fb64899a2e2d81622adbe07fe77d65
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2024-07-26 15:36:31 +01:00
Arvind Ram Prakash
33e6aaacf1 feat(fgt2): add support for FEAT_FGT2
This patch disables trapping to EL3 when the FEAT_FGT2
specific trap registers are accessed by setting the
SCR_EL3.FGTEn2 bit

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I6d2b614affb9067b2bc3d7bf0ae7d169d031592a
2024-07-18 13:49:43 -05:00
Arvind Ram Prakash
83271d5a5a feat(debugv8p9): add support for FEAT_Debugv8p9
This patch enables FEAT_Debugv8p9 and prevents EL1/0 from
trapping to EL3 when accessing MDSELR_EL1 register by
setting the MDCR_EL3.EBWE bit.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I3613af1dd8cb8c0d3c33dc959f170846c0b9695a
2024-07-18 13:49:43 -05:00
Olivier Deprez
0dfa07b6b2 refactor(cpufeat): feat detect helpers inlining
Force inlining feat detect helpers such that context save/restore
operations are flattened with sequences of ID regs read and conditional
instructions for system registers read/write. This is opposed to current
situation where with -Os optimization level, feat detect helpers get
called through non-inlined sequences of branch-link+ret.

Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Change-Id: I2633442fb0e69e4a4ed13467e65846fb66d214f6
2024-07-12 13:59:32 +02:00
Manish Pandey
2e0efb3f40 Merge "feat(cm): context switch MDCR_EL3 register" into integration 2024-06-27 23:18:27 +02:00
Jayanth Dodderi Chidanand
123002f917 feat(cm): context switch MDCR_EL3 register
Currently MDCR_EL3 register value is same for all the
worlds(Non-secure, Secure, Realm and Root).

With this approach, features enable/disable settings
remain same across all the worlds. This is not ideal as
there must be flexibility in controlling feature as per
the requirements for individual world.

The patch addresses this by providing MDCR_EL3 a per world
value. Features with identical values for all the worlds are
grouped under ``manage_extensions_common`` API.

Change-Id: Ibc068d985fe165d8cb6d0ffb84119bffd743b3d1
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2024-06-25 13:50:32 +01:00
Andre Przywara
515d2d46a3 feat(cpufeat): upgrade PMU to v8 (FEATURE_DETECTION)
The ARMv8 ARM J.a describes the ID_AA64DFR0_EL1.PMUver field as allowing
a maximum version number of 8 now. The added features extend the Common
event number space and clarify on some UNPREDICTABLE behaviour.

None of this affects TF-A or any system registers, so just increase the
maximum known version number to let the FEATURE_DETECTION test pass on
ARMv8.8 implementations.

Change-Id: Icab48630c1635bcd78a710b443f0db01b8ff7c9b
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2024-06-21 13:52:44 +01:00
AlexeiFedorov
ec0088bbab feat(gpt): add support for large GPT mappings
This patch adds support for large GPT mappings using
Contiguous descriptors. The maximum size of supported
contiguous block in MB is defined in RME_GPT_MAX_BLOCK
build parameter and takes values 0, 2, 32 and 512 and
by default set to 2 in make_helpers/defaults.mk.
Setting RME_GPT_MAX_BLOCK value to 0 disables use of
Contiguous descriptors.
Function gpt_tlbi_by_pa_ll() and its declaration
are removed from lib/aarch64/misc_helpers.S and
include/arch/aarch64/arch_helpers.h, because the
GPT library now uses tlbirpalos_xxx() functions.

Change-Id: Ia9a59bde1741c5666b4ca1de9324e6dfd6f734eb
Signed-off-by: AlexeiFedorov <Alexei.Fedorov@arm.com>
2024-05-14 10:36:42 +02:00
Arvind Ram Prakash
f99a69c386 feat(dsu): save/restore DSU PMU register
Adds driver support to preserve DSU PMU register values over a DSU
power cycle. This driver needs to be enabled by the platforms that
support DSU and also need it's PMU registers to be preserved

Change-Id: I7fc68a3d7d99ee369379aa5cd114fffc763fc0d2
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
2024-05-09 10:20:32 -05:00
Manish Pandey
2a0ca84f47 Merge changes from topic "sm/feat_detect" into integration
* changes:
  refactor(cpufeat): restore functions in detect_arch_features
  refactor(cpufeat): add macro to simplify is_feat_xx_present
  chore: simplify the macro names in ENABLE_FEAT mechanism
2024-05-07 11:17:02 +02:00
Hsin-Hsiung Wang
31857d4cba refactor(build): introduce adr_l macro
Introduce the macro "adr_l," which can handle symbols or labels that
exceed the 1MB access range compared to the "adr" instruction.

Change-Id: Iab2a2a2f8a11a5e21e386f1001ba27a8de621132
Signed-off-by: Hsin-Hsiung Wang <hsin-hsiung.wang@mediatek.com>
2024-05-03 14:08:13 +02:00
Sona Mathew
aaaf2cc313 refactor(cpufeat): add macro to simplify is_feat_xx_present
In this patch, we are trying to introduce the wrapper macro
CREATE_FEATURE_PRESENT to get the following capability and
align it for all the features:

-> is_feat_xx_present(): Does Hardware implement the feature.
-> uniformity in naming the function across multiple features.
-> improved readability

The is_feat_xx_present() is implemented to check if the hardware
implements the feature and does not take into account the
ENABLE_FEAT_XXX flag enabled/disabled in software.

- CREATE_FEATURE_PRESENT(name, idreg, shift, mask, idval)
The wrapper macro reduces the function to a single line and
creates the is_feat_xx_present function that checks the
id register based on the shift and mask values and compares
this against a determined idvalue.

Change-Id: I7b91d2c9c6fbe55f94c693aa1b2c50be54fb9ecc
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
2024-05-02 12:16:16 -05:00
Sona Mathew
9e51f15ed1 chore: simplify the macro names in ENABLE_FEAT mechanism
Currently, the macros used to denote feature implementation
in hardware follow a random pattern with a few macros having
suffix as SUPPORTED and a few using the suffix IMPLEMENTED.
This patch aligns the macro names uniformly using the suffix
IMPLEMENTED across all the features and removes unused macros
pertaining to the Enable feat mechanism.

FEAT_SUPPORTED --> FEAT_IMPLEMENTED
FEAT_NOT_SUPPORTED --> FEAT_NOT_IMPLEMENTED

Change-Id: I61bb7d154b23f677b80756a4b6a81f74b10cd24f
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
2024-05-02 08:53:01 -05:00
Madhukar Pappireddy
d6c76e6c65 fix(cm): add more feature registers to EL1 context mgmt
The following system registers are made part of save and restore
operations for EL1 context:

TRFCR_EL1
SCXTNUM_EL0
SCXTNUM_EL1
GCSCR_EL1
GCSCRE0_EL1
GCSPR_EL1
GCSPR_EL0

Change-Id: I1077112bdc29a6c9cd39b9707d6cf10b95fa15e3
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
2024-04-19 09:57:36 -05:00
Soby Mathew
832e4ed520 fix(gpt): declare gpt_tlbi_by_pa_ll()
The patch 8754cc5 accidentally removes the declaration of
gpt_tlbi_by_pa_ll() and hence breaks RME builds. This patch
fixes the same.

signed-off-by: Soby Mathew <soby.mathew@arm.com>
Change-Id: I2523982fc48bca2a1f1a36fd9bd3803b01c6916a
2024-04-19 12:27:36 +02:00
Soby Mathew
98f7b60ec6 Merge changes Ic40e1b7a,I0398b550,Ife594ed6,I3eb0f29b into integration
* changes:
  fix(gpt): unify logging messages
  chore(gpt): remove gpt_ prefix
  feat(aarch64): add functions for TLBI RPALOS
  feat(locks): add bitlock
2024-04-19 11:24:01 +02:00
AlexeiFedorov
8754cc5d1c feat(aarch64): add functions for TLBI RPALOS
This patch adds tlbirpalos_XYZ() functions to support
TLBI RPALOS instructions for the 4KB-512MB invalidation
range.

Change-Id: Ife594ed6c746d356b4b1fdf97001a0fe2b5e8cd0
Signed-off-by: AlexeiFedorov <Alexei.Fedorov@arm.com>
2024-04-17 12:59:04 +01:00
Madhukar Pappireddy
ed9bb824e4 fix(cm): add more system registers to EL1 context mgmt
The following system registers are made part of save and restore
operations for EL1 context:
  MDCCINT_EL1
  MDSCR_EL1
  DISR_EL1
  PIRE0_EL1
  PIR_EL1
  POR_EL1
  S2POR_EL1
  TCR2_EL1

Some of these registers are available as part of core Armv8-A
architecture while others are made available through various
architectural extensions.

Change-Id: I507dccb9053ba177e1b98100fceccd1f32bdfc5c
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
2024-04-03 11:08:35 -05:00
Govindraj Raja
c282384dbb refactor(mte): remove mte, mte_perm
Currently both FEAT_MTE and FEAT_MTE_PERM aren't used for enabling
of any feature bits in EL3. So remove both FEAT handling.

All mte regs that are currently context saved/restored are needed
only when FEAT_MTE2 is enabled, so move to usage of FEAT_MTE2 and
remove FEAT_MTE usage.

BREAKING CHANGE: Any platform or downstream code trying to use
SCR_EL3.ATA bit(26) will see failures as this is now moved to be
used only with FEAT_MTE2 with
commit@ef0d0e5478a3f19cbe70a378b9b184036db38fe2

Change-Id: Id01e154156571f7792135639e17dc5c8d0e17cf8
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
2024-03-26 14:30:58 -05:00
Harrison Mutai
33c665ae95 fix(cpus): workaround for Cortex-A715 erratum 2344187
Cortex-A715 erratum 2344187 is a Cat B erratum that applies to r0p0,
r1p0 and is fixed in r1p1. The workaround is to set GCR_EL1.RRND to
0b1, and apply an implementation specific patch sequence.

SDEN: https://developer.arm.com/documentation/SDEN2148827/latest

Change-Id: I78ea39a91254765c964bff89f771af33b23f29c1
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
2024-03-07 10:19:56 +00:00
Manish V Badarkhe
c2f9ba88f4 Merge changes from topic "mp/undef_injection" into integration
* changes:
  feat(el3-runtime): introduce UNDEF injection to lower EL
  feat(cpufeat): added few helper functions
2024-02-28 14:38:49 +01:00