Commit graph

74 commits

Author SHA1 Message Date
Vinoj Soundararajan
ec6f49c26b feat(ras): add eabort get helper function
Add EABORT get field helper function to obtain SET, AET (UET) values
from esr_el3/disr_el1 based on PE error state recording in the exception
syndrome refer to RAS PE architecture in
https://developer.arm.com/documentation/ddi0487/latest/

Change-Id: I0011f041a3089c9bbf670275687ad7c3362a07f9
Signed-off-by: Vinoj Soundararajan <vinojs@google.com>
2025-03-06 13:45:08 +00:00
Vinoj Soundararajan
daeae49511 feat(ras): add asynchronous error type corrected
Add asynchronous error type Corrected (CE) to error status
AET based on PE error state recording in the exception syndrome
Refer to https://developer.arm.com/documentation/ddi0487/latest/
RAS PE architecture.

Change-Id: I9f2525411b94c8fd397b4a0b8cf5dc47457a2771
Signed-off-by: Vinoj Soundararajan <vinojs@google.com>
2025-03-06 13:34:23 +00:00
Vinoj Soundararajan
e5cd3e81d1 fix(ras): fix typo in uncorrectable error type UEO
Fix spelling for UEO from restable to restartable
based on PE error state recording in the exception syndrome
Refer to https://developer.arm.com/documentation/ddi0487/latest/
RAS PE architecture.

Change-Id: I4da419f2120a7385853d4da78b409c675cdfe1c8
Signed-off-by: Vinoj Soundararajan <vinojs@google.com>
2025-03-06 13:30:19 +00:00
Vinoj Soundararajan
9c17687aab fix(ras): fix status synchronous error type fields
Based on SET bits of ISS encoding for an exception from Data or
Instruction Abort. (Refer to ESR_EL3)
1. Fix Synchronous error type restartable value from 1 to 3
2. Remove corrected CE field which is not applicable to SET

Change-Id: If357da9881bee962825bc3b9423ba7fc107f9b1d
Signed-off-by: Vinoj Soundararajan <vinojs@google.com>
2025-03-06 13:14:02 +00:00
Boyan Karatotev
0a580b5128 perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context
SVE and SME aren't enabled symmetrically for all worlds, but EL3 needs
to context switch them nonetheless. Previously, this had to happen by
writing the enable bits just before reading/writing the relevant
context. But since the introduction of root context, this need not be
the case. We can have these enables always be present for EL3 and save
on some work (and ISBs!) on every context switch.

We can also hoist ZCR_EL3 to a never changing register, as we set its
value to be identical for every world, which happens to be the one we
want for EL3 too.

Change-Id: I3d950e72049a298008205ba32f230d5a5c02f8b0
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-25 08:52:06 +00:00
Boyan Karatotev
83ec7e452c perf(amu): greatly simplify AMU context management
The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness in that this is rather slow. A large part of it is written in
assembly, making it opaque to the compiler for optimisations. The
future proofness requires reading registers that are effectively
`volatile`, making it even harder for the compiler, as well as adding
lots of implicit barriers, making it hard for the microarchitecutre to
optimise as well.

We can make a few assumptions, checked by a few well placed asserts, and
remove a lot of this burden. For a start, at the moment there are 4
group 0 counters with static assignments. Contexting them is a trivial
affair that doesn't need a loop. Similarly, there can only be up to 16
group 1 counters. Contexting them is a bit harder, but we can do with a
single branch with a falling through switch. If/when both of these
change, we have a pair of asserts and the feature detection mechanism to
guard us against pretending that we support something we don't.

We can drop contexting of the offset registers. They are fully
accessible by EL2 and as such are its responsibility to preserve on
powerdown.

Another small thing we can do, is pass the core_pos into the hook.
The caller already knows which core we're running on, we don't need to
call this non-trivial function again.

Finally, knowing this, we don't really need the auxiliary AMUs to be
described by the device tree. Linux doesn't care at the moment, and any
information we need for EL3 can be neatly placed in a simple array.

All of this, combined with lifting the actual saving out of assembly,
reduces the instructions to save the context from 180 to 40, including a
lot fewer branches. The code is also much shorter and easier to read.

Also propagate to aarch32 so that the two don't diverge too much.

Change-Id: Ib62e6e9ba5be7fb9fb8965c8eee148d5598a5361
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2025-02-25 08:50:46 +00:00
Boyan Karatotev
f808873372 fix(spe): add a psb before updating context and remove context saving
In the chapter about FEAT_SPE (D16.4 specifically) it is stated that
"Sampling is always disabled at EL3". That means that disabling sampling
(writing PMBLIMITR_EL1.E to 0) is redundant and can be removed. The only
reason we save/restore SPE context is because of that disable, so those
can be removed too.

There's the issue of draining the profiling buffer though. No new
samples will have been generated since entering EL3. However, old
samples might still be in-flight. Unless synchronised by a psb csync,
those might be affected by our extensive context mutation. Adding a psb
in prepare_el3_entry should cater for that. Note that prior to the
introduction of root context this was not a problem as context remained
unchanged and the hooks took care of the rest.

Then, the only time we care about the buffer actually making it to
memory is when we exit coherency. On HW_ASSISTED_COHERENCY systems we
don't have to do anything, it should be handled for us. Systems without
it need a dsb to wait for them to complete. There should be one already
in each cpu's powerdown hook which should work.

While on the topic of barriers, the esb barrier is no longer used.
Remove it.

Change-Id: I9736fc7d109702c63e7d403dc9e2a4272828afb2
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
2024-12-16 15:14:30 +00:00
Arvind Ram Prakash
a57e18e433 feat(fpmr): disable FPMR trap
This patch enables support of FEAT_FPMR by enabling access
to FPMR register. It achieves it by setting the EnFPM bit of
SCR_EL3. This feature is currently enabled for NS world only.

Reference:
https://developer.arm.com/documentation/109697/2024_09/
Feature-descriptions/The-Armv9-5-architecture-extension?lang=en

Change-Id: I580c409b9b22f8ead0737502280fb9093a3d5dd2
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
2024-12-12 10:03:23 -06:00
Govindraj Raja
306551362c feat(d128): add support for FEAT_D128
This patch disables trapping to EL3 when the FEAT_D128
specific registers are accessed by setting the SCR_EL3.D128En bit.

If FEAT_D128 is implemented, then FEAT_SYSREG128 is implemented.
With FEAT_SYSREG128 certain system registers are treated as 128-bit,
so we should be context saving and restoring 128-bits instead of 64-bit
when FEAT_D128 is enabled.

FEAT_SYSREG128 adds support for MRRS and MSRR instruction which
helps us to read write to 128-bit system register.
Refer to Arm Architecture Manual for further details.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I1a53db5eac29e56c8fbdcd4961ede3abfcb2411a
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
2024-10-24 14:51:55 -05:00
Jayanth Dodderi Chidanand
f4303d05ea feat(cm): handle asymmetry for FEAT_TCR2
With introduction of FEAT_STATE_CHECK_ASYMMETRIC, the asymmetry of cores
can be handled. FEAT_TCR2 is one of the features which can be
asymmetric across cores and the respective support is added here.

Adding a function to handle this asymmetry by re-visting the
feature presence on running core.
There are two possible cases:
 - If the primary core has the feature and secondary does not have it
   then the feature is disabled.
 - If the primary does not have the feature and secondary has it then,
   the feature need to be enabled in secondary cores.

Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Change-Id: I73a70891d52268ddfa4effe40edf04115f5821ca
2024-09-05 16:28:23 +01:00
Madhukar Pappireddy
6d5319afec feat(simd): add routines to save, restore sve state
This adds assembly routines to save and restore SVE registers. In order
to share between FPU and SVE the code to save and restore FPCR and
FPSR, the patch converts code for those registers into macro.
Since we will be using simd_ctx_t to save and restore FPU also, we use
offsets in simd_ctx_t for FPSR and FPCR. Since simd_ctx_t has the same
structure at the beginning as fp_regs_t, those offsets should be the
same as CTX_FP_* offsets, when SVE is not enabled. Note that the code
also saves and restores FPEXC32 reg along with FPSR and FPCR.

Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
Signed-off-by: Okash Khawaja <okash@google.com>
Change-Id: I120c02359794aa6bb6376a464a9afe98bd84ae60
2024-08-19 11:10:10 -05:00
Arvind Ram Prakash
b36e975ea3 feat(trbe): introduce trbe_disable() function
This patch adds trbe_disable() which disables Trace buffer access
from lower ELs in all security state. This function makes Secure
state the owner of Trace buffer and access from EL2/EL1 generate
trap exceptions to EL3.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: If3e3bd621684b3c28f44c3ed2fe3df30b143f8cd
2024-07-29 20:35:14 +01:00
Manish Pandey
651fe5073c feat(spe): introduce spe_disable() function
Introduce a function to disable SPE feature for Non-secure state and do
the default setting of making Secure state the owner of profiling
buffers and trap access of profiling and profiling buffer control
registers from lower ELs to EL3.

This functionality is required to handle asymmetric cores where SPE has
to disabled at runtime.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I2f99e922e8df06bfc900c153137aef7c9dcfd759
2024-07-29 20:34:18 +01:00
Manish Pandey
4de07b4be7 chore(spe): rename spe_disable() to spe_stop()
During CPU power down, we stop the profiling by calling spe_disable()
function. From TF-A point of view, enable/disable means the avaibility
of the feature for lower EL. In this case we are not actully disabling
the feautre but stoping it before power down.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I6e3b39c5c35d330c51e7ac715446a8b36bf9531f
2024-07-29 20:34:04 +01:00
Arvind Ram Prakash
33e6aaacf1 feat(fgt2): add support for FEAT_FGT2
This patch disables trapping to EL3 when the FEAT_FGT2
specific trap registers are accessed by setting the
SCR_EL3.FGTEn2 bit

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I6d2b614affb9067b2bc3d7bf0ae7d169d031592a
2024-07-18 13:49:43 -05:00
Arvind Ram Prakash
83271d5a5a feat(debugv8p9): add support for FEAT_Debugv8p9
This patch enables FEAT_Debugv8p9 and prevents EL1/0 from
trapping to EL3 when accessing MDSELR_EL1 register by
setting the MDCR_EL3.EBWE bit.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I3613af1dd8cb8c0d3c33dc959f170846c0b9695a
2024-07-18 13:49:43 -05:00
Jayanth Dodderi Chidanand
123002f917 feat(cm): context switch MDCR_EL3 register
Currently MDCR_EL3 register value is same for all the
worlds(Non-secure, Secure, Realm and Root).

With this approach, features enable/disable settings
remain same across all the worlds. This is not ideal as
there must be flexibility in controlling feature as per
the requirements for individual world.

The patch addresses this by providing MDCR_EL3 a per world
value. Features with identical values for all the worlds are
grouped under ``manage_extensions_common`` API.

Change-Id: Ibc068d985fe165d8cb6d0ffb84119bffd743b3d1
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2024-06-25 13:50:32 +01:00
Arvind Ram Prakash
ac4f6aaf85 refactor(cm): move MPAM3_EL3 reg to per world context
Refactor MPAM3_EL3 to be world-specific, eliminating redundant cross-CPU
value duplication and reducing memory footprint.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: Iddf020a5462737e01ac35e4f2b2b204a8759fafb
2023-12-21 12:37:21 +00:00
Elizabeth Ho
461c0a5d92 refactor(cm): move EL3 registers to global context
Currently, EL3 context registers are duplicated per-world per-cpu.
Some registers have the same value across all CPUs, so this patch
moves these registers out into a per-world context to reduce
memory usage.

Change-Id: I91294e3d5f4af21a58c23599af2bdbd2a747c54a
Signed-off-by: Elizabeth Ho <elizabeth.ho@arm.com>
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2023-10-31 11:18:42 +00:00
Arvind Ram Prakash
edebefbcbc fix(mpam): refine MPAM initialization and enablement process
Restricts MPAM to only NS world and enables trap to EL3 for access of
MPAM registers from lower ELs of Secure and Realm world.

This patch removes MPAM enablement from global context and adds it to
EL3 State context which enables/disables MPAM during world switches.
Renamed ENABLE_MPAM_FOR_LOWER_ELS to ENABLE_FEAT_MPAM and
removed mpam_init_el3() as RESET behaviour is trapping.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I131f9dba5df236a71959b2d425ee11af7f3c38c4
2023-10-25 09:04:29 -05:00
Govindraj Raja
4c700c1563 chore: update to use Arm word across TF-A
Align entire TF-A to use Arm in copyright header.

Change-Id: Ief9992169efdab61d0da6bd8c5180de7a4bc2244
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
2023-08-08 15:12:30 +01:00
Boyan Karatotev
ece8f7d734 refactor(cm): set MDCR_EL3/CPTR_EL3 bits in respective feat_init_el3() only
These bits (MDCR_EL3.{NSTB, NSTBE, TTRF, TPM}, CPTR_EL3.TTA) only affect
EL2 (and lower) execution. Each feat_init_el3() is called long before
any lower EL has had a chance to execute, so setting the bits at reset
is redundant. Removing them from reset code also improves readability of
the immutable EL3 state.

Preserve the original intention for the TTA bit of "enabled for NS and
disabled everywhere else" (inferred from commit messages d4582d3088 and
2031d6166a and the comment). This is because CPTR_EL3 will be contexted
and so everyone will eventually get whatever NS has anyway.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I3d24b45d3ea80882c8e450b2d9db9d5531facec1
2023-07-24 11:04:44 +01:00
Boyan Karatotev
4085a02c76 refactor(amu): separate the EL2 and EL3 enablement code
Combining the EL2 and EL3 enablement code necessitates that it must be
called at el3_exit, which is the only place with enough context to make
the decision of what needs to be set.
Decouple them to allow them to be called from elsewhere. Also take
some time to clarify and simplify AMU code.

The sanity check in the context_restore() is now wrong, as the cpu may
turn off on suspend, thus resetting the value of the counter enables.
Remove it.

Finally, this completes the migration to cm_manage_extensions_el3() and
manage_extensions_nonsecure() so manage_extensions_nonsecure_mixed() is
being removed.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I66399132364c32be66017506bb54cbadd8485577
2023-07-04 14:57:46 +01:00
Boyan Karatotev
60d330dc4d refactor(cpufeat): separate the EL2 and EL3 enablement code
Combining the EL2 and EL3 enablement code necessitates that it must be
called at el3_exit, which is the only place with enough context to make
the decision of what needs to be set.
Decouple them to allow them to be called from elsewhere.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I147764c42771e7d4100699ec8fae98dac0a505c0
2023-07-04 14:57:46 +01:00
Boyan Karatotev
c73686a11c feat(pmu): introduce pmuv3 lib/extensions folder
The enablement code for the PMU is scattered and difficult to track
down. Factor out the feature into its own lib/extensions folder and
consolidate the implementation. Treat it is as an architecturally
mandatory feature as it is currently.

Additionally, do some cleanup on AArch64. Setting overflow bits in
PMCR_EL0 is irrelevant for firmware so don't do it. Then delay the PMU
initialisation until the context management stage which simplifies the
early environment assembly. One side effect is that the PMU might count
before this happens so reset all counters to 0 to prevent any leakage.

Finally, add an enable to manage_extensions_realm() as realm world uses
the pmu. This introduces the HPMN fixup to realm world.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: Ie13a8625820ecc5fbfa467dc6ca18025bf6a9cd3
2023-06-29 09:59:06 +01:00
Jayanth Dodderi Chidanand
2b0bc4e028 feat(cpufeat): enable FEAT_SVE for FEAT_STATE_CHECKED
Add support for runtime detection (ENABLE_SVE_FOR_NS=2), by splitting
sve_supported() into an ID register reading function and a
second function to report the support status. That function considers
both build time settings and runtime information (if needed), and is
used before we do SVE specific setup.

Change the FVP platform default to the now supported dynamic
option (=2), so the right decision can be made by the code at runtime.

Change-Id: I1caaba2216e8e2a651452254944a003607503216
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2023-03-28 17:19:13 +02:00
Jayanth Dodderi Chidanand
45007acd46 feat(cpufeat): enable FEAT_SME for FEAT_STATE_CHECKED
Add support for runtime detection (ENABLE_SME_FOR_NS=2), by splitting
feat_sme_supported() into an ID register reading function and a
second function to report the support status. That function considers
both build time settings and runtime information (if needed), and is
used before we do SME specific setup.

Change the FVP platform default to the now supported dynamic option
(=2),so the right decision can be made by the code at runtime.

Change-Id: Ida9ccf737db5be20865b84f42b1f9587be0626ab
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2023-03-28 17:19:07 +02:00
Andre Przywara
b57e16a4f9 refactor(amu): use new AMU feature check routines
The AMU extension code was using its own feature detection routines.
Replace them with the generic CPU feature handlers (defined in
arch_features.h), which get updated to cover the v1p1 variant as well.

Change-Id: I8540f1e745d7b02a25a6c6cdf2a39d6f5e21f2aa
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2023-03-27 19:36:15 +01:00
Andre Przywara
603a0c6fae refactor(cpufeat): enable SYS_REG_TRACE for FEAT_STATE_CHECKED
At the moment we only support access to the trace unit by system
registers (SYS_REG_TRACE) to be either unconditionally compiled in, or
to be not supported at all.

Add support for runtime detection (ENABLE_SYS_REG_TRACE_FOR_NS=2), by
adding is_feat_sys_reg_trace_supported(). That function considers both
build time settings and runtime information (if needed), and is used
before we access SYS_REG_TRACE related registers.

The FVP platform decided to compile in support unconditionally (=1),
even though this is an optional feature, so it is not available with the
FVP model's default command line.
Change that to the now supported dynamic option (=2), so the right
decision can be made by the code at runtime.

Change-Id: I450a574a4f6bd9fc269887037049c94c906f54b2
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2023-03-22 13:33:22 +00:00
Andre Przywara
d7f3ed3655 fix(cpufeat): make stub enable functions "static inline"
For builds where we don't have a certain feature enabled, we provide
empty stub xxx_enable() functions in a header file. This way we
avoid #ifdef's in the code, and can call the enable function
unconditionally. When compiling with -O1 or higher, the compiler will
even optimise out the whole call, so the symbol will never make it into
any object file.
When compiling with optimisations turned off, the function stub will
survive, and could make it into multiple object files, which would lead
to a multiple definitons error.

Avoid this by defining those stub functions as "static inline". The
"static" will avoid the multiple definitions problems, the "inline" will
avoid a potential compiler warning about unused functions.
This patterns is used extensively in the Linux kernel.

Change-Id: Iad07bb946aab372587c83f2423b4983bf3817990
Reported-by: Chris Kay <chris.kay@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2023-03-22 13:33:22 +00:00
Andre Przywara
9448f2b88e refactor(mpam): enable FEAT_MPAM for FEAT_STATE_CHECKED
At the moment we only support FEAT_MPAM to be either unconditionally
compiled in, or to be not supported at all.

Add support for runtime detection (ENABLE_MPAM_FOR_LOWER_ELS=2), by
splitting get_mpam_version() into an ID register reading
function and a second function to report the support status. That
function considers both build time settings and runtime information (if
needed), and is used before we access MPAM related registers.
Also move the context saving code from assembly to C, and use the new
is_feat_mpam_supported() function to guard its execution.

ENABLE_MPAM_FOR_LOWER_ELS defaults to 0, so add a stub enable function
to cover builds with compiler optimisations turned off. The unused
mpam_enable() function call will normally be optimised away (because it
would never be called), but with -O0 the compiler will leave the symbol
in the object file.

Change-Id: I531d87cb855a7c43471f861f625b5a6d4bc61313
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2023-03-20 13:37:36 +00:00
Andre Przywara
6437a09a2d refactor(spe): enable FEAT_SPE for FEAT_STATE_CHECKED
At the moment we only support FEAT_SPE to be either unconditionally
compiled in, or to be not supported at all.

Add support for runtime detection (ENABLE_SPE_FOR_NS=2), by splitting
is_armv8_2_feat_spe_present() into an ID register reading function and
a second function to report the support status. That function considers
both build time settings and runtime information (if needed), and is
used before we access SPE related registers.

Previously SPE was enabled unconditionally for all platforms, change
this now to the runtime detection version.

Change-Id: I830c094107ce6a398bf1f4aef7ffcb79d4f36552
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
2023-03-20 13:37:36 +00:00
Jayanth Dodderi Chidanand
e8f0dd58da fix(cpufeat): resolve build errors due to compiler optimization
Currently most of the architectural feature build flags are set
to 2(FEATURE_STATE_CHECK) for fvp platform only.

However other platforms still configure them by default to 0, which
would lead to build failures in cases when compiler configured
to build TF-A  with zero optimization (CFLAGS='-O0').

This patch addresses such build issues and thereby resolves the failures
seen under CI-l3 test_configurations.

Change-Id: I45b82b821951bba6b9df08177f7d286e624a4239
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
2023-03-01 17:54:07 +01:00
johpow01
744ad97445 feat(brbe): add BRBE support for NS world
This patch enables access to the branch record buffer control registers
in non-secure EL2 and EL1 using the new build option ENABLE_BRBE_FOR_NS.
It is disabled for all secure world, and cannot be used with ENABLE_RME.

This option is disabled by default, however, the FVP platform makefile
enables it for FVP builds.

Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I576a49d446a8a73286ea6417c16bd0b8de71fca0
2022-05-05 19:43:10 +02:00
Jayanth Dodderi Chidanand
781d07a421 refactor(twed): improve TWED enablement in EL-3
The current implementation uses plat_arm API under generic code.
"plat_arm" API is a convention used with Arm common platform layer
and is reserved for that purpose. In addition, the function has a
weak definition which is not encouraged in TF-A.

Henceforth, removing the weak API with a configurable macro "TWED_DELAY"
of numeric data type in generic code and simplifying the implementation.
By default "TWED_DELAY" is defined to zero, and the delay value need to
be explicitly set by the platforms during buildtime.

Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Change-Id: I25cd6f628e863dc40415ced3a82d0662fdf2d75a
2022-04-17 23:48:10 +01:00
johpow01
dc78e62d80 feat(sme): enable SME functionality
This patch adds two new compile time options to enable SME in TF-A:
ENABLE_SME_FOR_NS and ENABLE_SME_FOR_SWD for use in non-secure and
secure worlds respectively. Setting ENABLE_SME_FOR_NS=1 will enable
SME for non-secure worlds and trap SME, SVE, and FPU/SIMD instructions
in secure context. Setting ENABLE_SME_FOR_SWD=1 will disable these
traps, but support for SME context management does not yet exist in
SPM so building with SPD=spmd will fail.

The existing ENABLE_SVE_FOR_NS and ENABLE_SVE_FOR_SWD options cannot
be used with SME as it is a superset of SVE and will enable SVE and
FPU/SIMD along with SME.

Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Iaaac9d22fe37b4a92315207891da848a8fd0ed73
2021-11-12 10:38:00 -06:00
Chris Kay
742ca2307f feat(amu): enable per-core AMU auxiliary counters
This change makes AMU auxiliary counters configurable on a per-core
basis, controlled by `ENABLE_AMU_AUXILIARY_COUNTERS`.

Auxiliary counters can be described via the `HW_CONFIG` device tree if
the `ENABLE_AMU_FCONF` build option is enabled, or the platform must
otherwise implement the `plat_amu_topology` function.

A new phandle property for `cpu` nodes (`amu`) has been introduced to
the `HW_CONFIG` specification to allow CPUs to describe the view of
their own AMU:

```
cpu0: cpu@0 {
    ...

    amu = <&cpu0_amu>;
};
```

Multiple cores may share an `amu` handle if they implement the
same set of auxiliary counters.

AMU counters are described for one or more AMUs through the use of a new
`amus` node:

```
amus {
    cpu0_amu: amu-0 {
        #address-cells = <1>;
        #size-cells = <0>;

        counter@0 {
            reg = <0>;

            enable-at-el3;
        };

        counter@n {
            reg = <n>;

            ...
        };
    };
};
```

This structure describes the **auxiliary** (group 1) AMU counters.
Architected counters have architecturally-defined behaviour, and as
such do not require DTB entries.

These `counter` nodes support two properties:

- The `reg` property represents the counter register index.
- The presence of the `enable-at-el3` property determines whether
  the firmware should enable the counter prior to exiting EL3.

Change-Id: Ie43aee010518c5725a3b338a4899b0857caf4c28
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:15:33 +01:00
Chris Kay
e747a59be4 refactor(amu): refactor enablement and context switching
This change represents a general refactoring to clean up old code that
has been adapted to account for changes required to enable dynamic
auxiliary counters.

Change-Id: Ia85e0518f3f65c765f07b34b67744fc869b9070d
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:14:33 +01:00
Chris Kay
31d3cc2570 refactor(amu): detect auxiliary counters at runtime
This change decouples the group 1 counter macros to facilitate dynamic
detection at runtime. These counters remain disabled - we will add
dynamic enablement of them in a later patch.

Change-Id: I820d05f228d440643bdfa308d030bd51ebc0b35a
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:14:33 +01:00
Chris Kay
81e2ff1f36 refactor(amu): detect architected counters at runtime
This change removes the `AMU_GROUP0_COUNTERS_MASK` and
`AMU_GROUP0_MAX_COUNTERS` preprocessor definitions, instead retrieving
the number of group 0 counters dynamically through `AMCGCR_EL0.CG0NC`.

Change-Id: I70e39c30fbd5df89b214276fac79cc8758a89f72
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:14:32 +01:00
Chris Kay
1fd685a74d refactor(amu): conditionally compile auxiliary counter support
This change reduces preprocessor dependencies on the
`AMU_GROUP1_NR_COUNTERS` and `AMU_GROUP1_COUNTERS_MASK` definitions, as
these values will eventually be discovered dynamically.

In their stead, we introduce the `ENABLE_AMU_AUXILIARY_COUNTERS` build
option, which will enable support for dynamically detecting and
enabling auxiliary AMU counters.

This substantially reduces the amount of memory used by platforms that
know ahead of time that they do not have any auxiliary AMU counters.

Change-Id: I3d998aff44ed5489af4857e337e97634d06e3ea1
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:14:32 +01:00
Chris Kay
b4b726ea86 refactor(amu)!: privatize unused AMU APIs
This change reduces the exposed surface area of the AMU API in order to
simplify the refactoring work in following patches. The functions and
definitions privatized by this change are not used by other parts of the
code-base today.

BREAKING CHANGE: The public AMU API has been reduced to enablement only
to facilitate refactoring work. These APIs were not previously used.

Change-Id: Ibf6174fb5b3949de3c4ba6847cce47d82a6bd08c
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:14:31 +01:00
Chris Kay
6c8dda19e5 refactor(amu)!: remove PLAT_AMU_GROUP1_COUNTERS_MASK
With the introduction of MPMM, the auxiliary AMU counter logic requires
refactoring to move away from a single platform-defined group 1 counter
mask in order to support microarchitectural (per-core) group 1 counters.

BREAKING CHANGE: The `PLAT_AMU_GROUP1_COUNTERS_MASK` platform definition
has been removed. Platforms should specify per-core AMU counter masks
via FCONF or a platform-specific mechanism going forward.

Change-Id: I1e852797c7954f92409222b066a1ae57bc72bb05
Signed-off-by: Chris Kay <chris.kay@arm.com>
2021-10-26 12:14:30 +01:00
Manish V Badarkhe
8fcd3d9600 feat(trf): enable trace filter control register access from lower NS EL
Introduced a build flag 'ENABLE_TRF_FOR_NS' to enable trace filter
control registers access in NS-EL2, or NS-EL1 (when NS-EL2 is
implemented but unused).

Change-Id: If3f53b8173a5573424b9a405a4bd8c206ffdeb8c
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>
2021-08-26 09:32:35 +01:00
Manish V Badarkhe
d4582d3088 feat(sys_reg_trace): enable trace system registers access from lower NS ELs
Introduced a build flag 'ENABLE_SYS_REG_TRACE_FOR_NS' to enable trace
system registers access in NS-EL2, or NS-EL1 (when NS-EL2 is
implemented but unused).

Change-Id: Idc1acede4186e101758cbf7bed5af7b634d7d18d
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>
2021-08-26 09:29:51 +01:00
Manish V Badarkhe
813524ea9d feat(trbe): enable access to trace buffer control registers from lower NS EL
Introduced a build flag 'ENABLE_TRBE_FOR_NS' to enable trace buffer
control registers access in NS-EL2, or NS-EL1 (when NS-EL2 is
implemented but unused).

Change-Id: I285a672ccd395eebd377714c992bb21062a729cc
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>
2021-08-26 09:29:39 +01:00
Arunachalam Ganapathy
68ac5ed049 fix(el3_runtime): fix SVE and AMU extension enablement flags
If SVE are enabled for both Non-secure and Secure world along with AMU
extension, then it causes the TAM_BIT in CPTR_EL3 to be set upon exit
from bl31. This restricts access to the AMU register set in normal
world. This fix maintains consistency in both TAM_BIT and CPTR_EZ_BIT
by saving and restoring CPTR_EL3 register from EL3 context.

Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>
Change-Id: Id76ce1d27ee48bed65eb32392036377716aff087
2021-07-23 10:33:59 +01:00
Manish V Badarkhe
52698a620b refactor(mpam): remove unused function declaration
Change-Id: Ia660b6554fe4544effd1810e1aca202f95e3c447
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>
2021-07-09 22:03:38 +02:00
Max Shvetsov
0c5e7d1ce3 feat(sve): enable SVE for the secure world
Enables SVE support for the secure world via ENABLE_SVE_FOR_SWD.
ENABLE_SVE_FOR_SWD defaults to 0 and has to be explicitly set by the
platform. SVE is configured during initial setup and then uses EL3
context save/restore routine to switch between SVE configurations for
different contexts.
Reset value of CPTR_EL3 changed to be most restrictive by default.

Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>
Change-Id: I889fbbc2e435435d66779b73a2d90d1188bf4116
2021-06-28 13:24:24 +01:00
johpow01
873d4241e3 Enable v8.6 AMU enhancements (FEAT_AMUv1p1)
ARMv8.6 adds virtual offset registers to support virtualization of the
event counters in EL1 and EL0.  This patch enables support for this
feature in EL3 firmware.

Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I7ee1f3d9f554930bf5ef6f3d492e932e6d95b217
2021-02-25 22:01:59 +00:00