The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness in that this is rather slow. A large part of it is written in
assembly, making it opaque to the compiler for optimisations. The
future proofness requires reading registers that are effectively
`volatile`, making it even harder for the compiler, as well as adding
lots of implicit barriers, making it hard for the microarchitecutre to
optimise as well.
We can make a few assumptions, checked by a few well placed asserts, and
remove a lot of this burden. For a start, at the moment there are 4
group 0 counters with static assignments. Contexting them is a trivial
affair that doesn't need a loop. Similarly, there can only be up to 16
group 1 counters. Contexting them is a bit harder, but we can do with a
single branch with a falling through switch. If/when both of these
change, we have a pair of asserts and the feature detection mechanism to
guard us against pretending that we support something we don't.
We can drop contexting of the offset registers. They are fully
accessible by EL2 and as such are its responsibility to preserve on
powerdown.
Another small thing we can do, is pass the core_pos into the hook.
The caller already knows which core we're running on, we don't need to
call this non-trivial function again.
Finally, knowing this, we don't really need the auxiliary AMUs to be
described by the device tree. Linux doesn't care at the moment, and any
information we need for EL3 can be neatly placed in a simple array.
All of this, combined with lifting the actual saving out of assembly,
reduces the instructions to save the context from 180 to 40, including a
lot fewer branches. The code is also much shorter and easier to read.
Also propagate to aarch32 so that the two don't diverge too much.
Change-Id: Ib62e6e9ba5be7fb9fb8965c8eee148d5598a5361
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Armv8.6 introduced the FEAT_LS64 extension, which provides a 64 *byte*
store instruction. A related instruction is ST64BV0, which will replace
the lowest 32 bits of the data with a value taken from the ACCDATA_EL1
system register (so that EL0 cannot alter them).
Using that ST64BV0 instruction and accessing the ACCDATA_EL1 system
register is guarded by two SCR_EL3 bits, which we should set to avoid a
trap into EL3, when lower ELs use one of those.
Add the required bits and pieces to make this feature usable:
- Add the ENABLE_FEAT_LS64_ACCDATA build option (defaulting to 0).
- Add the CPUID and SCR_EL3 bit definitions associated with FEAT_LS64.
- Add a feature check to check for the existing four variants of the
LS64 feature and detect future extensions.
- Add code to save and restore the ACCDATA_EL1 register on
secure/non-secure context switches.
- Enable the feature with runtime detection for FVP and Arm FPGA.
Please note that the *basic* FEAT_LS64 feature does not feature any trap
bits, it's only the addition of the ACCDATA_EL1 system register that
adds these traps and the SCR_EL3 bits.
Change-Id: Ie3e2ca2d9c4fbbd45c0cc6089accbb825579138a
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
This patch disables trapping to EL3 when the FEAT_D128
specific registers are accessed by setting the SCR_EL3.D128En bit.
If FEAT_D128 is implemented, then FEAT_SYSREG128 is implemented.
With FEAT_SYSREG128 certain system registers are treated as 128-bit,
so we should be context saving and restoring 128-bits instead of 64-bit
when FEAT_D128 is enabled.
FEAT_SYSREG128 adds support for MRRS and MSRR instruction which
helps us to read write to 128-bit system register.
Refer to Arm Architecture Manual for further details.
Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.
Change-Id: I1a53db5eac29e56c8fbdcd4961ede3abfcb2411a
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
Force inlining feat detect helpers such that context save/restore
operations are flattened with sequences of ID regs read and conditional
instructions for system registers read/write. This is opposed to current
situation where with -Os optimization level, feat detect helpers get
called through non-inlined sequences of branch-link+ret.
Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Change-Id: I2633442fb0e69e4a4ed13467e65846fb66d214f6
In this patch, we are trying to introduce the wrapper macro
CREATE_FEATURE_PRESENT to get the following capability and
align it for all the features:
-> is_feat_xx_present(): Does Hardware implement the feature.
-> uniformity in naming the function across multiple features.
-> improved readability
The is_feat_xx_present() is implemented to check if the hardware
implements the feature and does not take into account the
ENABLE_FEAT_XXX flag enabled/disabled in software.
- CREATE_FEATURE_PRESENT(name, idreg, shift, mask, idval)
The wrapper macro reduces the function to a single line and
creates the is_feat_xx_present function that checks the
id register based on the shift and mask values and compares
this against a determined idvalue.
Change-Id: I7b91d2c9c6fbe55f94c693aa1b2c50be54fb9ecc
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Currently, the macros used to denote feature implementation
in hardware follow a random pattern with a few macros having
suffix as SUPPORTED and a few using the suffix IMPLEMENTED.
This patch aligns the macro names uniformly using the suffix
IMPLEMENTED across all the features and removes unused macros
pertaining to the Enable feat mechanism.
FEAT_SUPPORTED --> FEAT_IMPLEMENTED
FEAT_NOT_SUPPORTED --> FEAT_NOT_IMPLEMENTED
Change-Id: I61bb7d154b23f677b80756a4b6a81f74b10cd24f
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Currently both FEAT_MTE and FEAT_MTE_PERM aren't used for enabling
of any feature bits in EL3. So remove both FEAT handling.
All mte regs that are currently context saved/restored are needed
only when FEAT_MTE2 is enabled, so move to usage of FEAT_MTE2 and
remove FEAT_MTE usage.
BREAKING CHANGE: Any platform or downstream code trying to use
SCR_EL3.ATA bit(26) will see failures as this is now moved to be
used only with FEAT_MTE2 with
commit@ef0d0e5478a3f19cbe70a378b9b184036db38fe2
Change-Id: Id01e154156571f7792135639e17dc5c8d0e17cf8
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
Following utility functions/bit definitions done
- Write a helper function to return the presence of following features
- FEAT_UAO
- FEAT_EBEP
- FEAT_SEBEP
- FEAT_SSBS
- FEAT_NMI
- FEAT_PAN
- Add definition of some missing bits of SPSR.
- Add GCSCR_EL1 register encoding and accessor function.
Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: Ifcead0dd8e3b32096e4ab810dde5d582a889785a
Add support for feat mte2. tfsr_el2 is available only with mte2,
however currently its context_save/restore is done with mte rather than
mte2, so introduce 'is_feat_mte2_supported' to check mte2.
Change-Id: I108d9989a8f5b4d1d2f3b9865a914056fa566cf2
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
This feature provides support to context save the
SCXTNUM_ELx register. FEAT_CSV2_3 implies the implementation
of FEAT_CSV2_2. FEAT_CSV2_3 is supported in AArch64 state only
and is an optional feature in Arm v8.0 implementations.
This patch adds feature detection for v8.9 feature FEAT_CSV2_3,
adds macros for ID_AA64PFR0_EL1.CSV2 bits [59:56] for detecting
FEAT_CSV2_3 and macro for ENABLE_FEAT_CSV2_3.
Change-Id: Ida9f31e832b5f11bd89eebd6cc9f10ddad755c14
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>
Currently CTX_INCLUDE_MTE_REGS is used for dual purpose,
to enable allocation tags register and to context save and restore
them and also to check if mte feature is available.
To make it more meaningful, remove CTX_INCLUDE_MTE_REGS
and introduce FEAT_MTE. This would enable allocation tags register
when FEAT_MTE is enabled and also supported from platform.
Also arch features can be conditionally enabled disabled based on
arch version from `make_helpers/arch_features.mk`
Change-Id: Ibdd2d43874634ad7ddff93c7edad6044ae1631ed
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
The FEAT_MTPMU feature disable runs very early after reset. This means,
it needs to be written in assembly, since the C runtime has not been
initialised yet.
However, there is no need for it to be initialised so soon. The PMU
state is only relevant after TF-A has relinquished control. The code
to do this is also very verbose and difficult to read. Delaying the
initialisation allows for it to happen with the rest of the PMU. Align
with FEAT_STATE in the process.
BREAKING CHANGE: This patch explicitly breaks the EL2 entry path. It is
currently unsupported.
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I2aa659d026fbdb75152469f6d19812ece3488c6f
The enablement code for the PMU is scattered and difficult to track
down. Factor out the feature into its own lib/extensions folder and
consolidate the implementation. Treat it is as an architecturally
mandatory feature as it is currently.
Additionally, do some cleanup on AArch64. Setting overflow bits in
PMCR_EL0 is irrelevant for firmware so don't do it. Then delay the PMU
initialisation until the context management stage which simplifies the
early environment assembly. One side effect is that the PMU might count
before this happens so reset all counters to 0 to prevent any leakage.
Finally, add an enable to manage_extensions_realm() as realm world uses
the pmu. This introduces the HPMN fixup to realm world.
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: Ie13a8625820ecc5fbfa467dc6ca18025bf6a9cd3
Many newer architecture features are defined for AArch64 only, so cannot
be used in an AArch32 build.
To avoid #ifdef-ing every single user, just provide trivial
implementations of the feature check functions is_feat_xxx_supported(),
which always return "false" in AArch32. The compiler will then optimise
out the dependent code automatically.
Change-Id: I1e7d653fca0e676a11858efd953c2d623f2d5c9e
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
FEAT_PAN is implemented in AArch32 as well, provide the helper functions
to query the feature availability at runtime.
Change-Id: I375e3eb7b05955ea28a092ba99bb93302af48a0e
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
At the moment we only support FEAT_DIT to be either unconditionally
compiled in, or to be not supported at all.
Add support for runtime detection (ENABLE_DIT=2), by splitting
is_armv8_4_dit_present() into an ID register reading function and a
second function to report the support status. That function considers
both build time settings and runtime information (if needed).
We use ENABLE_DIT in two occassions in assembly code, where we just set
the DIT bit in the DIT system register.
Protect those two cases by reading the CPU ID register when ENABLE_DIT
is set to 2.
Change the FVP platform default to the now supported dynamic
option (=2), so the right decision can be made by the code at runtime.
Change-Id: I506d352f18e23c60db8cdf08edb449f60adbe098
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
The AMU extension code was using its own feature detection routines.
Replace them with the generic CPU feature handlers (defined in
arch_features.h), which get updated to cover the v1p1 variant as well.
Change-Id: I8540f1e745d7b02a25a6c6cdf2a39d6f5e21f2aa
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
At the moment we only support access to the trace unit by system
registers (SYS_REG_TRACE) to be either unconditionally compiled in, or
to be not supported at all.
Add support for runtime detection (ENABLE_SYS_REG_TRACE_FOR_NS=2), by
adding is_feat_sys_reg_trace_supported(). That function considers both
build time settings and runtime information (if needed), and is used
before we access SYS_REG_TRACE related registers.
The FVP platform decided to compile in support unconditionally (=1),
even though this is an optional feature, so it is not available with the
FVP model's default command line.
Change that to the now supported dynamic option (=2), so the right
decision can be made by the code at runtime.
Change-Id: I450a574a4f6bd9fc269887037049c94c906f54b2
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
At the moment we only support FEAT_SPE to be either unconditionally
compiled in, or to be not supported at all.
Add support for runtime detection (ENABLE_SPE_FOR_NS=2), by splitting
is_armv8_2_feat_spe_present() into an ID register reading function and
a second function to report the support status. That function considers
both build time settings and runtime information (if needed), and is
used before we access SPE related registers.
Previously SPE was enabled unconditionally for all platforms, change
this now to the runtime detection version.
Change-Id: I830c094107ce6a398bf1f4aef7ffcb79d4f36552
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
At the moment we only support FEAT_TRF to be either unconditionally
compiled in, or to be not supported at all.
Add support for runtime detection (ENABLE_TRF_FOR_NS=2), by splitting
is_feat_trf_present() into an ID register reading function and a second
function to report the support status. That function considers both
build time settings and runtime information (if needed), and is used
before we access TRF related registers.
Also move the context saving code from assembly to C, and use the new
is_feat_trf_supported() function to guard its execution.
The FVP platform decided to compile in support unconditionally (=1),
even though FEAT_TRF is an ARMv8.4 feature, so is not available with the
FVP model's default command line.
Change that to the now supported dynamic option (=2), so the right
decision can be made by the code at runtime.
Change-Id: Ia97b01adbe24970a4d837afd463dc5506b7295a3
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Some MISRA test complains about our code to isolate CPU ID register
fields: the ID registers (and associated masks) are 64 bits wide, but
the eventual field is always 4 bits wide only, so we use an unsigned
int to represent that. MISRA dislikes the differing width here.
Since the code to extract a feature field from a CPU ID register is very
schematic already, provide a wrapper macro to make this more readable,
and do the proper casting in one central place on the way.
While at it, use the same macro for the AArch32 feature detection side.
Change-Id: Ie102a9e7007a386f5879ec65e159ff041504a4ee
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
The Generic Timer is an optional extension to an ARMv7-A implementation.
The generic delay timer can be used from any architecture supported by
the Trusted Firmware. In ARMv7 it is needed to check that this feature
is present. In ARMv8 it is always present.
Change-Id: Ib7e8ec13ffbb2f64445d4ee48ed00f26e34b79b7
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
ARMv8.2-TTCNP is mandatory from ARMv8.2 onwards, but it can be implemented
in CPUs that don't implement all mandatory 8.2 features (and so have to
claim to be a lower version).
This patch removes usage of the ARM_ARCH_AT_LEAST() macro and uses system
ID registers to detect whether it is needed to set the bit or not.
Change-Id: I7bcbf0c7c937590dfc2ca668cfd9267c50f7d52c
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
