diff --git a/docs/components/context-management-library.rst b/docs/components/context-management-library.rst new file mode 100644 index 000000000..56ba2ec1f --- /dev/null +++ b/docs/components/context-management-library.rst @@ -0,0 +1,501 @@ +Context Management Library +************************** + +This document provides an overview of the Context Management library implementation +in Trusted Firmware-A (TF-A). It enumerates and describes the APIs implemented +and their accessibility from other components at EL3. + +Overview +======== + +Arm TrustZone architecture facilitates hardware-enforced isolation between +software running in various security states (Secure/Non-Secure/Realm). +The general-purpose registers, most of the system registers and vector registers +are not banked per world. When moving between the security states it is the +responsibility of the secure monitor software (BL31(AArch64) / BL32(Aarch32)) +in TF-A, not the hardware, to save and restore register state. +Refer to `Trustzone for AArch64`_ for more details. + +EL3 Runtime Firmware, also termed as secure monitor firmware, is integrated +with a context management library to handle the context of the CPU, managing the +saving and restoring of register states across the worlds. + +TF-A Context +============ + +In TF-A, the context is represented as a data structure used by the EL3 firmware +to preserve the state of the CPU at the next lower exception level (EL) in a given +security state and save enough EL3 metadata to be able to return to that exception +level and security state. The memory for the context data structures are allocated +in BSS section of EL3 firmware. + +In a trusted system at any instance, a given CPU could be executing in one of the +security states (Non-Secure, Secure, Realm). Each world must have its +configuration of system registers independent of other security states to access +and execute any of the architectural features. + +If the CPU switches across security states (for example: from Non-secure to Secure +or vice versa), the register contents, especially the ones that are not banked +(EL2/EL1, vector, general-purpose registers), will be overwritten, as the software +running in either state has the privileges to access them. Additionally, some of +the architectural features enabled in the former security state will be unconditionally +accessible in the latter security state as well. This can be a major concern when +dealing with security-specific bits, as they need to be explicitly enabled or +disabled in each state to prevent data leakage across the worlds. + +In general, an ideal trusted system should have Secure world-specific configurations +that are not influenced by Normal World operations. Therefore, for each CPU, we +need to maintain world-specific context to ensure that register entries from one +world do not leak or impact the execution of the CPU in other worlds. +This will help ensure the integrity and security of the system, preventing any +unauthorized access or data corruption between the different security states. + +Design +====== + +The Context Management library in TF-A is designed to cover all the requirements +for maintaining world-specific context essential for a trusted system. +This includes implementing CPU context initialization and management routines, +as well as other helper APIs that are required by dispatcher components in EL3 +firmware, which are collectively referred to as CPU Context Management. +The APIs and their usecases are listed in detail under the :ref:`Library APIs` +section. + +Originally, the Context Management library in TF-A was designed to cater for a +two-world system, comprising of Non-Secure and Secure Worlds. In this case, the +EL3 Firmware is assumed to be running in Secure World. +With introduction of Realm Management Extension (RME), from Armv9.2 a system +can have four distinct worlds (Non-Secure, Secure, Realm, Root). +RME isolates EL3 from all other Security states and moves it into its own security +state called root. EL3 firmware now runs at Root World and thereby is +trusted from software in Non-secure, Secure, and Realm states. +Refer to `Security States with RME`_ for more details. + +Key principles followed in designing the context management library : + +1. **EL3 should only initialize immediate used lower EL** + +Context Management library running at EL3 should only initialize and monitor the +immediate used lower EL. This implies that, when S-EL2 is present in the system, +EL3 should initialise and monitor S-EL2 registers only. S-EL1 registers should +not be the concern of EL3 while S-EL2 is in place. In systems where S-EL2 is +absent, S-EL1 registers should be initialised from EL3. + +2. **Decentralized model for context management** + +Each world (Non-Secure, Secure, and Realm) should have their separate component +in EL3 responsible for their respective world context management. +Both the Secure and Realm world have associated dispatcher components in EL3 +firmware to allow management of the respective worlds. For the Non-Secure world, +PSCI Library (BL31)/context management library provides routines to help +initialize the Non-Secure world context. + +3. **Flexibility for Dispatchers to select desired feature set to save and restore** + +Each feature is supported with a helper function ``is_feature_supported(void)``, +to detect its presence at runtime. This helps dispatchers to select the desired +feature set, and thereby save and restore the configuration associated with them. + +4. **Dynamic discovery of Feature enablement by EL3** + +TF-A supports three states for feature enablement at EL3, to make them available +for lower exception levels. + +.. code:: c + + #define FEAT_STATE_DISABLED 0 + #define FEAT_STATE_ENABLED 1 + #define FEAT_STATE_CHECK 2 + +A pattern is established for feature enablement behavior. +Each feature must support the 3 possible values with rigid semantics. + +- **FEAT_STATE_DISABLED** - all code relating to this feature is always skipped. + Firmware is unaware of this feature. + +- **FEAT_STATE_ALWAYS** - all code relating to this feature is always executed. + Firmware expects this feature to be present in hardware. + +- **FEAT_STATE_CHECK** - same as ``FEAT_STATE_ALWAYS`` except that the feature's + existence will be checked at runtime. Default on dynamic platforms (example: FVP). + +.. note:: + ``FEAT_RAS`` is an exception here, as it impacts the execution of EL3 and + it is essential to know its presence at compile time. Refer to ``ENABLE_FEAT`` + macro under :ref:`Build Options` section for more details. + +Code Structure +============== + +`lib/el3_runtime/(aarch32/aarch64)`_ - Context library code directory. + +Source Files +~~~~~~~~~~~~ + +#. ``context_mgmt.c`` : consists of core functions that setup, save and restore + context for different security states alongside high level feature enablement + APIs for individual worlds. + +#. ``cpu_data_array.c`` : contains per_cpu_data structure instantiation. + +#. ``context.S`` : consists of functions that save and restore some of the context + structure members in assembly code. + +#. ``cpu_data.S`` : consists of helper functions to initialise per_cpu_data pointers. + +#. ``el3_common_macros.S`` : consists of macros to facilitate actions to be performed + during cold and warmboot and el3 registers initialisation in assembly code. + +Header Files +~~~~~~~~~~~~ + +#. ``context_mgmt.h`` : contains the public interface to Context Management Library. + +#. ``context.h`` : contains the helper macros and definitions for context entries. + +#. ``cpu_data.h`` : contains the public interface to Per CPU data structure. + +#. ``context_debug.h`` : contains public interface to report context memory + utilisation across the security states. + +#. ``context_el2.h`` : internal header consisting of helper macros to access EL2 + context entries. Used by ``context.h``. + +Apart from these files, we have some context related source files under ``BL1`` +and ``BL31`` directory. ``bl1_context_mgmt.c`` ``bl31_context_mgmt.c`` + +Bootloader Images utilizing Context Management Library +====================================================== + ++-------------------------------------------+-----------------------------+ +| Bootloader | Context Management Library | ++-------------------------------------------+-----------------------------+ +| BL1 | Yes | ++-------------------------------------------+-----------------------------+ +| BL2 | No | ++-------------------------------------------+-----------------------------+ +| BL31 (Aarch64- EL3runtime firmware) | Yes | ++-------------------------------------------+-----------------------------+ +| BL32 (Aarch32- EL3runtime firmware) | Yes | ++-------------------------------------------+-----------------------------+ + +CPU Data Structure +================== +For a given system, depending on the CPU count, the platform statically +allocates memory for the CPU data structure. + +.. code:: c + + /* The per_cpu_ptr_cache_t space allocation */ + cpu_data_t percpu_data[PLATFORM_CORE_COUNT]; + +This CPU data structure has a member element with an array of pointers to hold +the Non-Secure, Realm and Secure security state context structures as listed below. + +.. code:: c + + typedef struct cpu_data { + #ifdef __aarch64__ + void *cpu_context[CPU_DATA_CONTEXT_NUM]; + #endif + + .... + .... + + }cpu_data_t; + +|CPU Data Structure| + +At runtime, ``cpu_context[CPU_DATA_CONTEXT_NUM]`` array will be intitialised with +the Secure, Non-Secure and Realm context structure addresses to ensure proper +handling of the register state. +See :ref:`Library APIs` section for more details. + +CPU Context and Memory allocation +================================= + +CPU Context +~~~~~~~~~~~ +The members of the context structure used by the EL3 firmware to preserve the +state of CPU across exception levels for a given security state are listed below. + +.. code:: c + + typedef struct cpu_context { + gp_regs_t gpregs_ctx; + el3_state_t el3state_ctx; + el1_sysregs_t el1_sysregs_ctx; + + #if CTX_INCLUDE_EL2_REGS + el2_sysregs_t el2_sysregs_ctx; + #endif + + #if CTX_INCLUDE_FPREGS + fp_regs_t fpregs_ctx; + #endif + + cve_2018_3639_t cve_2018_3639_ctx; + #if CTX_INCLUDE_PAUTH_REGS + pauth_t pauth_ctx; + #endif + + #if CTX_INCLUDE_MPAM_REGS + mpam_t mpam_ctx; + #endif + + } cpu_context_t; + +Context Memory Allocation +~~~~~~~~~~~~~~~~~~~~~~~~~ + +CPUs maintain their context per world. The individual context memory allocation +for each CPU per world is allocated by the world-specific dispatcher components +at compile time as shown below. + +|Context memory allocation| + +NS-Context Memory +~~~~~~~~~~~~~~~~~ +It's important to note that the Normal world doesn't possess the dispatcher +component found in the Secure and Realm worlds. Instead, the PSCI library at EL3 +handles memory allocation for ``Non-Secure`` world context for all CPUs. + +.. code:: c + + static cpu_context_t psci_ns_context[PLATFORM_CORE_COUNT]; + +Secure-Context Memory +~~~~~~~~~~~~~~~~~~~~~ +Secure World dispatcher (such as SPMD) at EL3 allocates the memory for ``Secure`` +world context of all CPUs. + +.. code:: c + + static spmd_spm_core_context_t spm_core_context[PLATFORM_CORE_COUNT]; + +Realm-Context Memory +~~~~~~~~~~~~~~~~~~~~ +Realm World dispatcher (RMMD) at EL3 allocates the memory for ``Realm`` world +context of all CPUs. + +.. code:: c + + rmmd_rmm_context_t rmm_context[PLATFORM_CORE_COUNT]; + +To summarize, the world-specific context structures are synchronized with +per-CPU data structures, which means that each CPU will have an array of pointers +to individual worlds. The figure below illustrates the same. + +|CPU Context Memory Configuration| + +Context Setup/Initialization +============================ + +The CPU has been assigned context structures for every security state, which include +Non-Secure, Secure and Realm. It is crucial to initialize each of these structures +during the bootup of every CPU before they enter any security state for the +first time. This section explains the specifics of how the initialization of +every CPU context takes place during both cold and warm boot paths. + +Context Setup during Cold boot +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The cold boot path is mainly executed by the primary CPU, other than essential +CPU initialization executed by all CPUs. After executing BL1 and BL2, the Primary +CPU jumps to the BL31 image for runtime services initialization. +During this process, the per_cpu_data structure gets initialized with statically +allocated world-specific context memory. + +Later in the cold boot sequence, the BL31 image at EL3 checks for the presence +of a Secure world image at S-EL2. If detected, it invokes the secure context +initialization sequence under SPMD. Additionally, based on RME enablement, +the Realm context gets initialized from the RMMD at EL3. Finally, before exiting +to the normal world, the Non-Secure context gets initialized via the context +management library. At this stage, all Primary CPU contexts are initialized +and the CPU exits EL3 to enter the Normal world. + +|Context Init ColdBoot| + +.. note:: + The figure above illustrates a scenario on FVP for one of the build + configurations with TFTF component at NS-EL2. + +Context Setup during Warmboot +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +During a warm boot sequence, the primary CPU is responsible for powering on the +secondary CPUs. Refer to :ref:`CPU Reset` and :ref:`Firmware Design` sections for +more details on the warm boot. + +|Context Init WarmBoot| + +The primary CPU initializes the Non-Secure context for the secondary CPU while +restoring re-entry information for the Non-Secure world. +It initialises via ``cm_init_context_by_index(target_idx, ep )``. + +``psci_warmboot_entrypoint()`` is the warm boot entrypoint procedure. +During the warm bootup process, secondary CPUs have their secure context +initialized through SPMD at EL3. Upon successful SP initialization, the SPD +power management operations become shared with the PSCI library. During this +process, the SPMD duly registers its handlers with the PSCI library. + +.. code:: c + + file: psci_common.c + const spd_pm_ops_t *psci_spd_pm; + + file: spmd_pm.c + const spd_pm_ops_t spmd_pm = { + .svc_on_finish = spmd_cpu_on_finish_handler, + .svc_off = spmd_cpu_off_handler + } + +Secondary CPUs during their bootup in the ``psci_cpu_on_finish()`` routine get +their secure context initialised via the registered SPMD handler +``spmd_cpu_on_finish_handler()`` at EL3. +The figure above illustrates the same with reference of Primary CPU running at +NS-EL2. + +.. _Library APIs: + +Library APIs +============ + +The public APIs and types can be found in ``include/lib/el3_runtime/context_management.h`` +and this section is intended to provide additional details and clarifications. + +Context Initialization for Individual Worlds +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The library implements high level APIs for the CPUs in setting up their individual +context for each world (Non-Secure, Secure and Realm). + +.. c:function:: static void setup_context_common(cpu_context_t *ctx, const entry_point_info_t *ep); + +This function is responsible for the general context initialization that applies +to all worlds. It will be invoked first, before calling the individual +world-specific context setup APIs. + +.. c:function:: static void setup_ns_context(cpu_context_t *ctx, const struct entry_point_info *ep); +.. c:function:: static void setup_realm_context(cpu_context_t *ctx, const struct entry_point_info *ep); +.. c:function:: static void setup_secure_context(cpu_context_t *ctx, const struct entry_point_info *ep); + +Depending on the security state that the CPU needs to enter, the respective +world-specific context setup handlers listed above will be invoked once per-CPU +to set up the context for their execution. + +.. c:function:: void cm_manage_extensions_el3(void) + +This function initializes all EL3 registers whose values do not change during the +lifetime of EL3 runtime firmware. It is invoked from each CPU via the cold boot +path ``bl31_main()`` and in the WarmBoot entry path ``void psci_warmboot_entrypoint()``. + +Runtime Save and Restore of Registers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +EL1 Registers +------------- + +.. c:function:: void cm_el1_sysregs_context_save(uint32_t security_state); +.. c:function:: void cm_el1_sysregs_context_restore(uint32_t security_state); + +These functions are utilized by the world-specific dispatcher components running +at EL3 to facilitate the saving and restoration of the EL1 system registers +during a world switch. + +EL2 Registers +------------- + +.. c:function:: void cm_el2_sysregs_context_save(uint32_t security_state); +.. c:function:: void cm_el2_sysregs_context_restore(uint32_t security_state); + +These functions are utilized by the world-specific dispatcher components running +at EL3 to facilitate the saving and restoration of the EL2 system registers +during a world switch. + +Pauth Registers +--------------- + +Pointer Authentication feature is enabled by default for Non-Secure world and +disabled for Secure and Realm worlds. In this case, we don't need to explicitly +save and restore the Pauth registers during world switch. +However, ``CTX_INCLUDE_PAUTH_REGS`` flag is explicitly used to enable Pauth for +lower exception levels of Secure and Realm worlds. In this scenario, we save the +general purpose and Pauth registers while we enter EL3 from lower ELs via +``prepare_el3_entry`` and restore them back while we exit EL3 to lower ELs +via ``el3_exit``. + +.. code:: c + + .macro save_gp_pmcr_pauth_regs + func restore_gp_pmcr_pauth_regs + +Feature Enablement for Individual Worlds +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. c:function:: static void manage_extensions_nonsecure(cpu_context_t *ctx); +.. c:function:: static void manage_extensions_secure(cpu_context_t *ctx); +.. c:function:: static void manage_extensions_realm(cpu_context_t *ctx) + +Functions that allow the enabling and disabling of architectural features for +each security state. These functions are invoked from the top-level setup APIs +during context initialization. + +Further, a pattern is established for feature enablement code (AArch64). +Each feature implements following APIs as applicable: +Note: (``xxx`` is the name of the feature in the APIs) + +- ``is_feat_xxx_supported()`` and ``is_feat_xxx_present()`` - mandatory for all features. + +- ``xxx_enable(cpu_context * )`` and ``xxx_disable(cpu_context * )`` - optional + functions to enable the feature for the passed context only. To be called in + the respective world's setup_context to select behaviour. + +- ``xxx_init_el3()`` - optional function to enable the feature in-place in any EL3 + registers that are never context switched. The values they write must never + change, otherwise the functions mentioned in previous point should be used. + Invoked from ``cm_manage_extensions_el3()``. + +- ``xxx_init_el2_unused()`` - optional function to enable the feature in-place + in any EL2 registers that are necessary for execution in EL1 with no EL2 present. + +The above mentioned rules, followed for ``FEAT_SME`` is shown below: + +.. code:: c + + void sme_enable(cpu_context_t *context); + void sme_init_el3(void); + void sme_init_el2_unused(void); + void sme_disable(cpu_context_t *context); + +Per-world Context +================= + +Apart from the CPU context structure, we have another structure to manage some +of the EL3 system registers whose values are identical across all the CPUs +referred to as ``per_world_context_t``. +The Per-world context structure is intended for managing EL3 system registers with +identical values across all CPUs, requiring only a singular context entry for each +individual world. This structure operates independently of the CPU context +structure and is intended to manage specific EL3 registers. + +.. code-block:: c + + typedef struct per_world_context { + uint64_t ctx_cptr_el3; + uint64_t ctx_zcr_el3; + uint64_t ctx_mpam3_el3; + } per_world_context_t; + +These functions facilitate the activation of architectural extensions that possess +identical values across all cores for the individual Non-secure, Secure, and +Realm worlds. + +*Copyright (c) 2024, Arm Limited and Contributors. All rights reserved.* + +.. |Context Memory Allocation| image:: ../resources/diagrams/context_memory_allocation.png +.. |CPU Context Memory Configuration| image:: ../resources/diagrams/cpu_data_config_context_memory.png +.. |CPU Data Structure| image:: ../resources/diagrams/percpu-data-struct.png +.. |Context Init ColdBoot| image:: ../resources/diagrams/context_init_coldboot.png +.. |Context Init WarmBoot| image:: ../resources/diagrams/context_init_warmboot.png +.. _Trustzone for AArch64: https://developer.arm.com/documentation/102418/0101/TrustZone-in-the-processor/Switching-between-Security-states +.. _Security States with RME: https://developer.arm.com/documentation/den0126/0100/Security-states +.. _lib/el3_runtime/(aarch32/aarch64): https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime \ No newline at end of file diff --git a/docs/components/index.rst b/docs/components/index.rst index 18522f88f..36970260c 100644 --- a/docs/components/index.rst +++ b/docs/components/index.rst @@ -28,3 +28,4 @@ Components granule-protection-tables-design ven-el3-service ven-el3-debugfs + context-management-library diff --git a/docs/design_documents/context_mgmt_rework.rst b/docs/design_documents/context_mgmt_rework.rst index 59f9d4eab..b086e3c89 100644 --- a/docs/design_documents/context_mgmt_rework.rst +++ b/docs/design_documents/context_mgmt_rework.rst @@ -4,7 +4,7 @@ Enhance Context Management library for EL3 firmware :Authors: Soby Mathew & Zelalem Aweke :Organization: Arm Limited :Contact: Soby Mathew <soby.mathew@arm.com> & Zelalem Aweke <zelalem.aweke@arm.com> -:Status: RFC +:Status: Implementation is ongoing. Refer to :ref:`Context Management Library` for more details. .. contents:: Table of Contents @@ -194,4 +194,4 @@ improvements which are thought to have negligible impact on EL3 performance. -------------- -*Copyright (c) 2022, Arm Limited and Contributors. All rights reserved.* +*Copyright (c) 2022-2024, Arm Limited and Contributors. All rights reserved.* diff --git a/docs/resources/diagrams/context_init_coldboot.png b/docs/resources/diagrams/context_init_coldboot.png new file mode 100644 index 000000000..85606e451 Binary files /dev/null and b/docs/resources/diagrams/context_init_coldboot.png differ diff --git a/docs/resources/diagrams/context_init_warmboot.png b/docs/resources/diagrams/context_init_warmboot.png new file mode 100644 index 000000000..19f11e35a Binary files /dev/null and b/docs/resources/diagrams/context_init_warmboot.png differ diff --git a/docs/resources/diagrams/context_memory_allocation.png b/docs/resources/diagrams/context_memory_allocation.png new file mode 100644 index 000000000..a2e6a8e5c Binary files /dev/null and b/docs/resources/diagrams/context_memory_allocation.png differ diff --git a/docs/resources/diagrams/cpu_data_config_context_memory.png b/docs/resources/diagrams/cpu_data_config_context_memory.png new file mode 100644 index 000000000..3e64dddf4 Binary files /dev/null and b/docs/resources/diagrams/cpu_data_config_context_memory.png differ diff --git a/docs/resources/diagrams/percpu-data-struct.png b/docs/resources/diagrams/percpu-data-struct.png new file mode 100644 index 000000000..d8977d53d Binary files /dev/null and b/docs/resources/diagrams/percpu-data-struct.png differ