Introduction to SGX¶
Graphene project uses SGX to securely run software. SGX is a complicated topic, which may be hard to learn, because the documentation is scattered through official/reference documentation, blogposts and academic papers. This page is an attempt to curate a dossier of available reading material.
SGX is an umbrella name of technology that comprises several parts:
- CPU/platform hardware features: the new instruction set, new microarchitecture with the PRM (EPC) memory region and some new MSRs and some new logic in the MMU and so on;
- the SGX Remote Attestation infrastructure, online services provided by Intel and/or third parties (see DCAP);
- SDK and assorted software.
SGX is still being developed. The current (March 2020) version of CPU features is referred to as “SGX1” or simply “SGX” and is more or less finalized. All new/changed instructions from original SGX are informally referred to as “SGX2”.
Features which might be considered part of SGX2:
- EDMM (Enclave Dynamic Memory Management) is part of SGX2
- FLC (Flexible Launch Control), not strictly part of SGX2, but was not part of original SGX hardware either
As of now there is hardware support (on a limited set of CPUs) for FLC and (on an even more limited set of CPUs) SGX2/EDMM. Most of the literature available (especially introduction-level) concerns original SGX1 only.
- Quarkslab’s two-part “Overview of Intel SGX”:
- MIT’s deep dive in SGX architecture.
- Intel’s whitepapers:
- Hardware compatibility list (unofficial)
- Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3D: System Programming Guide, Part 4
- SDK for Linux (download of both the binaries and the documentation)
- Intel’s collection of academic papers, likely the most comprehensive list of references
Linux kernel drivers¶
For historical reasons, there are three SGX drivers currently (March 2020):
https://github.com/intel/linux-sgx-driver – old one, does not support DCAP, deprecated
https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/driver – new one, out-of-tree, supports both non-DCAP software infrastructure (with old EPID remote-attestation technique) and the new DCAP (with new ECDSA and more “normal” PKI infrastructure).
Upstreaming in-kernel SGX driver (see LKML patches) – will be upstreamed one day, supports both non-DCAP and DCAP. The DCAP driver closely matches this upstreaming version.
The in-tree driver will not be a module (https://lore.kernel.org/linux-sgx/20190401225717.GA13450@linux.intel.com/), so “installation instructions” will likely be minimal.
Also it will not require IAS and kernel maintainers consider non-writable FLC MSRs as non-functional SGX: https://lore.kernel.org/lkml/20191223094614.GB16710@zn.tnic/
The chronicle of kernel patchset:
- v1 (2016-04-25)
- v4 (2017-10-16)
- v5 (2017-11-13)
- v6 (2017-11-25)
- v7 (2017-12-07)
- v8 (2017-12-15)
- v9 (2017-12-16)
- v10 (2017-12-24)
- v11 (2018-06-08)
- v12 (2018-07-03)
- v13 (2018-08-27)
- v14 (2018-09-25)
- v15 (2018-11-03)
- v16 (2018-11-06)
- v17 (2018-11-16)
- v18 (2018-12-22)
- v19 (2019-03-20)
- v20 (2019-04-17)
- v21 (2019-07-13)
- v22 (2019-09-03)
- v23 (2019-10-28)
- v24 (2019-11-30)
- v25 (2020-02-04)
- v26 (2020-02-09)
- v27 (2020-02-23)
- v28 (2020-04-04)
- v29 (2020-04-22)
- v30 (2020-05-15)
- Architectural Enclaves
A set of “system” enclaves concerned with starting and attesting other enclaves.
- Data Center Attestation Primitives
A software infrastructure provided by Intel as a reference implementation for the new ECDSA/PCS-based remote attestation. Relies on the Flexible Launch Control hardware feature. In principle this is a special version of SDK/PSW that has a reference launch enclave and is backed by the DCAP-enabled SGX driver.
This allows for launching enclaves without Intel’s remote infrastructure. But this requires deployment of own infrastructure, so is operationally more complicated. Therefore it is intended for server environments (where you control all the machines).
- Orientation Guide
- A way to launch enclaves with Intel’s infrastructure, intended for client machines.
- Enclave Dynamic Memory Management
- A hardware feature of SGX2, allows dynamic memory allocation, which in turn allows dynamic thread creation.
- Enclave Page Cache
- Enclave Page Cache Map
- Enhanced Privacy Identification
- Enhanced Privacy Identifier
- Enhanced Privacy Identifier
Contrary to DCAP, EPID may be understood as “opinionated”, with most moving parts fixed and tied to services provided by Intel. This is intended for client enclaves and deprecated for server environments.
- A way to launch enclaves without relying on the Intel’s infrastructure.
- Flexible Launch Control
Hardware (CPU) feature that allows substituting Launch Enclave for one not signed by Intel. A change in SGX’s EINIT logic to not require the EINITTOKEN from the Intel-based Launch Enclave. An MSR, which can be locked at boot time, keeps the hash of the public key of the “launching” entity.
With FLC, Launch Enclave can be written by other companies (other than Intel) and must be signed with the key corresponding to the one locked in the MSR (a reference Launch Enclave simply allows all enclaves to run). The MSR can also stay unlocked and then it can be modified at run-time by the VMM or the OS kernel.
- Launch Enclave
- Local Attestation
- Intel Attestation Service
Internet service provided by Intel for “old” EPID-based remote attestation. Enclaves send SGX quotes to the client/verifier who will forward them to IAS to check their validity.
- Provisioning Certification Service, another Internet service provided by Intel.
- Memory Encryption Engine
- SGX Platform Software
- Software infrastructure provided by Intel with all special Architectural Enclaves (Provisioning Enclave, Quoting Enclave, Launch Enclave). This mainly refers to the “old” EPID/IAS-based remote attestation.
- Processor Reserved Memory
- Provisioning Enclave
- Intel Provisioning Certification Service
New internet service provided by Intel for new ECDSA-based remote attestation. Enclave provider creates its own internal Attestation Service where it caches PKI collateral from Intel’s PCS, and the verifier gets the certificate chain from the enclave provider to check validity.
- Intel Attestation Service, another Internet service.
- Quoting Enclave
- Remote Attestation
- Intel SGX Software Development Kit
- Intel SGX SDK
- SGX SDK
- Intel SGX SDK
- In the context of SGX, this means a specific piece of software
supplied by Intel which helps people write enclaves packed into
.sofiles to be accessible like normal libraries (at least on Linux). Available together with a kernel module and documentation.
- SGX Enclave Control Structure
This refers to all new SGX instructions and other hardware features that were introduced after the release of the original SGX1.
Encompasses at least EDMM, but is still work in progress.
- State Save Area
- Security Version Number
- Trusted Execution Environment
- Trusted Computing Base
In context of SGX this has the usual meaning: the set of all components that are critical to security. Any vulnerability in TCB compromises security. Any problem outside TCB is not a vulnerability, i.e. should not compromise security.
In context of Graphene there is also a different meaning (Thread Control Block). Those two should not be confused.
- Thread Control Structure