PAL host ABI

PAL Host ABI is the interface used by Graphene to interact with its host. It is translated into the host’s native ABI (e.g. system calls for UNIX) by a layer called the Platform Adaptation Layer (PAL). A PAL not only exports a set of APIs (PAL APIs) that can be called by the library OS, but also acts as the loader that bootstraps the library OS. The design of PAL Host ABI strictly follows three primary principles, to guarantee functionality, security, and portability:

  • The host ABI must be stateless.
  • The host ABI must be a narrowed interface to reduce the attack surface.
  • The host ABI must be generic and independent from the native ABI of any of the supported hosts.

Most of the PAL Host ABI is adapted from the Drawbridge library OS.

PAL as loader

Regardless of the actual implementation, we require PAL to be able to load ELF-format binaries as executables or dynamic libraries, and perform the necessary dynamic relocation. PAL needs to look up all unresolved symbols in loaded binaries and resolve the ones matching the names of PAL APIs. PAL does not and will not resolve other unresolved symbols, so the loaded libraries and executables must resolve them afterwards.

After loading the binaries, PAL needs to load and interpret the manifest files. The manifest syntax is described in Manifest syntax.

Manifest and executable loading

The PAL loader supports multiple ways of locating the manifest and executable. To run a program in Graphene properly, the PAL loader generally requires both a manifest and an executable, although it is possible to load with only one of them. The user shall specify either the manifest or the executable to load in the command line, and the PAL loader will try to locate the other based on the file name.

Precisely, the loading rules for the manifest and executable are as follows:

  1. The first argument given to pal_loader can be either a manifest file or an executable.
  2. If an executable is given to the command line, the loader will search for the manifest in the following order: the same file name as the executable with a .manifest or .manifest.sgx extension, a manifest file without any extension, or no manifest at all.
  3. If a manifest is given to the command line, then the manifest will be used to infer the executable. The potential executable file has the same file name as the manifest file except it doesn’t have the .manifest or .manifest.sgx extension.

Data types and variables

Data types

PAL handles

The PAL handles are identifiers that are returned by PAL when opening or creating resources. The basic data structure of a PAL handle is defined as follows:

typedef union pal_handle {
    struct {
        PAL_IDX type;
    } hdr;
    /* other resource-specific definitions */
}* PAL_HANDLE;
union pal_handle

Public Members

PAL_IDX type
struct pal_handle::@7 hdr
typedef union pal_handle* PAL_HANDLE

As shown above, a PAL handle is usually defined as a union data type that contains different subtypes that represent each resource such as files, directories, pipes or sockets. The actual memory allocated for the PAL handles may be variable-sized.

Basic types

typedef uint64_t PAL_NUM

a number

typedef uint32_t PAL_FLG

a set of flags

typedef void* PAL_PTR

a pointer to memory or buffer (something other than string)

typedef const char* PAL_STR

a pointer to a C-string

typedef uint32_t PAL_IDX

an index

typedef bool PAL_BOL

a boolean value (either #PAL_TRUE or #PAL_FALSE)

This data type is commonly used as the return value of a PAL API to determine whether the call succeeded

PAL_TRUE

True value for #PAL_BOL.

PAL_FALSE

False value for #PAL_BOL.

typedef struct PAL_PTR_RANGE_ PAL_PTR_RANGE
struct PAL_PTR_RANGE_

Graphene control block

The control block in Graphene is a structure that provides static information about the current process and its host. It is also a dynamic symbol that will be linked by the library OS and resolved at runtime. Sometimes, for the flexibility or the convenience of the dynamic resolution, the address of the control block may be resolved by a function (pal_control_addr()).

The fields of the Graphene control block are defined as follows:

typedef struct PAL_CONTROL_ PAL_CONTROL
struct PAL_CONTROL_

Public Members

PAL_NUM process_id

An identifier of current picoprocess

PAL_HANDLE manifest_handle

program manifest

PAL_STR executable

executable name

PAL_HANDLE parent_process

handle of parent process

PAL_HANDLE first_thread

handle of first thread

PAL_BOL enable_debug_log

enable debug log calls

PAL_BOL disable_aslr

disable ASLR (may be necessary for restricted environments)

PAL_PTR_RANGE user_address

The range of user addresses

PAL_PTR_RANGE executable_range

address where executable is loaded

PAL_PTR_RANGE manifest_preload

manifest preloaded here

PAL_NUM alloc_align

Host allocation alignment.

This currently is (and most likely will always be) indistinguishable from the page size, looking from the LibOS perspective. The two values can be different on the PAL level though, see e.g. SYSTEM_INFO::dwAllocationGranularity on Windows.

PAL_CPU_INFO cpu_info

CPU information (only required ones)

PAL_MEM_INFO mem_info

memory information (only required ones)

typedef struct PAL_CPU_INFO_ PAL_CPU_INFO
struct PAL_CPU_INFO_
typedef struct PAL_MEM_INFO_ PAL_MEM_INFO
struct PAL_MEM_INFO_
PAL_CONTROL* pal_control_addr(void)

Pal APIs

The PAL APIs contain a number of functions that can be called from the library OS.

Memory allocation

The ABI includes three calls to allocate, free, and modify the permission bits on page-base virtual memory. Permissions include read, write, execute, and guard. Memory regions can be unallocated, reserved, or backed by committed memory.

PAL_PTR DkVirtualMemoryAlloc(PAL_PTR addr, PAL_NUM size, PAL_FLG alloc_type, PAL_FLG prot)

Allocate virtual memory for the library OS and zero it out.

Parameters
  • addr: can be either NULL or any valid address aligned at the allocation alignment. When addr is non-NULL, the API will try to allocate the memory at the given address and potentially rewrite any memory previously allocated at the same address. Overwriting any part of PAL and host kernel is forbidden.
  • size: must be a positive number, aligned at the allocation alignment.
  • alloc_type: can be a combination of any of the #PAL_ALLOC flags
  • prot: can be a combination of the #PAL_PROT flags

void DkVirtualMemoryFree(PAL_PTR addr, PAL_NUM size)

This API deallocates a previously allocated memory mapping.

Both

addr and size must be non-zero and aligned at the allocation alignment.
Parameters
  • addr: the address
  • size: the size

PAL_ALLOC

Memory Allocation Flags

Values:

0x1

Only reserve the memory

0x2

Allocate for PAL (valid only if #IN_PAL)

0x3
PAL_PROT

Memory Protection Flags

Values:

0x0
0x1
0x2
0x4
0x8

Copy on write

0xF
PAL_BOL DkVirtualMemoryProtect(PAL_PTR addr, PAL_NUM size, PAL_FLG prot)

Modify the permissions of a previously allocated memory mapping.

Both

addr and size must be non-zero and aligned at the allocation alignment.
Parameters
  • addr: the address
  • size: the size
  • prot: see #DkVirtualMemoryAlloc()

Process creation

The ABI includes one call to create a child process and one call to terminate the running process. A child process does not inherit any objects or memory from its parent process and the parent process may not modify the execution of its children. A parent can wait for a child to exit using its handle. Parent and child may communicate through I/O streams provided by the parent to the child at creation.

PAL_HANDLE DkProcessCreate(PAL_STR uri, PAL_STR * args)

Create a new process to run a separate executable.

Parameters
  • uri: the URI of the manifest file or the executable to be loaded in the new process.
  • args: an array of strings the arguments to be passed to the new process.

void DkProcessExit(PAL_NUM exitCode)

Terminate all threads in the process immediately.

Parameters
  • exitCode: the exit value returned to the host.

Stream creation/connect/open

The stream ABI includes nine calls to open, read, write, map, unmap, truncate, flush, delete and wait for I/O streams and three calls to access metadata about an I/O stream. The ABI purposefully does not provide an ioctl call. Supported URI schemes include: file:, pipe:, http:, https:, tcp:, udp:, pipe.srv:, http.srv, tcp.srv: and udp.srv:. The latter four schemes are used to open inbound I/O streams for server applications.

PAL_HANDLE DkStreamOpen(PAL_STR uri, PAL_FLG access, PAL_FLG share_flags, PAL_FLG create, PAL_FLG options)

Open/create a stream resource specified by uri

Supported URI types:

  • file:..., dir:...: Files or directories on the host file system. If #PAL_CREATE_TRY is given in create flags, the file/directory will be created.
  • dev:...: Open a device as a stream. For example, dev:tty represents the standard I/O.
  • pipe.srv:<name>, pipe:<name>, pipe:: Open a byte stream that can be used for RPC between processes. The server side of a pipe can accept any number of connections. If pipe: is given as the URI (i.e., without a name), it will open an anonymous bidirectional pipe.
  • tcp.srv:<ADDR>:<PORT>, tcp:<ADDR>:<PORT>: Open a TCP socket to listen or connect to a remote TCP socket.
  • udp.srv:<ADDR>:<PORT>, udp:<ADDR>:<PORT>: Open a UDP socket to listen or connect to a remote UDP socket.
Return
If the resource is successfully opened or created, a PAL handle will be returned for further access such as reading or writing.
Parameters
  • uri: is the URI of the stream to be opened/created
  • access: can be a combination of the #PAL_ACCESS flags
  • share_flags: can be a combination of the #PAL_SHARE flags
  • create: can be a combination of the #PAL_CREATE flags
  • options: can be a combination of the #PAL_OPTION flags

PAL_HANDLE DkStreamWaitForClient(PAL_HANDLE handle)

Blocks until a new connection is accepted and returns the PAL handle for the connection.

This API is only available for handles that are opened with pipe.srv:..., tcp.srv:..., and udp.srv:....

PAL_NUM DkStreamRead(PAL_HANDLE handle, PAL_NUM offset, PAL_NUM count, PAL_PTR buffer, PAL_PTR source, PAL_NUM size)

Read data from an open stream.

If the handle is a file, offset must be specified at each call of DkStreamRead. source and size can be used to return the remote socket address if the handle is a UDP socket. If the handle is a directory, DkStreamRead fills the buffer with the names (NULL-ended) of the files or subdirectories inside of this directory.

PAL_NUM DkStreamWrite(PAL_HANDLE handle, PAL_NUM offset, PAL_NUM count, PAL_PTR buffer, PAL_STR dest)

Write data to an open stream.

If the handle is a file, offset must be specified at each call of DkStreamWrite. dest can be used to specify the remote socket address if the handle is a UDP socket.

Return
number of bytes written if succeeded, PAL_STREAM_ERROR on failure (in which case PAL_ERRNO() is set)

void DkStreamDelete(PAL_HANDLE handle, PAL_FLG access)

Delete files or directories on the host or shut down the connection of TCP/UDP sockets.

Parameters
  • access: which side to shut down (#PAL_DELETE), or both if 0 is given.

PAL_PTR DkStreamMap(PAL_HANDLE handle, PAL_PTR address, PAL_FLG prot, PAL_NUM offset, PAL_NUM size)

Map a file to a virtual memory address in the current process.

offset and size have to be non-zero and aligned at the allocation alignment

Parameters
  • address: can be NULL or a valid address that is aligned at the allocation alignment.
  • prot: see #DkVirtualMemoryAlloc()

void DkStreamUnmap(PAL_PTR addr, PAL_NUM size)

Unmap virtual memory that is backed by a file stream.

addr and size must be aligned at the allocation alignment

PAL_NUM DkStreamSetLength(PAL_HANDLE handle, PAL_NUM length)

Set the length of the file referenced by handle to length.

Return
Returns the 0 on success, a positive errno on failure.

PAL_BOL DkStreamFlush(PAL_HANDLE handle)

Flush the buffer of a file stream.

PAL_BOL DkSendHandle(PAL_HANDLE handle, PAL_HANDLE cargo)

Send a PAL handle over another handle.

Currently, the handle that is used to send cargo must be a process handle.

Parameters
  • cargo: the handle being sent

PAL_HANDLE DkReceiveHandle(PAL_HANDLE handle)

This API receives a handle over another handle.

PAL_BOL DkStreamAttributesQuery(PAL_STR uri, PAL_STREAM_ATTR * attr)

Query the attributes of a named stream.

This API only applies for URIs such as file:..., dir:..., and dev:....

typedef struct _PAL_STREAM_ATTR PAL_STREAM_ATTR
struct _PAL_STREAM_ATTR
PAL_BOL DkStreamAttributesQueryByHandle(PAL_HANDLE handle, PAL_STREAM_ATTR * attr)

Query the attributes of an open stream.

This API applies to any stream handle.

PAL_BOL DkStreamAttributesSetByHandle(PAL_HANDLE handle, PAL_STREAM_ATTR * attr)

Set the attributes of an open stream.

PAL_NUM DkStreamGetName(PAL_HANDLE handle, PAL_PTR buffer, PAL_NUM size)

Query the name of an open stream.

PAL_BOL DkStreamChangeName(PAL_HANDLE handle, PAL_STR uri)

This API changes the name of an open stream.

PAL_STREAM_ERROR

error value of read/write

Flags used for stream manipulation

PAL_ACCESS

Stream Access Flags

Values:

0
1
2
4
7
PAL_SHARE

Stream Sharing Flags

Values:

01
02
04
010
020
040
0100
0200
0400
01000
02000
04000
07777
PAL_CREATE

Stream Create Flags

Values:

1

Create file if file does not exist

2

Create file and fail if file already exists

4

Create dual-stack socket (opposite of IPV6_V6ONLY)

7
PAL_OPTION

Stream Option Flags

Values:

1
2

specific to eventfd syscall

4
7
PAL_DELETE

Values:

1

shut down the read side only

2

shut down the write side only

Thread creation

The ABI supports multithreading through five calls to create, sleep, yield the scheduler quantum for, resume execution of, and terminate threads, as well as seven calls to create, signal, and block on synchronization objects.

PAL_HANDLE DkThreadCreate(PAL_PTR addr, PAL_PTR param)

Create a thread in the current process.

Parameters
  • addr: is the address of an entry point of execution for the new thread
  • param: is the pointer argument that is passed to the new thread

PAL_NUM DkThreadDelayExecution(PAL_NUM duration)

Suspend the current thread for a certain duration.

Parameters
  • duration: the duration in microseconds

void DkThreadYieldExecution(void)

Yield the current thread such that the host scheduler can reschedule it.

void DkThreadExit(PAL_PTR clear_child_tid)

Terminate the current thread.

Parameters
  • clear_child_tid: is the pointer to memory that is erased on thread exit to notify LibOS (which in turn notifies the parent thread if any); if clear_child_tid is NULL, then PAL doesn’t do the clearing.

PAL_BOL DkThreadResume(PAL_HANDLE thread)

Resume a thread.

Exception handling

PAL_EVENT

Values:

1

arithmetic error (div-by-zero, floating point exception, etc.)

2

segmentation fault, protection fault, bus fault

3

illegal instructions

4

terminated by external program

5

suspended by external program

6

continued by external program

7

failure within PAL calls

8
typedef struct PAL_CONTEXT_ PAL_CONTEXT
struct PAL_CONTEXT_
typedef void(* PAL_EVENT_HANDLER)(PAL_PTR event, PAL_NUM arg, PAL_CONTEXT *)
PAL_BOL DkSetExceptionHandler(PAL_EVENT_HANDLER handler, PAL_NUM event)

Set the handler for the specific exception event.

Parameters
  • event: can be one of #PAL_EVENT values

void DkExceptionReturn(PAL_PTR event)

Exit an exception handler and restore the context.

Synchronization

PAL_HANDLE DkMutexCreate(PAL_NUM initialCount)

Create a mutex with the given initialCount.

Destroy a mutex using DkObjectClose.

Parameters
  • initialCount: 0 is unlocked, 1 is locked

void DkMutexRelease(PAL_HANDLE mutexHandle)

Unlock the given mutex.

PAL_HANDLE DkNotificationEventCreate(PAL_BOL initialState)

Creates a notification event with the given initialState.

The definition of notification events is the same as the WIN32 API. When a notification event is set to the signaled state it remains in that state until it is explicitly cleared.

PAL_HANDLE DkSynchronizationEventCreate(PAL_BOL initialState)

Creates a synchronization event with the given initialState.

The definition of synchronization events is the same as the WIN32 API. When a synchronization event is set to the signaled state, a single thread of execution that was waiting for the event is released, and the event is automatically reset to the not-signaled state.

void DkEventSet(PAL_HANDLE eventHandle)

Set (signal) a notification event or a synchronization event.

void DkEventClear(PAL_HANDLE eventHandle)

Clear a notification event or a synchronization event.

Objects

NO_TIMEOUT

block until the handle’s event is triggered

PAL_BOL DkSynchronizationObjectWait(PAL_HANDLE handle, PAL_NUM timeout_us)

Wait on a synchronization handle.

Return
true if this handle’s event was triggered, false otherwise
Parameters
  • timeout_us: is the maximum time that the API should wait (in microseconds), or #NO_TIMEOUT to indicate it is to be blocked until the handle’s event is triggered.

PAL_BOL DkStreamsWaitEvents(PAL_NUM count, PAL_HANDLE * handle_array, PAL_FLG * events, PAL_FLG * ret_events, PAL_NUM timeout_us)

Poll.

Return
true if there was an event on at least one handle, false otherwise
Parameters
  • count: the number of items in the array
  • handle_array:
  • events: user-defined events
  • ret_events: polled-handles’ events in ret_events
  • timeout_us: is the maximum time that the API should wait (in microseconds), or NO_TIMEOUT to indicate it is to be blocked until at least one handle is ready.

void DkObjectClose(PAL_HANDLE objectHandle)

Close (deallocate) a PAL handle.

Miscellaneous

The ABI includes seven assorted calls to get wall clock time, generate cryptographically-strong random bits, flush portions of instruction caches, increment and decrement the reference counts on objects shared between threads, and to obtain an attestation report and quote.

PAL_NUM DkDebugLog(PAL_PTR buffer, PAL_NUM size)

Output a message to the debug stream.

Works only if the debug stream has been initialized, which can be checked by looking at g_pal_control.enable_debug_log.

Return
number of bytes written if succeeded, PAL_STREAM_ERROR on failure (in which case PAL_ERRNO() is set)

PAL_NUM DkSystemTimeQuery(void)

Get the current time.

Return
the current time in microseconds

PAL_NUM DkRandomBitsRead(PAL_PTR buffer, PAL_NUM size)

Cryptographically secure random.

Return
0 on success, negative on failure
Parameters
  • buffer: is filled with cryptographically-secure random values
  • size: buffer size

PAL_PTR DkSegmentRegister(PAL_FLG reg, PAL_PTR addr)

Set segment register.

Parameters
  • reg: the register to be set (#PAL_SEGMENT)
  • addr: the address to be set; if NULL, return the current value of the segment register.

PAL_SEGMENT

Values:

0x1
0x2
PAL_NUM DkMemoryAvailableQuota(void)

Return the amount of currently available memory for LibOS/application usage.

PAL_BOL DkCpuIdRetrieve(PAL_IDX leaf, PAL_IDX subleaf, PAL_IDX values[PAL_CPUID_WORD_NUM])

Return CPUID information, based on the leaf/subleaf.

Parameters
  • values: the array of the results

PAL_CPUID_WORD

Values:

0
1
2
3
4
PAL_BOL DkAttestationReport(PAL_PTR user_report_data, PAL_NUM * user_report_data_size, PAL_PTR target_info, PAL_NUM * target_info_size, PAL_PTR report, PAL_NUM * report_size)

Obtain the attestation report (local) with user_report_data embedded into it.

Currently works only for Linux-SGX PAL, where user_report_data is a blob of exactly 64B, target_info is an SGX target_info struct of exactly 512B, and report is an SGX report obtained via the EREPORT instruction (exactly 432B). If target_info contains all zeros, then this function additionally returns this enclave’s target info in target_info. Useful for local attestation.

The caller may specify *user_report_data_size, *target_info_size, and *report_size as 0 and other fields as NULL to get PAL-enforced sizes of these three structs.

Parameters
  • user_report_data: Report data with arbitrary contents (typically uniquely identifies this Graphene instance). Must be a 64B buffer in case of SGX PAL.
  • user_report_data_size: Caller specifies size of user_report_data; on return, contains PAL-enforced size of user_report_data (64B in case of SGX PAL).
  • target_info: Target info of target enclave for attestation. If it contains all zeros, it is populated with this enclave’s target info. Must be a 512B buffer in case of SGX PAL.
  • target_info_size: Caller specifies size of target_info; on return, contains PAL-enforced size of target_info (512B in case of SGX PAL).
  • report: Attestation report with user_report_data embedded, targeted for an enclave with provided target_info. Must be a 432B buffer in case of SGX PAL.
  • report_size: Caller specifies size of report; on return, contains PAL-enforced size of report (432B in case of SGX PAL).

PAL_BOL DkAttestationQuote(PAL_PTR user_report_data, PAL_NUM user_report_data_size, PAL_PTR quote, PAL_NUM * quote_size)

Obtain the attestation quote with user_report_data embedded into it.

Currently works only for Linux-SGX PAL, where user_report_data is a blob of exactly 64B and quote is an SGX quote obtained from Quoting Enclave via AESM service.

Parameters
  • user_report_data: Report data with arbitrary contents (typically uniquely identifies this Graphene instance). Must be a 64B buffer in case of SGX PAL.
  • user_report_data_size: Size in bytes of user_report_data. Must be exactly 64B in case of SGX PAL.
  • quote: Attestation quote with user_report_data embedded.
  • quote_size: Caller specifies maximum size allocated for quote; on return, contains actual size of obtained quote.

PAL_BOL DkSetProtectedFilesKey(PAL_PTR pf_key_hex)

Set wrap key (master key) for protected files.

Currently works only for Linux-SGX PAL. This function is supposed to be called during remote attestation and secret provisioning, before the user application starts.

Parameters
  • pf_key_hex: Wrap key for protected files. Must be a 32-char null-terminated hex string in case of SGX PAL (AES-GCM encryption key).