Beyond Apple’s Promises: The Real Story on AI Privacy and Security
Apple has unveiled new AI capabilties and "Private Cloud Compute." Despite PCC's robust security measures, concerns remain about potential vulnerabilities inherent to Trusted Execution Environments (TEEs).
Apple just presented its new AI capabilities along with its new "Private Cloud Compute," which it intends to use to eliminate all privacy concerns regarding the cloud processing of sensitive iPhone user data. It makes very strong promises regarding privacy guarantees, verifiability, and trust. Let's investigate if it can hold up.
For context, Apple goes to great lengths to secure every step of its hardware and firmware supply chain. Still, Trusted Execution Environments have a long history of vulnerabilities, and the promises made are nothing new. At the same time, it's great that we see this mainstream push towards data protection, and Apple seems to be taking this issue seriously.
Apple’s Private Cloud Compute
Apple designed Private Cloud Compute (PCC), a trusted hardware-based, trusted execution platform specifically designed for processing work from their new AI solutions. Previously, most of their AI inference work was executed locally in their dedicated Neural Engine. This announcement shows a significant shift towards cloud-based inference, which raises privacy concerns, as Apple offloads user data to data centers. Additionally, Apple is greatly expanding the data used for those inference tasks, which raises even more privacy concerns. It's also unclear if cloud processing will remain optional (at the cost of slower and less reliable responses). To address these privacy concerns, Apple has introduced PCC.
PCC falls into the category of Secure Enclaves or Trusted Execution Environments (TEEs). TEEs promise hardware-based security and privacy by introducing secure hardware elements on chips. They have been around for many years and are utilized for secure data computation. We have even seen entire blockchain networks relying on them as hardware requirements in their nodes (I will get back to that later). However, TEEs have inherent trust assumptions associated with them on multiple layers of the stack and their supply chain. But first, let's look at what Apple is doing with PCC.
I think it's inspiring the lengths they have gone to harden every step of the data processing in their dedicated hardware, both on the hardware, firmware, and software levels. Firstly, they are not using Intel SGX, AMD TrustZone, or any other preexisting TEE stack; instead, they are building their own. They are running dedicated data centers for their hardware. For requests (AI inference), the user device first requests a set of nodes to potentially process the request from the load balancer. Apple claims the load balancer has no way of identifying the end user. After receiving encryption public keys for these nodes, the user verifies the attested measurement of the nodes' code and data (to be convinced that they are running authentically and untampered with hardware). If they match the publicly released measurements by Apple, the user sends their encrypted data off to those PCC nodes via the load balancer and a relayer to mask the IP addresses of the nodes. The request is processed in a secure enclave in the node, and the output is encrypted and sent back.
Still, the source code for the firmware and SDK remains (mostly) proprietary, and only parts of the binaries and measurements are published. Apple claims to have very high privacy guarantees, so let's dive into where problems can arise. I want to touch on general points from the history of TEEs, as Apple's architecture shares many similarities (from what we have seen so far) with previous designs.
The section titled “Non Targetability” in Apple’s Private Cloud Compute announcement explains the preventive measures taken to ensure security and protection of physical hardware. Apple says:
Private Cloud Compute hardware security starts at manufacturing, where we inventory and perform high-resolution imaging of the components of the PCC node before each server is sealed and its tamper switch is activated. When they arrive in the data center, we perform extensive revalidation before the servers are allowed to be provisioned for PCC. The process involves multiple Apple teams that cross-check data from independent sources, and the process is further monitored by a third-party observer not affiliated with Apple.
(https://security.apple.com/blog/private-cloud-compute/)
However, it also highlights the level of trust involved in the manufacturing and software distribution process. The need to physically secure each part of the supply chain and the active deployment shows the level of trust involved in the entire process.
TEE vulnerabilities and previous exploits
Last week, I spoke at the Confidential Computing Summit in San Fransisco with a talk titled "Trusted Execution Is Dead. And We Killed It.". There are many problems with the usage of TEEs. We can categorize them into:
- Side-channel attacks and hardware vulnerabilities
- Firmware, microcode, and SDK bugs
- Supply chain attacks
Given the history of TEEs and the similarity of the claims and hardware architecture employed by other TEE manufacturers (like Intel with SGX), we have to be skeptical. I will only be scratching the surface, but let's look at four exploits of the past (also from the blockchain space):
1. The Secret Network exploit
In 2022, researchers detailed at https://sgx.fail/ how they were able to exploit the xAPIC (CVE-2022-21233, CVE-2022-38090) and MMIO (CVE-2022-21166) vulnerabilities in Intel SGX TEEs to extract Secret Network's consensus seed, the network's entire secret master decryption key, which was shared and used by each node to decrypt the network's secret state. The vulnerabilities allowed any node that was part of this blockchain network to decrypt and view all private user transactions, exposing sensitive user data. Eventually, they were able to implement a more robust secret-sharing mechanism and upgrades in collaboration with Intel
2. Foreshadow attack
The https://foreshadowattack.eu/ (CVE-2018-3620 and CVE-2018-3646) exploited speculative execution vulnerabilities in Intel processors to breach the isolation of SGX enclaves, allowing attackers to extract sensitive data from third-party cloud environments. This is a prime example of a side-channel attack. It demonstrated that even advanced security measures could be bypassed through side-channel techniques.
3. Plundervolt
In the https://plundervolt.com/ attack (CVE-2019-11157), attackers extracted data by manipulating the voltage and frequency of Intel CPUs (exploiting the dynamic frequency and voltage scaling features). This enabled them to induce faults within SGX enclaves, leading to data leakage. By injecting precise undervoltage, the CPU can produce errors, allowing attackers to alter the enclave operations.
4. Supply chain attacks
We have to see publicly disclosed supply chain attacks for TEEs, but they have happened in hardware manufacturing and firmware/software distribution. A prime example is the ShadowHammer Attack against ASUS' software update tool, which became compromised and allowed attackers to insert and distribute malware via legitimate signing keys through ASUS' update distribution infrastructure. Attackers can target many parts of TEE supply chains, and I am convinced that Apple attempts its best to prevent these kinds of attacks.
Still, history has shown us that the sheer complexity of hardware-based security combined with proprietary software allows for many sophisticated attack vectors, only identified over time. Mainly, performance optimizations have often led to new attacks. While fixes might be implemented rapidly (Apple also has a dedicated bug bounty program), attackers can still extract sensitive user data. It's great that Apple has additionally implemented decentralization techniques. Still, Apple's claims in their docs reflect what we have heard from other players, which have regularly fallen victim to vulnerabilities and exploits. Trust remains a significant factor, as the name Trusted Execution Environment already communicates.
The future is trustless
We are actively confronting these challenges with Arcium. However, hardware-based security and cryptographic protocols like our implementation can be used symbiotically. For more details, read our documentation.
Apple's private compute is a step in the right direction, especially as a leading technology company influencing mainstream adoption, further hardening its infrastructure with decentralized attack surfaces, relaying, etc. As recent regulations and exploits have shown, private AI is becoming increasingly relevant. With this step, other companies will have to follow, and we are at the forefront of building a new kind of confidential computing that removes trust from these architectures.
