Within the enclave itself, DRAM and PCIe connections between the CPU and GPU are encrypted, but the CPU registers and the GPU onboard memory are plaintext. So the computation is happening on plaintext data, it’s just extremely difficult to access it from even the machine running the enclave.
How is it then much different than trusting the policies of Anthropic etc? To be fair you need some enterprise deal to get the truly zero retention policy.
Enclaves have a property that allows the hardware to compute a measurement (a cryptographic hash) of everything running inside it, such as the firmware, system software such as the operating system and drivers, the application code, the security configuration. This is signed by the hardware manufacturer (Intel/AMD + NVIDIA).
Then, verification involves a three part approach. Disclaimer: I'm the cofounder of Tinfoil: https://tinfoil.sh/, we also run inference inside secure enclaves. So I'll explain this as we do it.
First, you open source the code that's running in the enclave, and pin a commitment to it to a transparency log (in our case, Sigstore).
Then, when a client connects to the server (that's running in the enclave), the enclave computes the measurement of its current state and returns that to the client. This process is called remote attestation.
The client then fetches the pinned measurements from Sigstore and compares it against the fetched measurements from the enclave. This guarantees that the code running in the enclave is the same as the code that was committed to publicly.
So if someone claimed they were only analyzing aggregated metrics, they could not suddenly start analyzing individual request metrics because the code would change -> hash changes -> verification fails.
> First, you open source the code that's running in the enclave, and pin a commitment to it to a transparency log (in our case, Sigstore).
This means you have reproducible builds as well? (source+build-artifacts is signed)
Also - even if there are still some risk that the link is not 100% safe, maybe it's safe to assume vendors like yourself going through all that trouble are honorable? (alternatively - they are very curious of what "paranoid" people would send through LLMs :sweatsmile:)
We don't have reproducible builds because we attest the full OS image that we run, which is the Ubuntu image. Unfortunately bit-by-bit reproducible binaries for OS images is kind of an unsolved problem, because it requires the hundreds of package maintainers across all dependencies to eliminate any sources of non-determinism in the compilation. Things like timestamps and file reordering are very common and even one of these changes the entire hash.
So we do the next best thing. We decide to trust Github and rely on Github Actions to faithfully execute the build pipeline. We also make sure to pin all images and dependencies.
Arguably this is less useful for consumer hardware in the first place. This is mostly useful when I don’t trust the service provider with my data but still need to use their services (casting my vote, encrypted inference, and so forth)
True, in the case of casting a vote though for example, I would see it being used within the voting machines itself before sending off to be counted. Good application.
But getting them available for customers for example say even a PCIe card or something and then that automatically encrypting everything you ever run today over an encrypted connection would be a dream.
Most people outside of a narrow set of cryptography engineers are unfamiliar with the term anonymous credentials, while age and identity verification are two privacy-invasive requirements that are being heavily discussed and rapidly being written into laws lately. The post's intro discusses both quite heavily, and they form the author's entire motivation for writing the post.
The central question the post attempts to answer is "The problem for today is: how do we live in a world with routine age-verification and human identification, without completely abandoning our privacy?"
My rephrase is an attempt to surface that, compared to the dry and academic title that will get overlooked. I think this is a very important topic these days where we are rapidly ceding are privacy to at best, confused and at worst, malicious regulations.
AI has normalized single 9's of availability, even for non-AI companies such as Github that have to rapidly adapt to AI aided scaleups in patterns of use. Understandably, because GPU capacity is pre-allocated months to years in advance, in large discrete chunks to either inference or training, with a modest buffer that exists mainly so you can cannibalize experimental research jobs during spikes. It's just not financially viable to have spades of reserve capacity. These days in particular when supply chains are already under great strain and we're starting to be bottlenecked on chip production. And if they got around it by serving a quantized or otherwise ablated model (a common strategy in some instances), all the new people would be disappointed and it would damage trust.
Less 9's are a reasonable tradeoff for the ability to ship AI to everyone I suppose. That's one way to prove the technology isn't reliable enough to be shipped into autonomous kill chains just yet lol.
> AI has normalized single 9's of availability, ...
FWIW I use AI daily to help me code...
And apparently the output of LLMs are normalizing single 9's too: which may or may not be sufficient.
From all the security SNAFUs, performance issues, gigantic amount of kitchen-skinky boilerplate generated (which shall require maintenance and this has always been the killer) and now uptime issues this makes me realize we all need to use more of our brains, not less, to use these AI tools. And that's not even counting the times when the generated code simply doesn't do what it should.
For a start if you don't know jack shit about infra, it looks like you're already in for a whole world of hurt: when that agent is going to rm -rf your entire Git repo and FUBAR your OS because you had no idea how to compartmentalize it, you'll feel bad. Same once all your secrets are going to publicly exposed.
It looks like now you won't just be needing strong basis about coding: you'll also be needing to be at ease with the entire stack. Learning to be a "prompt engineer" definitely sounds like it's the very easy part. Trivial even.
we are going to do the same "everything is binary" engineer thing with bombs and innocent casualties we did with self driving? there is also an accountability crisis that will unfold if we loose these things on the world, it is not just one metric is better than human operators therefore take your hands off the wheel and hope for the best. Please file a ticket with support if your child's school was accidentally destroyed.
There's probably a curve of diminishing returns when it comes to how much effort you throw in to improve uptime, which also directly affects the degree of overengineering around it.
I'm not saying that it should excuse straight up bad engineering practices, but I'd rather have them iterate on the core product (and maybe even make their Electron app more usable: not to have switching conversations take 2-4 seconds sometimes when those should be stored locally and also to have bare minimum such as some sort of an indicator when something is happening, instead of "Let me write a plan" and then there is nothing else indicating progress vs a silently dropped connection) than pursue near-perfect uptime.
Sorry about the usability rant, but my point is that I'd expect medical systems and planes to have amazing uptime, whereas most other things that have lower stakes I wouldn't be so demanding of. The context I've not mentioned so far is that I've seen whole systems get developed poorly, because they overengineered the architecture and crippled their ability to iterate, sometimes thinking they'd need scale when a simpler architecture, but a better developed one would have sufficed!
Ofc there's a difference between sometimes having to wait in a queue for a request to be serviced or having a few requests get dropped here and there and needing to retry them vs your system just having a cascading failure that it can't automatically recover from and that brings it down for hours. Having not enough cards feels like it should result in the former, not the latter.
Ehh, I'd say there's not much difference between the UI being in a state that's for all intents being frozen due to not having a status indicator or actually having a dropped request and the UI doing nothing because there's nothing going on, you know?
Or having the Electron UI being sluggish 99% of the time during daily use vs dealing with that 1% of the time when there's outages. I'd rather have that 99% be good and 1% not work, than 99.9% be miserable and 0.01% not work.
Yep, it's not like electricity which is an essential service.
If electricity faults for one second, chaos breaks loose as so many things depend on it. Imagine having several microblackouts a day? We wouldn't tolerate. It's so reliable that normal people don't need redundancy for it, they just tap into the stream which is always available.
AI is definitely shaping up to NOT become like that. We're designing for an unreliable system (try again buttons, etc), and the use cases follow that design.
Recent paper by Nicholas Carlinini and others really showcases how little it takes to deanonymize users across platforms with LLMs: https://arxiv.org/abs/2602.16800
I would be interested to see Apple Silicon in the future, given its much stronger isolation and integrity guarantees. But that is an entirely different tech stack.
Apple does something very similar with Apple Private Cloud Compute. It's interesting cause their isolation argument is different. For instance, memory is not encrypted (so weaker protection against physical attacks), but they measure and guarantee integrity (and need to trust) all code running on the machine, not just inside the secure enclave.
Good question is how many lines of code do you need to trust at the end of the day between these different designs.
Lines of code hardly means anything, but I'd believe Apple has far fewer, given how aggressively they curtail their platforms rather than letting them collect legacy cruft.
Also around to answer any questions!
reply