Using local models + zkAPI + Tor for private AI

Submit
vitalik.eth@vitalik.eth

Doing a bit of a self-experiment. Goal: use my personal health and travel data to provide personalized diet and exercise recommendations for me, using frontier models but in a way that avoids leaking to them any private information. Strategy: use a local model (Qwen 3.8 Flash Next) to orchestrate, and use powerful remote models as a tool call to benefit from higher-level thinking and knowledge that the local model does not have. Three-layer approach to privacy: Avoid leaking personally identifiable information, or leaking my identity through writing style -> local model writes the queries to the frontier model, not me Avoid leaking who I am through the payment channel -> zkAPI Avoid leaking who I am through networking / IP -> Tor You need all three (and finally we have all three, at least to some extent) A skill file teaches the local model when and how to construct minimally-data-revealing requests to remote models. Use zkAPI-wrapped-with-Tor as a CLI tool. And everything works! I got the recommendations back, info from frontier models helped to improve them. Main deficiencies: * Tor is really not optimized for request-by-request de-linking, which is the only form of network-layer privacy that really makes sense in today's world (long-running identifiers are too fragile). Probably not private enough, and latency 10-100x higher than it could be, at the same time. * The skill file's request construction strategies are definitely far from optimal. * Qwen 3.8 Flash Next is still too slow for comfort. It's comfortably running at 20-30 TPS, but it would only really start to feel fast at 100+ * There is a tradeoff: the more careful you are about what data you give to a remote model, the less it can help you https://github.com/ethereum/zkapi/pull/1…

Cast image
farcaster.xyz

Using local models + zkAPI + Tor for private AI

AI summary of the linked article

The author describes a self-experiment to get personalized diet and exercise recommendations from frontier AI models using their personal health and travel data without leaking private information. A local model, Qwen 3.8 Flash Next, orchestrates the process and writes the queries to remote models, which the author says helped improve the recommendations. The approach uses three privacy layers: the local model to avoid leaking identity through content or writing style, zkAPI to avoid leaking identity through payments, and Tor to avoid leaking it through networking. The author lists deficiencies, including Tor being poorly suited to request-by-request de-linking and adding high latency, the skill file's query strategies being far from optimal, the local model running at 20-30 TPS, and a tradeoff where more privacy limits how much remote models can help.