Skip to main content

Architecture

Voxa is a phone app talking to a small server on your laptop, which drives Claude Code and speaks the result back to you. This page maps the pieces and walks through the two ways they can be wired together.

The components​

  • Phone client: the native iOS app, or a plain browser page served by the laptop itself. It captures your microphone, plays back the operator's voice, and shows live transcripts, terminal output, and approval cards.
  • Laptop FastAPI server: the process started by voxa (or npx voxa-code). It serves the phone web client, terminates the phone's WebSocket connection, and hosts the orchestrator, controllers, and session registry. It listens on 127.0.0.1:8787 by default.
  • Voice operator: a Gemini Live session, either running directly against Google (GeminiOperator) or through Voxa's metered proxy (RemoteOperator). It listens, talks, and calls tools; it does not write code itself.
  • Orchestrator: the piece that maps a tool call from the operator onto an actual action, whether that's starting a session, sending text to Claude, reading git status, or building an approval card.
  • Claude controllers: the thing actually driving Claude Code. By default Voxa launches a visible, interactive Claude session in tmux that you can watch and type into yourself. There's also a headless mode that runs Claude through the Claude Agent SDK with a read-only watch log, and controllers that attach to a terminal you already have open (iTerm2, Terminal.app, or generic apps via Accessibility).
  • Notifier / call manager: decides what happens when Claude finishes work in the background, whether that's speaking the result if you're on a call, ringing the phone, or staying quiet.
  • Hosted Voxa Cloud: the service at api.voxa.space. It combines a relay (matches your phone to your laptop), a metered Gemini Live proxy, and push notification / call routing.

Two topologies​

Voxa runs one of two ways, chosen by whether you've set your own GEMINI_API_KEY.

(a) Hosted, zero-config mode​

This is the default for voxa and npx voxa-code. There's no port to open and no API key to manage on the laptop.

┌──────────────┐                          ┌───────────────────────────┐                         ┌─────────────┐
│  Phone app    │                         │      Voxa Cloud            │                         │   Laptop     │
│               │                         │   (api.voxa.space)          │                         │              │
│               │  wss /ws?code=<code>    │  ┌───────────┐              │  wss /agent?code=<code> │  FastAPI     │
│               │◄───────────────────────►│  │  relay    │◄────────────┼─────────────────────────┤  server      │
│               │                         │  └───────────┘              │  (outbound, no inbound  │  (127.0.0.1) │
│               │                         │                             │   port needed)          │              │
│               │  audio, transcripts     │  ┌───────────┐              │  wss /live?account=...  │              │
│               │  (via relay + local ws) │  │  /live    │◄────────────┼─────────────────────────┤  Remote      │
│               │                         │  │  proxy    │              │  RPC tool calls back     │  Operator    │
│               │                         │  │ (Gemini,  │──────────────┼─────────────────────────►│              │
│               │                         │  │  cloud's  │              │                          │  Orchestrator│
│               │                         │  │  API key) │              │                          │  → Claude    │
│               │                         │  └───────────┘              │                          │              │
└──────────────┘                          └───────────────────────────┘                         └─────────────┘

Matching happens with a persisted 128-bit pairing code, not a phone number or account lookup:

  1. The laptop dials outbound to wss://api.voxa.space/agent?code=<code>&token=<relay-token>. No inbound port or tunnel is needed.
  2. The phone connects to the same relay's /ws?code=<code>. The relay matches phone and laptop purely by that code and copies frames both ways; it doesn't parse audio or transcript content.
  3. When the phone shows up, the relay tells the laptop a peer is connected. The laptop bridge then opens its own local /ws (the same one the browser client would use) and starts pumping frames between the relay and that local socket.
  4. Voice runs through the metered /live proxy: the cloud runs the actual Gemini Live session, using the cloud's own Gemini key, meters the minutes, and RPCs every tool call back to the laptop for execution.

So there are two cloud sockets doing different jobs: a dumb relay that just moves phone traffic, and a metered proxy that hosts the Gemini session. Both live in the same deployable. The laptop never holds a Gemini key in this mode, and there's nothing to expose to the internet.

(b) Fully self-hosted​

Set your own GEMINI_API_KEY and the laptop talks to Google directly instead of going through the cloud proxy.

┌──────────────┐        Tailscale HTTPS serve         ┌──────────────┐        wss (direct)        ┌──────────────┐
│  Phone app    │◄────────or Cloudflare tunnel────────►│  Laptop       │◄──────────────────────────►│   Google      │
│               │        (or your own relay)           │  FastAPI      │   your GEMINI_API_KEY       │  Gemini Live  │
└──────────────┘                                       └──────────────┘                             └──────────────┘
  • With GEMINI_API_KEY set, the laptop runs GeminiOperator and connects to Google Gemini Live directly, no cloud involved for voice.
  • For phone reachability without the hosted relay, the CLI prefers a Tailscale HTTPS serve, and falls back to a Cloudflare quick tunnel if Tailscale isn't set up. Either way, a QR code and pairing URL are printed so the phone can connect.
  • If you want the relay/proxy convenience without depending on Voxa's cloud, you can point VOXA_RELAY_URL / VOXA_LIVE_PROXY at your own deployment of the same cloud service.
  • If GOOGLE_GENAI_USE_VERTEXAI is set, the local operator authenticates via a Google Cloud service account and project instead of an API key.

See self-hosting for the full setup.

The life of a voice session​

  1. Connect: the phone opens /ws?token=<auth_token>. A mismatched token closes the connection immediately.
  2. Free setup gate: the metered voice operator doesn't open, and no minutes get billed, until you actually tap Start or start talking. Before that point you can still pick a working folder, list open terminals, or attach to one, all for free.
  3. Audio up: your mic is captured in the browser or app, downsampled to 16 kHz mono PCM, and streamed to the operator as binary WebSocket frames. Muting sends silence rather than closing the channel.
  4. Audio down: Gemini's spoken reply comes back as 24 kHz mono PCM and plays through a jitter buffer on the phone.
  5. Transcripts: both sides of the conversation are captioned live on the phone as they happen.
  6. Tool calls: when you ask for something, the operator calls a tool (say, send_to_claude), the orchestrator dispatches it, and Claude gets to work in its controller (visible tmux session by default).
  7. Claude works: you can watch the terminal feed stream to the phone while it runs. The operator relays Claude's on-screen questions back to you if it needs input.
  8. Result spoken back: when Claude finishes, the result is spoken to you if a line is open, or the phone rings if you've stepped away.

A Claude session persists in the server's session registry across phone reconnects: hanging up doesn't stop the work.

Keeping a session alive​

Gemini Live connections cap out at roughly 10 minutes each. Voxa works around this with session resumption: it holds a resume handle and transparently reopens the connection mid-call, so a long conversation doesn't feel interrupted. Context compression keeps the conversation window manageable across those reconnects.

To avoid burning minutes on a call nobody's using, the operator auto-disconnects after a period of no speech and no Claude activity, 180 seconds by default (configurable via VOXA_IDLE_TIMEOUT).

What data flows where​

The short version: your microphone audio and tool-call arguments/results go to whichever Gemini Live endpoint you're using (Google directly in self-hosted mode, or the cloud's metered proxy in zero-config mode). The relay in zero-config mode only ever sees encrypted-in-transit frames matched by pairing code, not their contents. Claude Code itself, your files, and your command execution never leave the laptop, regardless of mode.

For the full breakdown of what each component can see and what's logged, read security.