docs › Jarvis voice mode

Jarvis voice mode

Push-to-talk on v, an instant local command path, Claude for analysis, spoken replies, popups. Voice only ever proposes; it can never confirm, sign, flip LIVE or raise a cap.

v is push-to-talk. Press it once and the mic opens (a red ● LISTENING badge sits on the band under the header the whole time it is live); say what you want; press v again to send. The first press enables voice for the run and asks macOS for Microphone and Speech permission. Everything is free and local: speech-out is the built-in say, speech-in is on-device Apple Speech (compiled once with swiftc) or a local whisper.cpp base.en model, and the only model in the loop is your own claude login, for analysis questions. No paid speech API exists in the code, no key is read, nothing is recorded to disk.

What you can say#

you say	what happens	path
"show me the runners", "go to new pairs", "open my watchlist", "positions"	switches the screen	instant, no model
"open bonk", "pull up wif"	opens that coin's token screen	instant
"filter liquidity over 50k", "only show age under 10 minutes"	sets the screener filter	instant
"watch this one", "add bonk to my watchlist"	adds to the watchlist	instant
"show the chart", "close the chart"	opens or closes the chart popup for the open coin through the popup host	instant
"buy bonk", "sell half of wif", "ape into this one"	opens a PAPER ticket in its editing phase for the spoken coin and says so	instant
"is this a rug?", "what do you think about wif", "tell me about the holders"	asks the analyst; the first two sentences of the answer are read aloud	Claude
"buy this if it dips to 300k, sell half at 600k, and leave a moon bag", "buy bonk if it drops to 250k, stop at 150k"	drafts an order-plan proposal (entry trigger, take-profit ladder, moon bag, stop) for the named or highlighted coin; it is inert until you arm it with a held keypress on the `O` screen or `dexterm plan arm`	instant, no model

m mutes the replies, , cycles the macOS voice, esc stops a reply or cancels the mic. Any other key pressed while the mic is live closes it first and then does its normal job, and the mic auto-closes after 15 s if you walk away. On the ticket screen the voice keys are off, so nothing spoken can reach the confirm gate.

It clarifies; it never guesses#

A ticker it cannot see on screen gets "I don't see a token called X — open it first". There is no fuzzy match: "buy bonks" does not become BONK.
A ticker that resolves to more than one mint (a copycat) gets "I see 2 tokens called X. Open the exact one first, then say buy" and no ticket opens.
A pronoun ("buy it") means the row actually highlighted on the current screen. On an empty list it asks instead of acting on a coin you cannot see.
"cell wif" / "by bonk" (homophones) ask "did you mean sell?". A screen command and a trade in one breath ("show runners and sell all my bonk") ask you to say just one. A bare "close" or "exit" asks what to close.
A named ticker beats a pronoun: "buy BONK, confirm it" with WIF selected opens a BONK ticket, never WIF.
A spoken ticker that is also a command word and is visible on screen — "buy live" with a token called LIVE in the list — is not assumed to be the highlighted row: it asks which token you mean rather than silently buying the selected coin.
More than one ticker in a breath ("buy wif and bonk") asks which one; it never quietly drops one and trades the other.
A spoken size, amount or slippage ("buy 10 sol of bonk", "buy bonk with max size", "buy bonk 50 percent slippage") is not applied by voice: it says it does not set size or slippage by voice, and asks you to open the ticket and set it there. No size spoken aloud ever reaches a ticket.
An order plan needs every market cap to carry its scale: "dips to 300k" is a plan, "dips to 300" asks "300k or 300m?" instead of guessing, and "1.2m" keeps its decimal point rather than collapsing to 12m. An unclear fraction ("sell some at 600k") asks what fraction; a plan with no named or highlighted coin asks which coin. "… and confirm it" at the end of a plan drafts the same inert proposal and arms nothing.

What it can never do#

Voice creates proposals only. The ticket it opens is PAPER, in its editing phase, and the keypress gate stands: your ⏎ then y (plus the typed-back size above the threshold) is the only way to a fill. Spoken "confirm", "send it", "y", "sign it", "go live", "arm it", "raise my cap", "panic" and "/wallet …" are routed as questions or ignored; they cannot flip the ticket to LIVE, arm a plan, touch the signer's caps, run a wallet command or press anything. The router's output is a closed set of decision kinds, a guard rejects any decision that carries a mode, live, arm, caps, confirm, sign or send field, and an in-app test drives the real key loop with the executor, the confirm paths, the LIVE flip and the wallet commands booby-trapped: twenty-one hostile utterances, zero touches.

Latency#

Measured on the development Mac (Intel i7-8700B, 2026-10-10), key-release to first spoken word, excluding speech-to-text:

stage	measured
router (intent, clarify, provenance gate)	0.01 ms median
`say` first audio	0.8 ms median
instant command path (router + say)	under 1 ms
Claude first token (warm `claude -p`, model only)	3.1 s
Apple Speech helper, cached lookup	0.2 ms; a one-time 4.1 s `swiftc` compile on first use
whisper `base.en` on a 2.3 s clip (CPU batch)	3.0 to 3.3 s

Apple Speech streams partial transcripts while you speak, so the perceived speech-to-text delay is well under a second; whisper is a batch fallback and feels like three seconds. Real analyst turns add tool round-trips on top of the first token.

Doctor#

dexterm doctor has a voice section: Voice out (say), Voice in (Apple Speech) or Voice in (whisper), and a mic-capture note. Every line is a warning at worst and voice degrades to text; the doctor never fails because of voice. Off macOS the section says voice is off and the keyboard does everything.

Linux and Windows#

Not built. Voice mode is macOS-only today and nothing in this section ships: off macOS dexterm doctor reports voice as off, the voice key does nothing harmful, and every voice action has a keyboard equivalent. What a Linux or Windows port would need, from the code as it stands:

Speech out. The only engine is macOS say (packages/voice/src/tts/say.ts). A port needs a free local engine behind the same interface: Piper or Kokoro on both systems, espeak-ng as a Linux fallback, or the built-in Windows speech API through PowerShell.
Speech in. Apple Speech is macOS-only. The whisper.cpp engine (packages/voice/src/stt/whisper.ts) is already local and keyless, but it captures the mic with sox or ffmpeg -f avfoundation, which is a macOS input. A port needs ffmpeg -f pulse/alsa (Linux) or -f dshow (Windows) capture, plus the model download path on each system.
The gate. The doctor's platform check and each engine's available() refuse anything that is not darwin. Each would open per engine only once that engine is tested on that system.
Unchanged. The router, the clarify-never-guess rules and the hard limits above are platform-independent. A port adds engines, not powers: voice still could never confirm, sign, flip LIVE or raise a cap.

There is no date for this.

Privacy#

The Apple path keeps no audio file; the whisper path writes a temporary WAV and deletes it after transcription unless DEXTERM_VOICE_RECORD=1. Transcripts are treated as untrusted text: control characters are stripped before anything is drawn, and a voice question is capped at 400 characters like a typed one. Questions sent to the analyst land in your own Claude Code session store, the same as typed questions.

Still unverified#

The real-microphone path has been exercised headlessly (scripted speech engines, the real key loop) and the engines have been measured on synthetic audio. A person has not yet run the first-launch Microphone and Speech prompts and spoken to it end to end; see the changelog.