hermes-skills/voice-systems/SKILL.md

name, description, version, author, license, platforms, metadata

name	description	version	author	license	platforms	metadata
voice-systems	TTS, STT, and voice interaction patterns for Hermes Agent — timeout diagnostics, voice parity, provider quirks, and systematic fixes.	1.3.0	Hermes Agent	MIT	linux macos windows	hermes tags related_skills voice tts stt audio troubleshooting timeouts hermes-agent lxc-container-gpu-tools

Voice Systems

Class-level skill for anything involving Hermes Agent's voice pipeline: text-to-speech (TTS), speech-to-text (STT), voice message parity, and the timeouts/threads that can silently truncate audio.

Trigger Conditions

Use this skill when:

• Voice messages from the user are received but responses come back as text instead of voice
• TTS-generated voice messages are truncated or cut off at the end
• text_to_speech tool returns success: true but the resulting audio is partial or silent
• TTS fails with connection errors, timeouts, or provider-specific errors
• The user expresses voice-parity expectations ("I sent voice, why didn't you respond with voice?")
• Configuring TTS providers (OpenAI, Edge, ElevenLabs, Mistral, xAI, MiniMax, local)
• STT (faster-whisper) falls back to CPU silently or fails to load on GPU
• Voice message is received but agent responds with generic "How can I help you today?" (voice file download failed, empty transcription)
• Gateway logs show Failed to cache voice: Not Found or telegram.error.InvalidToken during voice handling
• Configuring or troubleshooting a local Telegram Bot API server (base_url, base_file_url, local_mode, Docker permissions, missing TELEGRAM_BOT_TOKEN in container env, container-internal paths not resolving on host)
• STT returns "Unsupported format: .oga" error (.oga not in SUPPORTED_FORMATS — Telegram voice files use .oga extension, not .ogg)
• TTS output starts with reasoning/thinking content ("Thought balloon reasoning...") instead of the actual answer (show_reasoning leaking into TTS)
• Telegram gateway crash-looping with token already in use despite correct voice config — see telegram-integration skill, pitfall "multi-profile gateway token conflict"

Core Patterns

Voice Parity: Voice In → Voice Out ONLY (Zero Text)

When a user sends a voice message, the required pattern is to respond with a voice message (TTS) and nothing else — no transcript, no summary, no markdown. The MEDIA tag required for delivery is platform-internal and invisible to the user. This is non-negotiable; the user has corrected this multiple times.

Critical anti-patterns:

1. Voice + text: Never accompany a voice response with any text whatsoever.
2. Text in → Voice out: Do NOT respond to text messages with voice unless explicitly asked. Voice is not a "bonus" feature — it's a specific format request matching the input format.
3. Over-correcting: After a voice-heavy session, do not start responding to new text messages with voice.

TTS Timeout Root Cause: Thread Join Truncation

The #1 cause of TTS audio being cut off: Hermes has hardcoded thread-join timeouts that cap how long the system waits for TTS generation to complete. If generation exceeds the timeout, the thread is abandoned and the partially-generated audio is delivered.

Key locations (as of May 2026):

File	Location	Default	What it controls
cli.py	~line 10249 / ~11360	120s	tts_thread.join(timeout=120) — waits for streaming TTS thread after response
cli.py	~line 10465 / ~11574	5s	tts_thread.join(timeout=5) — emergency cleanup on interrupt/exit
cli.py	~line 13078	60s	_voice_tts_done.wait(timeout=60) — continuous voice mode: waits for TTS to finish before auto-restarting recording
hermes_cli/voice.py	~line 692	60s	_tts_playing.wait(timeout=60) — waits for TTS to finish before re-arming mic
tools/tts_tool.py / build/lib/tools/tts_tool.py	multiple	60s	HTTP request timeouts for xAI, MiniMax, Gemini, Edge TTS

Fix recipe:

1. grep -rn "timeout.*120\|timeout.*60" cli.py hermes_cli/voice.py tools/tts_tool.py
2. For each TTS-related timeout, bump to 600 (10 minutes):
   • tts_thread.join(timeout=120) → tts_thread.join(timeout=600)
   • tts_thread.join(timeout=5) → tts_thread.join(timeout=600)
   • _tts_playing.wait(timeout=60) → _tts_playing.wait(timeout=600)
   • HTTP timeout=60 in provider request blocks → timeout=600
3. Patch BOTH hermes-agent/cli.py AND hermes-agent/build/lib/cli.py (build caches)
4. If the user wants NO timeout (wait forever), omit the timeout= kwarg entirely

Why 600 seconds: TTS generation over network providers (OpenAI, ElevenLabs, local Kokoro) can take 30-120 seconds for multi-paragraph text. 2 minutes (120s) is often just barely too short. 10 minutes is generous but safe.

Diagnosing TTS Issues

Step 1: Check if the tool itself succeeded

# The text_to_speech tool returns:
{"success": true, "file_path": "/path/to/audio.ogg", "media_tag": "MEDIA:/path/to/audio.ogg"}
# If success is true but audio is cut off → timeout/thread join issue (see above)
# If success is false → provider error, read the error message

Step 2: Check TTS config

# Current provider
grep -A 20 "^tts:" ~/.hermes/config.yaml
# Provider-specific keys (e.g., openai base_url, voice, model)

Step 3: Verify the TTS endpoint is reachable

# For local endpoints (Kokoro, Ollama TTS):
curl http://<host>:<port>/v1/audio/speech -d '{"model":"tts","input":"test","voice":"af_heart"}' -o /tmp/test_tts.ogg
# If this hangs or times out, the issue is upstream, not Hermes

Step 4: Check logs

grep -i "tts\|voice\|speech" ~/.hermes/logs/agent.log | tail -30

Provider-Specific Notes

OpenAI TTS (default in this environment)

• Config: tts.openai.base_url, tts.openai.model (e.g. kokoro), tts.openai.voice (e.g. af_heart)
• Uses standard OpenAI audio.speech endpoint, not /tts
• No explicit HTTP timeout in _generate_openai_tts() — relies on the OpenAI SDK's default (usually generous)
• Thread joins (cli.py) are the primary truncation risk

xAI TTS

• Uses dedicated /v1/tts endpoint, NOT OpenAI-compatible audio.speech
• Has explicit timeout=60 in requests.post (patch if needed)

Edge TTS (free, default fallback)

• Thread pool executor with .result(timeout=60) — patchable
• MP3 output; needs ffmpeg for Opus conversion on Telegram

Mistral Voxtral

• Uses /v1/audio/speech endpoint
• Explicit timeout in tts_tool.py

Kokoro (local, free, high quality)

• Runs as a standalone OpenAI-compatible TTS server on a dedicated LXC (10.0.0.16), NOT on the Hermes host.
• Docker container: kokoro-tts, image ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master, port 8880.
• Endpoint: POST /v1/audio/speech with {"model":"kokoro","input":"text","voice":"af_heart","response_format":"mp3"}
• Voice af_heart — female, warm, natural. Best-tested voice. Also available: af_nova (friendly, kid-appropriate), af_sky (alternative).
• Models: kokoro (standard), tts-1 (fast), tts-1-hd (high quality). Use kokoro.
• Latency: ~300-800ms for short sentences. Fully local, no API key, no cloud.
• Canonical IP: 10.0.0.16:8880. Do NOT use 10.0.0.228 or 10.0.0.61 — those are stale references from earlier experiments.
• Proxied through the unified voice server at 10.0.0.42:9000 so Android clients only need one host for both STT and TTS.
• See references/tts-lxc-infrastructure.md for the full Docker setup, running/stopped containers, and disk usage.
• See references/stt-server-fastapi.md for the proxy/STT server setup.

⚠️ CRITICAL: Kokoro streaming API truncation bug. The with_streaming_response.create() + response.read() pattern in _generate_openai_tts() silently truncates the final seconds of Kokoro audio. The streaming connection closes before Kokoro finishes muxing the last audio packets. The fix: use the synchronous client.audio.speech.create() + response.content instead. This was the root cause of persistent "voice messages cut off" reports despite all timeouts already being set to 600s. See references/kokoro-streaming-truncation-fix.md for the exact patch and verification recipe.

Pitfalls

1. TTS generation succeeds but audio is cut off — the most confusing failure mode. The tool returns success: true, the MEDIA tag is generated, but the last few seconds of audio are missing. This is ALWAYS the thread-join timeout. Don't blame the provider.

2. Voice file exists but the gateway cannot read it — the aiogram/telegram-bot-api container creates voice files with mode 640 owned by postfix:_ssh (uid 101:gid 101). The gateway running as user n8n cannot read them, so PTB falls back to HTTP and returns InvalidToken: Not Found. The fix must chmod from inside the container; host-side chmod fails because the host user is not in the _ssh group. The container uses BusyBox find, so commands like find -printf fail; use sh -c with simple -type d/-type f actions. See references/telegram-bot-api-permission-fix.md for the exact working recipe and a persistent cron fix.

3. base_file_url must be correct in every profile — the active Telegram profile is not the only place that matters. Other profiles that may spawn agents or subagents (gateway, research, ai, etc.) also have platforms.telegram.extra blocks and some may be misconfigured with http://localhost:8081/bot instead of http://localhost:8081/file/bot. A 404 on any of these paths produces the same empty-transcription "How can I help you today?" symptom.

4. "Connection error" from TTS — could be a genuine network issue OR the gateway/agent timeout firing before the TTS thread completes. Check whether the audio file was partially written.

5. Voice parity forgotten — when a user sends voice and you respond with text, it's not "neutral." The user may correct you. Take the correction as a skill-level preference, not a one-off complaint.

6. Voice responded to text — the opposite error. After a voice-heavy session, don't autopilot to voice for new text messages. Text in → text out.

7. Only patching source, not build — Hermes has both hermes-agent/cli.py and hermes-agent/build/lib/cli.py. The build directory is what actually runs in some contexts. Patch both or the fix is incomplete.

8. Forgetting provider HTTP timeouts — while tts_thread.join is the most common culprit, provider HTTP timeouts (xAI=60s, MiniMax=60s, Gemini=60s) can also abort generation. Bump those too.

9. External API URLs in config — the default config ships with skills.external_dirs pointing to https://hermes-agent.nousresearch.com/docs/skills and model_catalog.url pointing to the public model catalog. For local-only/privacy-sensitive deployments, strip these from ALL profiles (not just the active one). See references/external-url-cleanup.md.

10. Reasoning/thinking tokens TTS'd aloud — when display.show_reasoning: true is set and the model supports thinking (e.g., glm-5.2, DeepSeek, Kimi with thinking mode), the gateway prepends reasoning content to the response text at gateway/run.py:~9143 with response = f"💭 **Reasoning:**\n```\n{display_reasoning}\n```\n\n{response}". This prepended reasoning then flows into prepare_tts_text() and gets spoken by TTS — the user hears "Thought balloon reasoning..." before the actual answer. Fix: set display.show_reasoning: false in the Telegram profile config (hermes config set display.show_reasoning false --profile telegram). The channel_prompt should also explicitly suppress thinking output: add "no reasoning, no thinking blocks" to the voice-mode channel prompt.

11. Docker bind-mount permission drift — the cron-based permission fix (* * * * * /path/to/fix-telegram-perms.sh) only works if the host n8n user can chmod the Docker-owned files. On this system, n8n is NOT in the _ssh group (gid 101) that owns the Bot API data, so chmod from the host fails silently. The fix must run from inside the Docker container (docker exec telegram-bot-api find ... -exec chmod) or the container must be recreated with uid matching the gateway user.

11b. Cron script uses wrong container path — when the Bot API container is created with TELEGRAM_WORK_DIR=/home/n8n/telegram-bot-api-data, the cron permission-fix script must use that path, NOT the default /var/lib/telegram-bot-api/. If the script targets the wrong directory, find finds nothing and permissions never get fixed. Check with docker inspect telegram-bot-api | grep TELEGRAM_WORK_DIR and update the script to match. This caused a voice message to fail on 2026-06-19 — the cron job ran every minute but was looking in an empty directory.

12. Local Bot API container missing TELEGRAM_BOT_TOKEN — the aiogram/telegram-bot-api Docker container stores voice files under an anonymized directory (8135234357:*** — literal :***). To serve files via HTTP, the server must map the full token in the URL to this directory, which requires TELEGRAM_BOT_TOKEN in the container's environment. Without it, every file download returns 404 even though files exist on disk and permissions are correct. The container's docker inspect output will show no TELEGRAM_BOT_TOKEN in Config.Env. Fix: recreate the container with -e TELEGRAM_BOT_TOKEN=.... See references/telegram-voice-download-failures.md Root Cause 3 for the exact recreate command.

13. Container-internal paths don't resolve on the host (symlink required) — even with TELEGRAM_BOT_TOKEN set and local_mode: true, the Bot API server's getFile returns container-internal paths like /var/lib/telegram-bot-api/8135234357:***/voice/file_N.oga. PTB's is_local_file() checks if this path exists on the host filesystem — it doesn't, because the bind mount is at /home/n8n/telegram-bot-api-data, not /var/lib/telegram-bot-api. PTB falls back to HTTP download, which returns 404 because the local Bot API server does not serve files over HTTP (confirmed by tdlib/telegram-bot-api issue #540 — the server only downloads files, it does not serve them). The fix: create a symlink so container paths resolve on the host: sudo ln -sf /home/n8n/telegram-bot-api-data /var/lib/telegram-bot-api. After this, is_local_file() returns True and PTB reads files directly from disk, bypassing HTTP entirely. See references/local-bot-api-symlink-fix.md for the full recipe.

14. Token special characters break Python string parsing — the Telegram bot token contains :*** (colon + asterisks) which breaks Python string literals in execute_code and write_file when the token is embedded in source code. The linter reports SyntaxError: unterminated string literal. Workaround: write the token to a temp file first (/tmp/bot_token.txt), then read it at runtime with open('/tmp/bot_token.txt').read().strip(). Never embed the raw token in Python source strings.

15. .oga format not in SUPPORTED_FORMATS — Telegram voice messages are stored as .oga files (Ogg audio container), but tools/transcription_tools.py only lists .ogg in SUPPORTED_FORMATS. When STT receives a .oga file, _validate_audio_file() rejects it with "Unsupported format: .oga". The file exists, is readable, and contains valid audio — it just fails the format check. Fix: add .oga to SUPPORTED_FORMATS in BOTH tools/transcription_tools.py AND build/lib/tools/transcription_tools.py. The set should be: {".mp3", ".mp4", ".mpeg", ".mpga", ".m4a", ".wav", ".webm", ".ogg", ".oga", ".aac", ".flac"}. After patching, restart the gateway.

16. LD_LIBRARY_PATH must be set at process launch, not inside Python — when running faster-whisper on a CUDA 13 host with CUDA 12 ctranslate2 libraries (Ollama's /usr/local/lib/ollama/cuda_v12/), setting os.environ["LD_LIBRARY_PATH"] inside the Python script does NOT work. The dynamic linker reads LD_LIBRARY_PATH at process startup, and ctranslate2 loads libcublas.so.12 at import time. The fix: always launch with the env var on the command line: LD_LIBRARY_PATH=/usr/local/lib/ollama/cuda_v12 python3 script.py. This was discovered when a FastAPI STT server loaded the model successfully (health check showed device=cuda) but failed on first transcription with RuntimeError: Library libcublas.so.12 is not found or cannot be loaded.

17. Forgetting existing local STT infrastructure — when discussing voice options, always check what's already configured before presenting analysis. This environment has local faster-whisper STT running on the RTX 4090 (CUDA, base model, configured in both base and telegram profiles). Presenting options that ignore this (e.g., suggesting cloud STT or on-device-only STT without mentioning the existing GPU setup) wastes the user's time and erodes trust. Check grep -A10 'stt:' ~/.hermes/config.yaml and verify GPU status before any voice architecture discussion.

18. Re-suggesting eliminated categories — when the user explicitly eliminates a category ("no browser", "no Home Assistant"), do not re-suggest options from that category in later analysis. The elimination is durable for the current task.

19. Docker root-owned files blocking cleanup (no sudo) — when Docker containers wrote files as root (e.g., HuggingFace model checkpoints, torch hub caches) and the host user can't sudo, rm -rf fails with "Permission denied". The fix: use a throwaway Docker container with the same bind mount to delete as root: docker run --rm -v /path:/path alpine:latest rm -rf /path/to/delete. This works because the container's root maps to the host's root for the bind-mounted files. Used successfully on 10.0.0.16 to remove /opt/tts/fish-speech, /opt/tts/parakeet, and /opt/tts/chatterbox (all root-owned from prior Docker mounts).

Voice-Optimizing LLM Responses via channel_prompts

When a local/smaller model struggles with voice output (generates long, markdown-heavy, tool-description-laden responses that TTS can't speak naturally), use channel_prompts — Hermes' built-in per-chat ephemeral system prompt injection. It constrains the LLM at API call time without polluting session history or requiring a model swap.

How It Works

channel_prompts is a dict in the platform config (telegram.channel_prompts, discord.channel_prompts, slack.channel_prompts, mattermost.channel_prompts) mapping channel/chat IDs to prompt strings. The gateway resolves it at message dispatch time:

1. Looks up channel_prompts in the adapter's config.extra dict
2. Prefers exact match on channel_id; falls back to parent_id (forum threads inherit parent prompt)
3. Injects the matched prompt as channel_prompt on the inbound message
4. The agent prepends it to the system prompt for that turn only — never persisted to transcript

Key insight: This is ephemeral. It does NOT mutate the user profile, agent memory, or session history. It's a per-turn injection that the model sees but the transcript doesn't record.

Voice-Optimized Prompt Template

telegram:
  channel_prompts:
    '1544075739': 'VOICE MODE. You are speaking aloud — keep every response to 1-3 short sentences. No markdown, no code blocks, no lists, no tables. Speak naturally like a conversation. If you need tools, use them silently and only speak the final answer. Never describe what you are doing — just give the result.'

What this does:

• Caps response length (1-3 sentences) → TTS generation stays fast, no timeout risk
• Bans markdown/code/lists/tables → TTS doesn't try to "speak" formatting
• Forces tool-use silence → the model runs tools but only vocalizes the result
• Natural conversation tone → works with any TTS voice

Tuning for weaker models: If the model still over-generates, escalate:

• "Respond in exactly 1 sentence."
• "No tools. Answer from your knowledge only."
• "If you don't know, say 'I'm not sure' and stop."

Complementary Levers

• smart_model_routing: Route simple messages (under 200 chars/28 words) to a cheap/fast model. Voice queries tend to be short → they hit the cheap model automatically.
• gateway_voice_mode.json: voice_only mode suppresses all text alongside TTS audio. Already active for this user's Telegram chat.
• Separate profile: For extreme cases, create a Telegram-specific profile with a smaller model (e.g., kimi or glm via Ollama) and bind it to the Telegram platform.

Discovery Path

Found by searching the installed package at /opt/hermes/hermes_cli/config.py and /opt/hermes/build/lib/gateway/platforms/base.py. The channel_prompts key exists in the default config for Discord, Telegram, Slack, and Mattermost but is empty ({}) by default. The resolution function resolve_channel_prompt() at base.py:1170 handles lookup and parent fallback. Tests at tests/gateway/test_config.py:698 confirm Telegram bridging.

See references/channel-prompts-voice-optimization.md for the full technical trace.

STT GPU Verification

When diagnosing whether local STT (faster-whisper) is using the GPU, don't trust the config alone — verify at runtime:

# From the gateway's Python venv:
$HERMES_SRC/venv/bin/python3 -c "
from faster_whisper import WhisperModel
m = WhisperModel('tiny', device='auto', compute_type='auto')
print(f'Device: {m.model.device}')
" 2>&1

What to look for:

• device='auto' with LD_LIBRARY_PATH pointing to matching CUDA libs → GPU used
• CUDA library version mismatch (CUDA 12 ctranslate2 vs CUDA 13 host) → silently falls back to CPU with device='auto'
• Fix for mismatch: point LD_LIBRARY_PATH to the correct lib version (e.g., Ollama ships CUDA 12 libs at /usr/local/lib/ollama/cuda_v12/)

Key insight: faster-whisper's auto device detection is quiet about fallback. A model loading successfully with device='auto' does NOT guarantee GPU is in use. Check the model's .device property after loading. If it fell back to CPU, it uses compute_type='int8' instead of float16.

STT File Download Failures (Pre-Transcription)

Before STT can transcribe a voice message, the gateway must download the audio file from Telegram. This step can fail silently — the gateway logs a warning, the transcription gets empty input, and the agent responds with a generic "How can I help you today?" instead of answering the actual question.

Key symptom: Gateway logs show Failed to cache voice: Not Found with telegram.error.InvalidToken: Not Found. This error is misleading — the token is fine. PTB maps HTTP 404 responses to InvalidToken (see _baserequest.py lines ~360-370).

Three root causes when using a local Telegram Bot API server:

1. Wrong base_file_url — PTB has separate URL bases: base_url for API calls (default https://api.telegram.org/bot) and base_file_url for file downloads (default https://api.telegram.org/file/bot — note the /file/ segment). The local server config must set base_file_url: http://localhost:8081/file/bot, NOT http://localhost:8081/bot. If base_file_url is missing from config, the gateway falls back to base_url (wrong — returns 404).

2. Docker bind-mount permissions — When local_mode: true, PTB reads files from disk. The local Bot API server (Docker container aiogram/telegram-bot-api) stores files at a bind-mounted path owned by postfix:_ssh (uid 101). The gateway process (user n8n, uid 1000) can't read them. PTB's is_local_file() returns False, falls back to HTTP, and hits the wrong base_file_url. Fix permissions from inside the container with docker exec ... find ... -exec chmod 755 {} \\;.

3. Container-internal paths don't resolve on host (symlink required) — even with correct config and permissions, getFile returns container-internal paths like /var/lib/telegram-bot-api/8135234357:***/voice/file_N.oga. PTB's is_local_file() checks if this path exists on the host — it doesn't, because the bind mount is at /home/n8n/telegram-bot-api-data. PTB falls back to HTTP, which returns 404 because the local Bot API server does not serve files over HTTP (tdlib/telegram-bot-api issue #540). Fix: sudo ln -sf /home/n8n/telegram-bot-api-data /var/lib/telegram-bot-api. See references/local-bot-api-symlink-fix.md.

Important: The bot directory name contains *** (e.g., 8135234357:***) which is a shell glob pattern. Always use find -name "8135234357*" with -exec, never pass the literal path to chmod/ls without escaping.

See references/telegram-voice-download-failures.md for the full diagnostic flow, config fix, and Docker permission commands.

Quick Fix Script

Run this from the LXC host (or via execute_code) to patch all TTS timeouts to 600s:

#!/bin/bash
set -e
HERMES_SRC="$HOME/.hermes/hermes-agent"

for file in \
    "$HERMES_SRC/cli.py" \
    "$HERMES_SRC/build/lib/cli.py" \
    "$HERMES_SRC/hermes_cli/voice.py" \
    "$HERMES_SRC/build/lib/hermes_cli/voice.py" \
    "$HERMES_SRC/tools/tts_tool.py" \
    "$HERMES_SRC/build/lib/tools/tts_tool.py"; do
    if [ -f "$file" ]; then
        sed -i 's/tts_thread.join(timeout=120)/tts_thread.join(timeout=600)/g' "$file"
        sed -i 's/tts_thread.join(timeout=5)/tts_thread.join(timeout=600)/g' "$file"
        sed -i 's/_tts_playing.wait(timeout=60)/_tts_playing.wait(timeout=600)/g' "$file"
    sed -i 's/_voice_tts_done.wait(timeout=60)/_voice_tts_done.wait(timeout=600)/g' "$file"
        # Only replace timeout=60 in TTS-related contexts (xAI, MiniMax, Gemini, Edge)
        # Be careful not to touch non-TTS timeouts
        echo "Patched: $file"
    fi
done

echo "Done. Restart gateway/CLI for changes to take effect."

For a more targeted patch, see references/tts-timeout-fix-recipe.md.

Gateway-Level Voice-Only Text Suppression

When a chat is in voice_only mode (set via /voice tts or persisted in gateway_voice_mode.json), the gateway's base adapter auto-TTS path must suppress ALL text — no caption on the TTS audio, no separate text message. The agent's text_to_speech tool call is not enough; the gateway itself must enforce zero-text delivery.

How it works (3-part mechanism):

1. gateway_voice_mode.json persists per-chat mode: {"telegram:1544075739": "voice_only"}. Modes: off, voice_only, all.

2. GatewayRunner._sync_voice_mode_state_to_adapter() pushes state to the adapter:

   • _auto_tts_enabled_chats ← chats with mode voice_only/all
   • _auto_tts_disabled_chats ← chats with mode off
   • _voice_mode_getter ← lambda that returns the raw mode string for a chat_id

3. BasePlatformAdapter._process_message() uses _voice_mode_getter to gate delivery:

   • When voice_only: skip text caption on TTS audio, skip text message entirely
   • When all: send TTS audio with text caption (normal behavior)

Files to patch for voice_only support:

File	Change
gateway/platforms/base.py	Add _voice_mode_getter field; check it before setting TTS caption and before sending text
gateway/run.py	Wire _voice_mode_getter in _sync_voice_mode_state_to_adapter (was only on Discord before)

Verification: After restart, send a voice message to a voice_only chat — response should be TTS audio bubble with NO text alongside.

See references/voice-only-gateway-suppression.md for the exact code patches.

• user-response-style — the text counterpart to voice parity. This user wants short, no-padding text output by default. The voice rule here is a special case of that general style preference (zero text, TTS only).

Voice Surfaces: Android Tablet (Always-On, LAN)

When the user wants an "Alexa-like" always-listening voice assistant on an Android tablet connected to Hermes over LAN, evaluate options through the latency chain framework below. Full details in references/android-tablet-voice-options.md. The unified voice server (STT + TTS proxy) is documented in references/stt-server-fastapi.md.

Latency Chain Framework

Every voice assistant has 4 stages. For LAN-only setups, evaluate where each runs fastest:

Stage	On-tablet	On-server (LAN)	Typical winner
Wake word	Vosk/microWakeWord ~50ms	N/A (can't do it)	Tablet
STT	Web Speech API ~100ms (mediocre accuracy) or Vosk ~100ms	RTX 4090 + faster-whisper ~200ms + 2ms LAN (excellent accuracy)	Server for accuracy; tablet for raw speed
LLM	Impossible on tablet	Cloud model or local Ollama	Server
TTS	Android SpeechSynthesis ~50ms, instant	Edge TTS (cloud, not LAN) or local Kokoro ~500ms-2s	Tablet — biggest latency win

Key insight: TTS on-tablet is the single biggest latency win. Android's built-in SpeechSynthesis is instant (~50ms) with zero network. Server-side TTS adds 500ms-2s. STT on a server GPU (RTX 4090) adds ~200ms over on-device but is dramatically more accurate — worth it.

Quality-First Ranking (no browser, no Home Assistant)

See references/android-tablet-voice-options.md for full details. Summary:

#	Option	STT	TTS	Always-on?	Effort
1	Custom Android app (server Whisper STT)	RTX 4090 faster-whisper	Android native	Yes	High (build from scratch)
2	WakeHermesClaw	Vosk on-device	Android native	Yes	Low (install APK)
3	Fork WakeHermesClaw (swap STT)	RTX 4090 faster-whisper	Android native	Yes	Medium
4	Telegram Voice Mode	Existing gateway STT	Existing pipeline	No (push-to-talk)	Zero

User preferences applied: no browser-based solutions, no Home Assistant, cloud LLM acceptable, LAN-only for voice pipeline (STT/TTS).

Prerequisite: Hermes API Server

The API server must be enabled for any LAN voice client to connect. Add to the profile's .env (e.g., ~/.hermes/profiles/telegram/.env):

API_SERVER_ENABLED=true
API_SERVER_KEY=<generate-a-key>
API_SERVER_HOST=0.0.0.0

Then restart the gateway. The server listens on port 8642 by default. Verify:

curl http://<host-ip>:8642/v1/models -H "Authorization: Bearer <key>"
# → {"object":"list","data":[{"id":"telegram",...}]}

Test a chat completion:

curl http://<host-ip>:8642/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{"model":"telegram","messages":[{"role":"user","content":"Hello"}]}'

Pitfall: The API server reads env vars at gateway startup. Adding them to .env while the gateway is running has no effect — you must restart. The model field in requests must match the profile name (e.g., "telegram" for the telegram profile gateway).

Workflow Rule: Research Before Building

When the user asks about options, architectures, or comparisons ("is there an app for...", "what are the options...", "does this change results..."), present analysis and rankings FIRST. Do not start building, enabling services, or writing code until the user explicitly says to proceed. The user will say "Do not BUILD now, just discuss solutions ONLY" if you jump ahead.

Pitfall: Enabling the Hermes API server or building a FastAPI STT endpoint during the research phase is premature. The user wants to evaluate options before committing to implementation. Wait for a clear go-ahead like "enable it now" or "build that."

References

• references/tts-timeout-fix-recipe.md — Step-by-step patch commands with line numbers, plus verification.
• references/kokoro-streaming-truncation-fix.md — Kokoro with_streaming_response silently truncates final audio seconds. Switch to synchronous client.audio.speech.create() + response.content. Root cause of persistent cut-off voice messages despite all timeouts at 600s.
• references/voice-parity-patterns.md — When to use voice responses, platform constraints, the zero-text rule, and etiquette.
• references/voice-only-gateway-suppression.md — Code patches for voice_only mode: suppress text captions and text delivery in the gateway base adapter.
• references/telegram-voice-download-failures.md — Diagnosing InvalidToken: Not Found on voice file download: wrong base_file_url config, Docker bind-mount permission issues, missing TELEGRAM_BOT_TOKEN in container env, and container-internal path resolution.
• references/local-bot-api-symlink-fix.md — Container-internal paths don't resolve on host: symlink fix AND TELEGRAM_WORK_DIR alternative (no sudo) for is_local_file() to bypass HTTP download.
• references/oga-format-fix.md — Telegram voice files use .oga extension; add it to SUPPORTED_FORMATS in transcription_tools.py (both source and build copies).
• references/telegram-bot-api-permission-fix.md — Working recipe and persistent cron fix for the aiogram/telegram-bot-api container writing files that the gateway user cannot read.
• references/reasoning-tts-leak.md — Reasoning/thinking tokens leaking into TTS output when show_reasoning: true is set with thinking-capable models.
• references/external-url-cleanup.md — Stripping hermes-agent.nousresearch.com URLs from all profiles for local-only deployment.
• references/ollama-model-switch-cleanup.md — When switching the default model via Ollama, update every profile, fix dependent Telegram base_file_url values, clear caches, and verify the WebUI cache.
• references/android-tablet-voice-options.md — Full ranking of Android tablet always-on voice options for Hermes over LAN (quality-first, no browser, no HA). Includes latency chain framework, WakeHermesClaw details, custom app architecture, and existing infrastructure map.
• references/stt-server-fastapi.md — FastAPI + faster-whisper STT server built for the custom Android app. Endpoint spec, launch command, CUDA 12/13 LD_LIBRARY_PATH pitfall, verification steps.