huggingface/speech-to-speech

原文摘要

Build local voice agents with open-source models Speech To Speech: Build voice agents with open-source models A low-latency, fully modular voice-agent pipeline: VAD -> STT -> LLM -> TTS , exposed through an OpenAI Realtime-compatible WebSocket API . Every component is swappable. The LLM slot speaks OpenAI-compatible protocols, so you can point it at a hosted provider, at HF Inference Providers , or at a vLLM or llama.cpp server on your own hardware for a fully local, fully open stack. This pipeline runs in production as the conversation backend for thousands of Reachy Mini robots. Quickstart pip install speech-to-speech export OPENAI_API_KEY=... speech-to-speech This starts an OpenAI Realtime-compatible server at ws://localhost:8765/v1/realtime using Parakeet TDT for local STT, an OpenAI-compatible LLM, and Qwen3-TTS for local speech output. From a source checkout, talk to it from a second terminal: python scripts/listen_and_play_realtime.py --host 127.0.0.1 --port 8765 Prefer to keep the LLM on your own machine? Serve Gemma 4 with llama.cpp: llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full Then point the OpenAI-compatible LLM backend at it: speech-to-…