Space Adventure Story Voice Mode: Build an AI-Powered Voice Game

Space Adventure Story: Voice Mode is a voice-driven immersive experience. You take on the role of the lone crew member aboard a crippled spacecraft. To survive, you must communicate with the onboard AI to diagnose critical failures and navigate the ship to safety. Every decision and interaction happens through spoken word.

The framework is powered by Pipecat, which coordinates gpt-oss for the AI's dialogue, Whisper for speech-to-text recognition, and Kokoro TTS to provide the computer’s voice. The result is a fluid, reactive environment: you speak, the ship responds, and the story unfolds in real time.

The architecture is fully modular and designed for local deployment. While it runs entirely on your own hardware by default, you can swap individual components for cloud services or alternative self-hosted API endpoints. The project remains open-source and highly customizable.

Installation

1. gpt-oss Setup

gpt-oss is compatible with any standard chat completion endpoint. For voice-based AI, you should set the reasoning level to "low" to minimize latency (the default setting is "medium"). Running the model locally via llama-server from the llama.cpp project is the most efficient method for this setup.

First, download the llama.cpp binaries or build them from the source code. You will need a specific system instruction template to enforce the low reasoning level. Use the gpt-oss-template.jinja file provided in the repository. This is necessary because llama-server does not automatically pass the reasoning parameter from API requests to the chat template.

Start the LLM server with the following command:

# Small model option
MODEL=ggml-org/gpt-oss-20b-GGUF
# Large model option
MODEL=ggml-org/gpt-oss-120b-GGUF

llama-server -hf $MODEL --verbose-prompt --chat-template-file gpt-oss-template.jinja --jinja --cache-reuse 128 -fa

2. Pipecat Voice Bot Setup

Navigate to the server directory and initialize the Python environment:

cd server
python3.12 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

3. voice-ui-kit React Frontend Setup

Switch to the client directory and install the necessary dependencies:

cd client
npm i

4. Run the Bot and Frontend

Open two terminal windows to start the services.

Terminal 1 (Backend):

cd server
source venv/bin/activate
python bot.py

Terminal 2 (Frontend):

npm run dev

Once both are running, open your browser and navigate to localhost:3000 (or the specific port assigned by the npm command). The ship's AI systems are online and awaiting your command.