🖐️🎤 Micdrop: Real-Time Voice Conversations with AI

Micdrop website | Documentation

Micdrop is a set of open source Typescript packages to build real-time voice conversations with AI agents. It handles all the complexities on the client and server side (microphone, speaker, VAD, network communication, etc) and provides ready-to-use implementations for various AI providers.

📦 Packages

Core Packages (start here)

• @micdrop/web - Browser library handling microphone input, audio playback, and real-time communication
• @micdrop/react-native - The same call on iOS and Android
• @micdrop/server - Server implementation for audio streaming and AI integration orchestration

Both client packages are built on @micdrop/client, which holds the protocol, the call state and the voice activity detection, and knows nothing about the platform it runs on. You install a platform package, not this one.

AI Implementations

• @micdrop/openai - OpenAI integration providing LLM agent and speech-to-text capabilities
• @micdrop/gemini - Google Gemini agent, speech-to-text, text-to-speech, and Gemini Live realtime model
• @micdrop/ai-sdk - AI SDK agent compatible with a lot of LLM providers.
• @micdrop/elevenlabs - ElevenLabs text-to-speech integration with streaming support
• @micdrop/cartesia - Cartesia text-to-speech integration for real-time voice synthesis
• @micdrop/gradium - Gradium speech-to-text and text-to-speech integration with WebSocket streaming
• @micdrop/mistral - Mistral AI agent and speech-to-text integration for conversation handling
• @micdrop/gladia - Gladia speech-to-text integration for audio transcription
• @micdrop/typesafe - Jev, the System One model of TypeSafe, classifying each turn of the user into typed decisions in a few hundred ms

Local Implementations

Running on your own machine, with no API key and no data leaving it.

• @micdrop/whisper - Local Whisper speech-to-text, in your Node process
• @micdrop/kokoro - Local Kokoro text-to-speech, in your Node process, English only
• @micdrop/piper - Local Piper text-to-speech, around forty languages
• @micdrop/pocket-tts - Local Kyutai Pocket TTS text-to-speech, in your Node process, cloning a voice, English only
• @micdrop/qwen-tts - Local Qwen3-TTS text-to-speech, ten languages, on an mlx-audio server

The agent runs locally through @micdrop/ai-sdk pointed at Ollama or any local server speaking the OpenAI protocol.

Utility Packages

• @micdrop/react - React hooks for Micdrop, in the browser and in React Native

Examples and demos

The examples folder holds example apps, from the smallest call to full demos, each with its README. The docs list them all, with what each one shows and the command that runs it, in Examples and demos.

🎥 Demo and technical details (video)

See the author Godefroy de Compreignac talking about Micdrop and voice AI in this video:

🤔 Why Micdrop?

While real-time multimodal models (voice-to-voice) offer impressive capabilities, they often come with limitations in terms of customization and cost. Micdrop takes a different approach by:

• 🎯 Allowing you to choose the best-in-class API for each component:
  • Select specific voices from TTS providers
  • Use different LLMs optimized for your use case
  • Pick STT engines suited for specific languages/accents
• 💰 Reducing costs by letting you:
  • Use more cost-effective API providers
  • Mix open source and commercial solutions
  • Control exactly when APIs are called
• 🔧 Providing granular control over the conversation flow
• 🌐 Supporting a wider range of languages and voices through specialized providers

This modular approach gives you the flexibility to build voice applications that are both powerful and cost-effective.

🌟 Features

• 🎙️ Microphone handling with:
  • Streaming support
  • Voice Activity Detection (VAD)
• 🔊 Advanced audio playback with:
  • Streaming support
  • Device selection and control
• 📱 The same call in the browser and in a React Native app, against one server
• 🌐 WebSocket communication
• 📦 AI implementations provided for OpenAI, ElevenLabs, Gradium, Mistral, Gladia, and more
• 🔌 Bring your own AI components (framework agnostic)
  • Large Language Models (LLM)
  • Text-to-Speech (TTS)
  • Speech-to-Text (STT)

🧪 Development

For detailed development instructions, including how to build, test, and publish packages, please see DEVELOPMENT.md.

📄 License

MIT License - see the LICENSE file for details

Author

Originally developed for Raconte.ai, created and open sourced by Godefroy de Compreignac

Acknowledgements

Thanks to ricky0123/vad for their work on voice activity detection.