iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Build the conversation as a stream, not a sequence of completed files: send browser audio over WebRTC, process speech and generate speech incrementally, and play audio as soon as useful output is ready. Use FastAPI for application and session control, Modal for hosting the real-time and inference services, and PostgreSQL for durable state—not for storing or synchronously handling every audio frame.
There is no established end-to-end benchmark or latency guarantee for this exact Modal, FastAPI, and PostgreSQL combination. The right design depends on your models, deployment region, concurrency, and tolerance for cold starts, so measure the stages and user-visible behavior under your own workload.
What a low-latency voice pipeline needs to do
A voice agent has several pieces of work that can otherwise add delay one after another: capture microphone audio, transport it, recognize or otherwise process speech, assemble context, generate a response, synthesize speech, and return playable audio. The key is to stream across those stages wherever the chosen models and providers allow it, rather than waiting for a full recording, transcript, model response, and audio file before starting the next step.
Modal’s published voice example describes a flow that receives audio over WebRTC, transcribes it, queries documentation, assembles context, generates and parses a response, synthesizes speech, and returns audio and related data over WebRTC. That example uses a Pipecat bot, separate inference deployments, and WebSockets over Modal Tunnels to communicate with its STT and TTS services. Treat this as one documented architecture, not a requirement for every application or proof of the latency you will achieve.
#1 Best Overall
- BUILT FOR DICTATION & VIBE CODING – Talk to your AI assistant, dictate code, or draft documents by voice. The Movo WebMic's clear, close-up capture means fewer transcription errors so your words land right the first time.
- CARDIOID PICKUP FOR CLEAN VOICE-TO-TEXT – The directional cardioid capsule focuses on your voice and rejects noise from behind, giving speech-to-text engines and AI prompts the clean input they need to stay accurate.
- HANDS-ON CONTROLS, ONE-TOUCH MUTE – Built-in knobs adjust mic gain and headphone monitoring level, a 3.5mm headphone jack lets you hear yourself live, and one-touch mute keeps you in control during calls and long coding sessions.
- PLUG AND PLAY ON PC & MAC – Connect over USB with no drivers or extra hardware. Works instantly with your dictation app, AI coding tools, and voice typing — the LED glows to show you're connected and turns red when muted.
- DESKTOP STAND + 1-YEAR WARRANTY – Includes a desktop stand that keeps the mic at talking distance on your desk, backed by friendly US-based support and a 1-year warranty.
Separate the media path from application state
The media path carries audio and time-sensitive events. The application path handles authorization, tenant routing, session setup, tool or business actions, and durable records. Keeping these responsibilities distinct makes it easier to change a model or transport without tying correctness to a particular container’s memory.
Choose transports for their jobs
| Connection | Good fit | Important constraint |
|---|---|---|
| Browser to voice service: WebRTC | Interactive, audio-first browser sessions and media tracks. | Browser microphone access requires permission and a secure context: HTTPS or localhost. OpenAI’s browser guide describes negotiated media tracks for microphone input and generated speech, plus a data channel for events. |
| Service to service: WebSocket | Persistent, bidirectional communication such as a voice service connecting to separate STT or TTS deployments. | Modal supports WebSockets on ASGI web functions, but a WebSocket connection occupies a function call. Plan for lifecycle, concurrency, and container capacity. |
| Application setup and control: HTTP or ASGI endpoints | Authentication, session creation, routing, and non-streaming application operations. | Keep control requests separate from a long-lived media session where that makes the system easier to operate. |
Modal’s FastRTC example demonstrates WebRTC with an ASGI application. FastAPI can provide the ASGI application boundary and WebSocket endpoints, while a media framework or dedicated service owns the real-time audio handling. Do not assume that putting all these responsibilities in one endpoint makes the system simpler: it can couple request validation, long-lived connections, inference, and state management to the same scaling limits.
Arrange the services around the conversation
- Establish a trusted session. Have the browser request session setup from your application. Authenticate the user, apply tenant and authorization checks, and return only the connection details the browser needs. Keep provider credentials on trusted server infrastructure; OpenAI’s browser WebRTC flow also uses a trusted server for session setup.
- Open the media connection. Negotiate the browser’s WebRTC session and request microphone permission. Use HTTPS outside localhost. Keep signaling and application events distinct from the media track where possible.
- Process incoming speech incrementally. Send audio to the speech-recognition or speech-processing stage as it arrives. If STT runs in another deployment, a persistent service connection such as WebSocket is one option; Modal’s example uses WebSockets over Tunnels for its separate STT and TTS deployments.
- Start response work as soon as useful input is available. Feed partial or final transcript information and relevant context to the response stage according to your model’s interface. Avoid adding an unnecessary wait for a complete transcript if the product can safely act on partial results.
- Return generated speech in chunks. Start playback when the first usable audio is ready instead of buffering the entire response. Make playback buffering and cancellation behavior explicit so a new user utterance can interrupt stale output.
- Persist only durable application facts. Record session metadata, required user or account state, tool/action results, and summaries that need to survive container replacement. Keep this work independent of each audio frame’s real-time path.
Choose Modal Functions or Servers deliberately
ASGI or WebSocket web functions
Modal documents WebSocket support for @modal.web_server, @modal.asgi_app, and @modal.wsgi_app. A WebSocket connection is maintained as a function call. A container may serve multiple concurrent inputs when configured to do so, and Modal can add containers as demand grows; a request may also wait for a cold start when no container is active. This differs from treating a short stateless HTTP request as the unit of work.
Rank #2
- Built-in AI Noise Reduction: Compared to the base model, G11 pro upgraded AI noise cancellation, effectively eliminates distractions like fan noise, keyboard clicks. It delivers clear, crisp teleconferencing experiences, making it perfect for conference calls, online learning and chatting
- Omnidirectional Conference Mic: Features omnidirectional pickup pattern with a pickup distance of 11.5 ft, making it easy to capture sounds from 360° directions. Highly sensitive pickup ensures participants hear everything clearly. Tips: This is not a speaker
- Effortless Control: Physical volume and monitoring control buttons are built into the microphone body, allowing you to effortlessly adjust both microphone and monitoring volume. Click to adjust volume between 4 levels
- Mute & Monitor: Quickly mute/unmute your microphone by one tap. Built-in 3.5mm jack allows connection of headphones for monitoring. Long press for 3 seconds to enable/disable: Blue-Mic mode, Red-Mute, Purple-Monitoring. Note: Do not connect the 3.5mm jack to external speakers, as this may cause feedback interference
- Plug & Play: Compatible with all operating systems,both Windows and macOS. No additional drivers needed . If there is no response after inserting the mic, please go to the microphone setting of your computer and select the mic as the INPUT device
Set concurrency from measurements of per-session CPU, GPU use, memory, provider limits, and tail latency. The Modal FastRTC demo uses ten concurrent streams and a ten-minute time limit as sample configuration values; they are example settings, not production recommendations. Load-test with representative sessions before choosing limits.
Modal Servers
Modal Servers are an alternative for low-latency HTTP workloads served by a process in a container. The process is expected to handle concurrency, and autoscaling is configured separately from Functions. Modal documents that a request during a zero-to-one transition can receive HTTP 503 when no container is active. If using this option, make retry behavior safe for the operation or keep suitable capacity warm when avoiding that rejection is worth the cost.
Session affinity and reconnects
Modal documents session affinity for workloads such as voice and video, subject to session-lifetime and concurrency constraints. Affinity can make reconnects or locality-sensitive session work easier, but it is not durable storage: persist any conversation state that must survive a container replacement in an appropriate database.
Rank #3
- Smart Magnetic Design, 3 Ways to Use - Lay flat on your desk, clip it to your clothing, or attach it to any metal surface with the magnetic patch. Includes 3 reusable nano adhesive dots for extra placement options. The 2m cable gives you room to move. Perfect for laptop, desktop, and travel
- 360° Omnidirectional Pickup - With high sensitivity 360° voice pickup and a 6 ft (2 m) range, this desktop microphone ensures everyone is heard clearly. Great for gaming, streaming, video conferences, and online classes
- Mute Button & LED Indicator - Instantly mute/unmute with one press, ensuring your privacy during meetings. The three color LED clearly shows mic status (Blue: AI Noise Reduction; Green: Original Mode; Red: Muted)
- AI Noise Reduction vs Original Mode - Switch between two powerful modes. Use AI Mode (Blue Light) to filter out background noises like fans or keyboard clicks. For a natural, realistic sound that captures your true voice with high fidelity, we recommend switching to Original Mode (Green Light). Pro Tip: If you experience transcription issues, toggling between these two modes can help optimize performance based on your specific setup
- Plug & Play, Wide Compatibility - No drivers needed. This pc microphone for desktop works instantly with Windows 7/8/10/11 and Mac OS. Supports major platforms (Zoom, Teams, Skype) and recording apps. Quick Tip: If your device is not detected, please check your computer's privacy settings to allow microphone access for the app you're using
Use PostgreSQL for durable state, not audio-frame round trips
PostgreSQL is a good fit for durable session metadata, account state, tool/action records, and summaries needed for product behavior or audit. Do not synchronously query it for every audio frame, and do not hold a transaction or checked-out connection while waiting for transcription, model generation, or speech synthesis. Keep database transactions short and persist only what correctness or the product requires.
PostgreSQL uses a process-per-client connection model. Its PostgreSQL 18 documentation says max_connections is typically 100 by default, although system capability can make the default lower. That is a database setting, not a recommended pool size for a particular voice application.
Plan for the aggregate connection count across scaled containers: active containers × pool size per container, plus connections for migrations, administration, and other services. Set explicit pool limits, acquire connections only for short operations, and load-test the aggregate against the actual database limit. Use a pooling arrangement appropriate to how the database is hosted.
Rank #4
- Without Built in Speaker- Please note that AIRHUG 21 microphone for pc does not have a speaker function. Built in an excellent 360° omnidirectional microphone pick up your voice within adius 6 ft. You don't have to loudly speak up to the computer or laptop
- Be Hear Your Clear Voice - With an advanced AIRHUG noise-canceling technology, better than traditional microphone technology. The sampling rate of the pc microphone is 48k hz. When at the online calls, the other side hear your clear and real voice
- AI Noise Reduction Mode - AIRHUG 21 USB microphone is with AI Noise Reduction Mode,eliminating background noise such as fans noise, keyboard clicks, and general background noise.Provide clear and crisp online calls for you.Great for your online learning,podcasting,conferencing and gaming. For a natural, realistic sound that captures your true voice with high fidelity, we recommend switching to Original Mode (Green Light)
- Smart Memory& Mute Function& LED Indicator - Every restart, the computer microphone starts in recording mode (not muted), so you never miss sound by accident. It also remembers your last sound mode (noise reduction or original). No need to adjust every time. Every recording starts the way you like, easy and simple. You can direct operate mute mode for this pc microphone. The built-in indicator light of mic informs the status(Blue: AI Noise Reduction; Green: Original Mode; Red: Muted)
- Widely Compatible Feature - AIRHUG 21 external microphone for laptop is great for small conference with 1-3 participants. The conference microphone is compatible with Zoom,Skype,Microsoft,Teams,Google meeting,Webex,Facetime, and most of the online meeting apps. It is a great choice for anyone who needs to make video meeting, online education,seminars, remote training, business negotiations,etc
Measure latency where users experience it
Track time to first audible response and interruption recovery as primary user-facing measures. Add stage timings to identify where delay comes from:
- Connection and session setup, including network and cold-start time.
- Time to first partial transcript and time to final transcript.
- Model time to first token (TTFT) and time per output token (TPOT) for streamed responses.
- Time to first audio (TTFA), audio chunk timing, and playback buffering.
- Total response completion time and the time needed to stop or replace output after an interruption.
Modal recommends streaming to improve perceived responsiveness and defines TTFT and TPOT for streamed model responses. These measures answer different questions: a quick first token does not guarantee quick first audio, and quick first audio does not mean the whole response finishes quickly.
Modal’s engineering article describes a general conversational target of around one second or less and says distributed computation makes that difficult. The same vendor article describes some natural voice-to-voice exchanges as short as 100 milliseconds, while qualifying that conversational context and interruption behavior matter. These are illustrative vendor statements, not an SLA, independent benchmark, or expected result for a Modal, FastAPI, and PostgreSQL deployment.
Best Value
- 360 Voice Pickup for Clear Group Meetings: Designed with high-sensitivity microphones and 360 omnidirectional voice pickup, this conference speakerphone captures voices clearly from all directions, ensuring everyone in the room can be heard without raising their voice. Suitable for small to medium-sized meetings, video calls, online classes, and remote collaboration where clear communication matters.
- AI Noise Reduction & Echo Cancellation: Advanced noise reduction and echo cancellation technology intelligently filters out background noise while enhancing human voices. Whether you're working from a home office, shared workspace, or busy environment, your voice stays clear and natural for professional-quality meetings without distractions.
- Plug & Play USB Connectivity: No drivers or software required. Simply connect the speakerphone to your computer via USB and start your meeting instantly. Works seamlessly with popular conferencing platforms like Zoom, making it convenient for hassle-free daily meetings and remote work. You may need to manually select this speakerphone as the input and output device in your operating system settings.
- Clear Speaker Sound with Full-Duplex Audio: Built-in speaker delivers balanced, room-filling sound while full-duplex audio allows both sides to speak and be heard at the same time without cutting off. Enjoy smooth, natural conversations for presentations, team discussions, and client calls without interruptions.
- Compact, Portable & Home Office Ready: Sleek, lightweight, and easy to carry, this desktop conference speakerphone fits into any home office or small meeting room. Simple touch controls, stable design, and wide device compatibility make it a reliable upgrade over laptop microphones and speakers for everyday professional use.
Compare architecture choices with representative audio and the same workload. Record the region, model and provider versions, concurrency, warm or cold state, and measurement method alongside results. Evaluate first-audio time, full completion, barge-in behavior, cold starts and reconnects, concurrency under expected simultaneous sessions, and operational complexity—not a single isolated latency figure.
Test the failure modes before launch
- Cold start: Measure session setup and first-audio delay when no container is warm, not only in a steady-state run.
- Overload: Increase simultaneous sessions until tail latency or resource use becomes unacceptable. Confirm that WebSocket concurrency and per-container capacity reflect measured behavior.
- Server scale-up: If using Modal Servers, test the documented zero-to-one HTTP 503 case and ensure safe retry handling or an intentional warm-capacity policy.
- Disconnect and reconnect: Verify which state is restored from durable storage and whether a reconnect can resume or must create a new session.
- Interrupted speech: Speak again during generated playback and confirm that obsolete audio is canceled quickly rather than continuing over the new turn.
- Database saturation: Exercise connection pools across all scaled containers, including administrative and migration traffic, and observe what happens when the database cannot grant a connection.
Set up a realistic browser test
For local development, test from localhost or an HTTPS environment so the browser can request microphone access. A USB microphone is an optional way to make local voice testing more representative; it is not a backend requirement. Test with the browser, network conditions, and audio quality your users are likely to encounter, rather than inferring production performance from a single local conversation.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
