Skip to main content

Voice Pipeline

The Voice Pipeline is the core component within the HoloMIT SDK responsible for the entire voice and audio streaming lifecycle. It is contained within its own dedicated library, Voice.DLL, and handles everything from capturing local microphone input to encoding, network transmission, decoding, and spatialized audio playback in the scene.

Voice Pipeline Overview

Overview​

This system inherits from BasePipeline and automatically orchestrates multi-threaded workers to ensure high performance and low latency during real-time voice communication. It abstracts the complexity of audio streaming so you can focus on the user experience.

Architecture & Workflow​

The pipeline operates in two distinct modes depending on the SourceType: Self (Producer) and Remote (Consumer).

1. Producer (Self Representation)​

When a user is the source of the audio (e.g., speaking into their local microphone), the pipeline acts as a Producer. The workflow is as follows:

  • Reader (VoiceReader): Captures raw PCM data from the hardware microphone configured in the user's data profile.
  • Encoder (NSpeexEncoder / OpusEncoder): Compresses the audio data. The exact codec worker is dynamically instantiated based on your project's global Config.AudioCodec settings.
  • Writer (AudioWriter): Packages the encoded audio streams and transmits them over the network.

2. Consumer (Remote Representation)​

When a user is receiving audio from other users in the session, the pipeline acts as a Consumer. The workflow is reversed:

  • Reader (AudioReader): Receives the encoded audio streams from the network.
  • Decoder (VoiceDecoder): Decompresses the network packets back into raw audio buffers.
  • Preparer (AudioPreparer): Prepares the raw audio buffers for playback synchronization.
  • Renderer (VoicePipeline): Intercepts Unity's audio DSP (via OnAudioFilterRead) to mix the prepared audio directly into the scene.
tip

Automatic 3D Spatialization As a developer, you don't need to manually setup audio playback components. When a Remote Voice Pipeline is initialized, it automatically adds an AudioSource to the GameObject and completely configures it for 3D spatialization (spatialBlend = 1.0f, minimum and maximum distances, and loop settings). The SDK seamlessly takes care of the mixing!

Found something wrong or missing?

🐞 Report issue✏️ Missing information❌ Incorrect information🤔 Confusing information💡 Something else