Skip to main content

Voice Pipeline

The VoicePipeline class is the orchestrator for all audio data within the HoloMIT SDK. It inherits from BasePipeline and sets up the necessary worker threads and queues depending on whether the voice is local (microphone) or remote (network).

Pipeline Flow​

Local Voice (Producer)​

When the SourceType is Self, the pipeline captures local microphone input and prepares it for transmission.

  1. Queue Initialization: Allocates encoderQueue and senderQueue.
  2. Microphone Check: Verifies that a valid microphone is available in the user's data.
  3. Codec Loading: Instantiates either an NSpeexEncoder or OpusEncoder depending on the global Config.AudioCodec settings.
  4. VoiceReader: Initializes the VoiceReader worker to capture hardware microphone data and push it to the encoder queue.
  5. AudioWriter: Pushes the encoded packets from the sender queue out to the network orchestrator.

Remote Voice (Consumer)​

When the SourceType is Remote, the pipeline receives audio packets from the network and plays them back.

  1. Queue Initialization: Allocates decoderQueue and preparerQueue.
  2. AudioSource Configuration: Spawns an AudioSource component on the target GameObject, configuring it for loop and 3D spatialization.
  3. AudioReader: Initializes the AudioReader worker to pull packets from the network into the decoder queue.
  4. VoiceDecoder: Decompresses the network packets back into raw audio buffers.
  5. AudioPreparer: Prepares the raw buffers for playback.
  6. Playback: The pipeline intercepts Unity's audio DSP via the OnAudioFilterRead callback, extracting audio from the AudioPreparer and mixing it directly into the Unity audio stream.

Key Worker Scripts​

While orchestrated by VoicePipeline, the heavy lifting is done by specialized workers located in Assets\UserRepresentation\Voice\Scripts\Workers\:

  • VoiceReader: Interacts directly with microphone hardware to capture PCM data.
  • VoiceEncoder / VoiceDecoder: Wrappers for Opus and Speex implementations.
  • AudioPreparer: A bridge between the asynchronous decoder threads and Unity's synchronous audio DSP loop.

Found something wrong or missing?

🐞 Report issue✏️ Missing information❌ Incorrect information🤔 Confusing information💡 Something else