Skip to main content

Media Manager

1. Overview​

The MediaManager is the most data-intensive component in HoloMIT. It is responsible for orchestrating the transmission, reception, and synchronization of Volumetric Video, Point Clouds, Audio, and pre-recorded VOD (Video on Demand) content via the SFU (Selective Forwarding Unit) servers.

2. MediaManagerController (High-Level API)​

The MediaManagerController exposes the Unity-facing logic for interacting with media streams.

2.1. VOD (Video on Demand) Orchestration​

It acts as the primary interface for playing back volumetric recordings:

  • GetRecordedContents(): Retrieves a list of available recordings from the index server.
  • StartRecordedContent(...): Spawns a pre-recorded player into the live session.
  • StopRecordedContent(...): Despawns the recording.

2.2. Event Delegation​

It exposes actions like OnMediaReceivedEvent, OnStartRecordedContentEvent, and OnAudioReceivedEvent that the underlying renderers and audio decoders subscribe to.

3. MediaManagerWrapper (Low-Level Transport Layer)​

Because of the massive amount of data processed, the MediaManagerWrapper is highly complex and optimized.

3.1. Multi-Socket Architecture​

Unlike other managers, the Media Wrapper maintains multiple Socket.IO connections (MMListeningSocketWrapper list). This allows a single Unity client to listen to media streams distributed across multiple SFU servers simultaneously to balance the load.

3.2. RTCStreamPacket System​

Real-time media is transmitted as raw byte arrays over WebSockets to minimize overhead. The Wrapper parses incoming SocketIO packets into the RTCStreamPacket class.

These packets support various configurations based on the data type:

  • Base Media: Contains Timestamp, PlayerId, StreamType, and a primary Data buffer.
  • Dual Media (MediaTsDual / MediaInfoDual): Designed for RGB-D (RGB + Depth) streams, containing two separate binary buffers (Color and Depth) attached to a single timestamped frame.
  • Audio: Managed through the Audio socket event, providing dedicated low-latency delivery.

3.3. REST API Fallback​

Like the Session Manager, it uses the HttpManager for non-realtime operations, specifically for querying the Index Server database for VOD content (/api/nm/sfu/getContents).

Found something wrong or missing?

🐞 Report issue✏️ Missing information❌ Incorrect information🤔 Confusing information💡 Something else