Skip to main content

HoloCapturer Pipeline

The HoloCapturer Pipeline is the core component within the HoloMIT SDK responsible for the entire volumetric video lifecycle. It handles everything from capturing data from physical sensors (like Kinect or Orbbec) to encoding, network transmission, decoding, and rendering the 3D representation in the scene.

HoloCapturer Pipeline Overview

Overview​

This system inherits from BasePipeline and heavily relies on multi-threading (via Thread Safe Queues and Workers) to ensure high performance and low latency during volumetric streaming.

Architecture & Workflow​

The pipeline operates in two distinct modes depending on the SourceType: Self (Producer) and Remote (Consumer).

1. Producer (Self Representation)​

When a user is the source of the volumetric data (e.g., they are being captured by local cameras), the pipeline acts as a Producer. The workflow is as follows:

  • Reader (HoloCapturerReader): Captures the raw frames from the configured cameras.
  • Preparer (HoloCapturerPreparer): Processes and reconstructs the Volumetric data for enconding and rendering.
  • Renderer (HoloCapturerRenderer): Takes the reconstructed data and renders it using specific shaders.
  • Encoder (HoloCapturerEncoder): Compresses the Volumetric data using hardware-accelerated codecs.
  • Writer (MediaWriterDual): Packages the encoded streams and transmits them over the network (e.g., via Cloud or LAN).

2. Consumer (Remote Representation)​

When a user is receiving volumetric data from other users in the session, the pipeline acts as a Consumer. The workflow is reversed:

  • Reader (MediaReaderDual): Receives the encoded streams from the network (e.g., via Cloud or LAN).
  • Decoder (HoloCapturerDecoder): Decompresses the Volumetric data using hardware-accelerated codecs.
  • Preparer (HoloCapturerPreparer): Reconstructs the decoded frames into Volumetric data.
  • Renderer (HoloCapturerRenderer): Takes the prepared data and renders using specific shaders.

Configuration and TrafoData​

The pipeline depends heavily on the configuration files to understand the camera layout and transformations (TrafoData). It reads the cameraconfig.json file during initialization to get the intrinsic and extrinsic parameters of the sensors.

For more details on setting up live cameras for this pipeline, please refer to the Live Cameras section.

Configuration Setup

Found something wrong or missing?

🐞 Report issue✏️ Missing information❌ Incorrect information🤔 Confusing information💡 Something else