User Representations Overview

User representations define how a user is rendered and perceived in a HoloMIT session: whether they use a 3D avatar, webcam feed, or a full volumetric capture pipeline. The SDK offers a modular and extensible framework to register, instantiate, and extend these representations at runtime, connecting them to different capture pipelines and UI metadata.

---

What Is a User Representation Type?

A User Representation Type defines the visual appearance and the capabilities of a user (e.g., audio, video, interaction). Examples include:

• A 3D avatar
• A 2D webcam window
• A full volumetric reconstruction (TSDF/Pointcloud)
• A non-visual participant (spectator/audio-only)

These representations are identified via UserRepresentationType, a strongly-typed ID system that can be built-in or user-defined.

---

Built-in Representations

Out-of-the-box, HoloMIT includes predefined types such as:

ID	Display Name	Interactions
AVATAR	3D Avatar	✅
WEBCAM	2D Video	✅
HOLOCAPTURER_DEPTH	HoloCapturer PointCloud	✅
HOLOCAPTURER_TSDF	HoloCapturer TSDF	✅
SPECTATOR	Audio only / No rendering	❌
NONE	No audio / No rendering	❌

You can list all registered types at runtime using:

UserRepresentationType.GetAll();

---

Core Components

The system is built around these main building blocks:

User Representation Type

A strongly typed identifier that declares what kind of representation a user is using. Unlike enums, these types can be registered dynamically, discovered at runtime, and used in tooling.

Handler Logic (IUserRepresentationHandler)

Each representation type is paired with a handler that implements the behavior:

• Enabling/disabling GameObjects
• Connecting pipelines (RGB-D, PM, etc.)
• Applying logic for local vs. remote users
• Adapting the scene for producers vs. consumers

Registry (RepresentationHandlerRegistry)

The central registry used to bind each UserRepresentationType to its corresponding IUserRepresentationHandler.

Setup Context (RepresentationSetupContext)

Context object with all information required to configure a player’s representation.

UI Integration (RepresentationUIRegistry)

Metadata for UI (icons, labels, descriptions) that allows the system to automatically populate dropdowns and selection widgets.

On top of that, you will typically have pipelines (derived from BasePipeline) which define how data (PointCloud, TSDF, audio, etc.) is captured and streamed. The RepresentationSetupContext.UserSourceType property links a representation with the underlying pipeline source type.

---

Typical Workflow

1. Choose or create a representation type Use existing ones (AVATAR, WEBCAM, HOLOCAPTURER_DEPTH …) or create custom ones via UserRepresentationType.CreateAndRegister.
2. Implement the handler Implement IUserRepresentationHandler.Setup to configure the scene (turn on/off objects, attach pipelines, etc.).
3. Register the handler Bind your handler in RepresentationHandlerRegistry.Register so the system can dispatch to it at runtime.
4. Register UI metadata (Optional) Add icons and labels in RepresentationUIRegistry.Register to expose it in dropdowns or menus.
5. Hook from UI/Lobby
   • Build dropdowns using RepresentationUIRegistry.GetDropdownOptions()
   • Resolve the type from the user’s selection (by display name or ID)
   • Retrieve the handler from RepresentationHandlerRegistry and call Setup(context).

---

Runtime Setup Flow

At runtime, when a player joins or changes their view:

1. The system checks the assigned UserRepresentationType.
2. It retrieves the handler via RepresentationHandlerRegistry.TryGetHandler(type, out handler).
3. If found, the handler receives a RepresentationSetupContext and initializes the representation.

The context includes:

• The player instance and manager.
• Locality (is this the local user or a remote one?).
• Role (is the user a producer or a consumer?).
• UserSourceType: Links the representation with the underlying pipeline source.

---

Benefits of This System

• 🧩 Modular – Add or replace representations without changing core logic.
• ♻️ Extensible – Add custom pipelines, rendering systems, or third-party solutions.
• 🧠 Consistent – Uses the same flow across built-in and external handlers.
• 🎨 UI-ready – Centralized registry for icons, previews, and tooltips.

---

See Also:

• UserRepresentationType
• IUserRepresentationHandler
• RepresentationSetupContext
• How to: Create a Custom User Representation

tip

If you're building your own capture/rendering pipeline, integrate it by creating a custom handler and registering it with your own UserRepresentationType.