Camera Configuration WIP

To use the HoloCapturerPipeline, we must set up the cameras using HoloCalibration and the cameraconfig.json file according to our preferences.

Requirements

• A data provider is required. Currently, there are 3 options:
  • Kinect camera, Open Azure Kinect SDK, and Azure Kinect Viewer.
  • Orbbec camera, OrbbecSDK, and OrbbecViewer.
  • Pre-recorded dataset.
• HoloCalibration is required for multi-cam usage. For single cam, it is recommended but not necessary.

Integration

1. Open the Unity project in the explorer and open the file cameraconfig.json.

2. There are 3 main parameters to be aware of:
   • camera_type: the type of camera input you want to use.
   • device_config
     • input_path: the path to a pre-recorded dataset. Not relevant for live playback.
     • reconstruction_step: the type of data we want to use. Not relevant for live playback.
   • cameras: the section where the cameras in use are defined and their transformation matrices.

3. Change the camera_type to the type of camera you want to use. If you want to use the recorder features (such as playing pre-recorded datasets or generating one), use the keyword recorder. For live cameras, use either kinect or orbbec.

4. Change the input_path if you intend to play a pre-recorded dataset. You must indicate its path in this parameter with bars in this direction /.

5. Change the reconstruction_step if you intend to play a pre-recorded dataset according to the type of data present in the recording. It must match the HoloCapturerPipeline's representation type. The available options are depth, pm, and xyz.

6. Change the cameras section according to the cameras you have in use. Not relevant for pre-recorded datasets, unless it's an xyz type. It includes:

   • serial: the serial number of the camera in use. For xyz, it must be ply.
   • trafo: the transformation matrix of said camera. For single cam usage, the identity matrix will suffice. For multi-cam, please use HoloCalibration. For xyz, it is irrelevant.

Configuration parameters

There are many other configuration parameters. They are not necessary but can be very helpful to get the best possible volumetric video quality. Here are the most relevant ones:

• camera_config
  • fps: Only 15 and 30 are supported.
  • map_color_to_depth: For real-time cameras, choose 0 for color-to-depth transformation and 1 for the depth to color.
• device_config
  • loop_frames: For pre-recorded datasets, set to true to loop the recording.
• postprocessing
  • threshold_near: Minimum distance (z) at which capturing starts.
  • threshold_far: Maximum distance (z) at which capturing stops.
  • remove_floor: Removes points below floor_height. Default: false.
  • floor_height: Minimum height (y) at which capturing starts.
  • remove_roof: Removes points above roof_height. Default: false.
  • roof_height: Maximum height (y) at which capturing stops.

Use cases and good practices

Capturer.log

For more elaborate debugging, please consult the capturer.log file that will be generated after every run. The most recent one is capturer.log, while the others are from previous runs.

Pre-recorded paths

Make sure that the path is correct and that the bars are in this direction /.

Camera serial number

To ensure that the serial number of the cams is correct, consult its pertinent viewer. In the following image, you can observe that the Orbbec camera in use has a serial number of CL8MB3300BH.

Alternatively, run the app with an incorrect serial and consult the capturer.log, where the connected cameras serial numbers will appear.

Generate on build

When importing the HoloMIT SDK, the cameraconfig.json file is generated automatically. The same cannot be said for when generating a build of the project. Please make sure to copy-paste the file at the root of the application build.