What is camera masking?  How do I use this functionality?  Why should I use this functionality?

What is Camera Masking?

Recent builds of Vicon iQ software have a powerful new feature called camera masking.  These masks are regions of a camera's sensor that are ignored from consideration in calibration and reconstruction.  In many cases, using camera masks can greatly improve calibration robustness and reconstruction quality.  In particular, in capture volumes where large amounts of spurious data are present  (e.g. incident reflections from floor/furniture, or opposing camera strobe rings) can see significant improvements in calibration robustness and reconstruction quality (both offline and real-time) by using iQ's camera masking capability.

How do I Create Camera Masks?

Mask information for the cameras in your Vicon system are stored in external files with a .msk extension.  Vicon iQ software has the ability to generate these .msk files, either automatically through a pipeline operation, or manually via simple drawing tools.  Once you have a .msk file generated, it is simply a matter of inserting the file path of the .msk file into the calibration and reconstruction parameters dialog boxes, and the camera mask information will be used.  Below are step-by-step instructions for creating a .msk file:

1. Begin by positioning and aiming your cameras for your desired capture volume.  Threshold the camera sensitivites and strobe intensity settings appropriately for your planned capture session.
2. Setup your Eclipse database, and create a new session.
3. Using Workstation software, perform a 3 second "background" trial capture.  Make sure that no subjects, calibration objects, or other unnecessary markers are present in any camera view.  The goal of this trial is to simply capture any spurious data in your capture volume -- stray reflections, opposing camera strobes, etc.  When the capture is complete, do not do any post processing in Workstation.
4. Launch Vicon iQ, and in Data Management mode, locate the background trial you just collected.  Double click on the .tvd icon () to open the .tvd file.  You may be prompted for a calibration file or system/camera type choices.  For best results during the masking process, use the drop-downs to select your Datastation and Camera types. 
5. After the file is open, you should see the 2D camera views.  Begin by clicking the "Raw Camera" button in the top-center portion of the screen.  You should now have a view similar to the image below.  At this point, camera masks can either be completed manually, or automatically via a pipleine operation:

• Manual Camera Mask Generation:
  1. With a raw camera view open, click the Masks button in the upper left portion of the screen.  Upon clicking this button, several tools will become available immediately below the Masks button.
  2. Using the camera channel buttons, navigate to a camera view that has some data you wish to mask out.  If you need to zoom or pan the image, hold down ALT and Left Mouse or Right Mouse button + drag.  Click on the shape that will best represent the shape of the area you wish to mask out -- either the ellipse or rectangle.  Your mouse is now a drawing tool, and by clicking and dragging over the desired mask region, a simple mask will be created. The figure below depicts a simple circular mask being created.

     

       
  3. Once created, the mask shape can be scaled using the blue selection handles, rotated using the red selection handles, and translated by selecting inside the mask and LMB + dragging.  Scale and position the mask to your liking.  In general, the mask should extend slightly beyond the region of interest, as seen in the figure above.
  4. To create another mask, click the desired shape tool, and repeat steps 2-3.  The shape creation tool can be locked by holding SHIFT key when selecting the shape tool
  5. Repeat steps 2-3 for all cameras, creating masks for any unwanted regions of the camera views.
  6. When finished, choose File >> Export, and pick a suitable file name and location for your .msk file.  By default, the .msk file will be given the same name as your trial, and will be stored in the same session folder as the trial.  Save out the file, and you are now ready to start using your .msk file in calibration and reconstruction.
• Automatic Camera Mask File Generation
  1. Click the Pipeline tab on the right side of the screen to bring up iQ's pipeline pane.  Clear the Pipeline, and then add the Autocreate Camera Masks step, and the Export Camera Masks step.
  2. Default parameters should be fine for most situations, but if you like, have a look at the pipeline options.  Frame Range to run the operation can be altered.  Additionally, the mask shape can be selected, and a flag can be set to remove any already existing masks.  In the advanced parameters section, the slack factor controls the amount that the calculated mask will shrink or expand beyond the edges of a given blob.  A value of 1.3 has shown to give good overall results.
  3. With the options set to your liking, run the Autocreate Camera Masks pipeline op.  Don't run the export step yet--it's a good idea to have a quick look at the calculated camera masks, and make sure the pipeline op generated good results.  When the operation is done, you should see red-outlined masks generated for all camera views.  
  4. If you need to edit any of the automatically generated masks, the procedure is very similar to the manual creation of the masks from above.  Click the "Masks" button to activate the mask toolbar.  If a mask need to be scaled/reshaped/moved, just select it, and grab the appropriate handles to adjust.  To delete a mask, select it and click the "Del" button in the masks tool area.
  5. Once the masks are to your liking, open up the pipeline options for the Export Camera Masks operation.  Choose a file name, and run the operation.  The default file path will be the session folder of the trial.  Run the operation, and your .msk file will be generated.

How do I use Camera Masks?

Once your .msk file has been generated, it can be used in Calibration, offline Reconstruction, and Real Time reconstruction.  Below are instructions on using the .msk file in each of these applications.

Using a Mask File during Workstation Volume Calibration:

Mask files can be used by recent builds of Workstation software (Build 125 or Higher) during calibration.  In situations where spurious data such as opposing strobes or incident reflections are unavoidable, camera masking can significantly improve calibration robustness.  To use the .msk file in Workstation, open the System>>Calibrate Cameras dialog box.  At the very bottom of the dialog box is a Camera Mask File entry box.  Use the browse button to locate your .msk file.  Collect static and dynamic calibration data.  The mask file will be input into the calibration algorithm.

 

Using a Mask File for Offline Reconstruction:

Vicon iQ (build 107 or Higher) is necessary to use .msk files for offline reconstruction.   Workstation software does not have this capability.  To use masks in iQ offline reconstruction, the .msk file needs to be loaded prior to reconstruction.   This can be done manually via File | Import and selecting your .msk file, or by running the pipeline operation Import Camera Masks.

Once your mask file has been loaded, open the options dialog for the Circle Fit, Reconstruct, Trajectory Fit pipeline operation.  Set the "Use Camera Masks" option to YES, and then run the operation.  Mask information will be used by the operation.

 

Using a Mask File for Real-Time Reconstruction:

Tarsus 2.3 (Build 39 or Higher) and, hence, iQ 1.5 (Build 119 or higher) are requred to apply camera masks in real time reconstruction.  In Real Time mode, select the appropriate Real Time Mode, and the appropriate Configuration Options from the 2 drop-down lists.  Click the Edit button under Configuration Options, and look for the Mask File parameter.  Use the browse button and locate the appropriate .msk file, and set any other pertinent parameters.  Close the parameters screen with OK.  Select appropriate Real Time Subjects, and then connect to Tarsus.

 

Where can I find More Information?

Full documentation for Vicon iQ 1.5 is available for download.