Depth Tracking¶
Depth tracking uses 3D surface data from depth sensors to match the real object's shape against the 3D model. It provides strong geometric constraints that complement silhouette and edge modalities.
Supported Depth Sources¶
| Source | Platform | Notes |
|---|---|---|
| ARKit LiDAR | iPhone Pro, iPad Pro | Requires AR Foundation |
| ARCore Depth | Android devices with ToF sensor | Requires AR Foundation |
| Intel RealSense | Windows (desktop) | Direct SDK integration |
| Recorded depth sequences | All platforms | For offline testing |
When to Use¶
Works well for:
- Textureless objects (depth doesn't need visual features)
- Symmetric objects (depth resolves ambiguities that silhouette can't)
- Large objects with distinct 3D shape
- Any scenario where a depth sensor is available
Struggles with:
- Shiny or reflective surfaces — metallic, glossy, or mirror-like materials scatter or deflect the infrared light used by depth sensors, producing missing or wildly incorrect depth readings
- Transparent objects — glass, clear plastic, and liquids let infrared pass through rather than reflecting it back to the sensor
- Very dark or light-absorbing surfaces — matte black materials can absorb too much infrared, returning weak or no depth signal
- Direct sunlight — outdoor or strong IR light sources overwhelm the depth sensor's infrared emitter, causing noisy or missing depth across the entire frame
- Very thin objects — insufficient surface area for a reliable depth measurement
- Objects far from the camera — depth resolution and accuracy decrease with distance (most sensors are effective within 0.2m - 5m)
Configuration¶
On the TrackedBody component:
- Enable the Depth Tracking checkbox
Depth tracking is a complementary modality — it is always used in addition to a contour modality (silhouette or edge) and cannot be used alone.
Note
A depth camera feed must be available. For AR Foundation, depth is provided automatically when the device supports it. For desktop, use RealSense or recorded depth sequences.
Quality Metrics¶
- Depth Quality — How closely the measured depth aligns with the model surface. Higher = better alignment.
- Quality ranges from 0.0 (no alignment) to 1.0 (perfect match)
- When both depth and contour modalities are active, the overall Tracking Quality uses the best of both
Tips¶
- Depth tracking significantly improves robustness when the background is cluttered
- For best results, ensure the object is within the depth sensor's effective range (typically 0.2m - 5m)
- Combine with silhouette or edge tracking — depth excels at position accuracy while contour modalities provide rotational precision