Starting & Recovery¶
This guide explains how a body goes from a placed outline to tracking, what XRTracker does when a body is lost, and which settings control each step. The settings are in the Lifecycle section of the Tracked Body inspector unless the text says otherwise.
The Guide¶
Before its first start, a body is drawn at the guide, which is the pose you placed it at. The Initialization setting decides where the guide comes from.
| Initialization | Guide |
|---|---|
| Scene Position | The guide is the pose of the body in the scene, taken relative to the main camera at startup. |
| Viewpoint | The guide is the pose of the body relative to a viewpoint transform. You line the viewpoint up with the view that the real camera will have. |
The outline stays at the guide while the tracker looks for the object. Each step refines a copy of the guide against the current camera image, and the body starts from that copy only when it clears the start score on five steps in a row. The guide itself never walks onto whatever fits. With AR pose fusion this holds for stationary and non-stationary bodies alike.
The Guide Follows the Camera (AR)¶
With AR pose fusion on a live camera, the guide of a body that has no parent and starts from a Viewpoint stays in front of the camera until the body first starts. XRTracker moves the guide with the camera on every tracking step and draws it on every rendered frame, so the person lines the object up with an outline that moves with the device. If your application moves the guide, it takes up the new pose in front of the camera and follows from there.
After the first start, the guide stays where the body started. A body that starts from Scene Position stays where it was placed, and a recording keeps the body's own world transform.
Use the Current Pose as the Initial Viewpoint¶
When a body starts from a Viewpoint, you can take the viewpoint from a pose that the tracker found. In Play mode, while the body is tracking, the Tracked Body inspector shows the Use Current Pose as Initial Viewpoint button under the Viewpoint field. The button needs a Viewpoint to be assigned, and it keeps the pose the body is shown at, seen from the camera.
When Play mode ends, XRTracker moves the Viewpoint to the place from which the body starts at that pose. The change has an undo step, and the scene is marked as changed so that you can save it. A body that starts from the Viewpoint then starts at the pose you kept.
Search Before Start¶
An edge body without a parent searches for its own shape before its first start. The search is on by default through Search Before Start.
XRTracker draws outline views of the body around the guide. The views lie in a cone of 5 degrees around the guide's view, at three distances, with small turns in the image plane. XRTracker scores the views inside a window around the guide, which is the box of the guide's outline grown by 96 pixels, and it keeps the best match as the winner.
- When the guide stands within Reach of the winner (16 pixels by default), the search changes nothing. The body starts exactly as it does without the search.
- When the guide stands farther away, the tracker refines from the winner and not from the guide. The body starts only after the same winner was found on three searches in a row, and only when the winner reaches the Margin Bar and the Score Floor.
The search proposes a pose and nothing more. The start score still verifies the pose before the body starts, and the placed guide stays drawn where it is until the start.
The first tracking steps take longer while XRTracker draws the views, which takes about a quarter of a second with the default settings. A guide that is in reach starts as it always did during that time.
Note
The search covers an object that stands within about 90 pixels of the guide and within about 5 degrees of the guide's view. The tracker refines the last few degrees on its own after the start. A body can still start with a wrong tilt or depth, because the winner can stand a few degrees off and the tracker does not always correct that. You can switch Search Before Start off on a body that starts wrongly with it.
Search Settings¶
Turning on Search Before Start shows these settings.
| Setting | Default | Description |
|---|---|---|
| Search Before Start | On | It switches the search on or off for this body. A scene or prefab that saved the setting before 2.0 keeps its saved value. |
| Search Period (steps) | 2 | It sets the number of tracking steps between two searches. A value of 1 searches on every step. |
| Margin Bar | 1.4 | The score of the winner divided by the score of the best match that stands far from it must reach this value before a start is let through. A value of 0 sets no margin bar. |
| Score Floor | 0.6 | The share of outline points that find an edge of the same direction must reach this value before a start is let through. A value of 0 sets no floor. |
| Reach (pixels) | 16 | It sets how far the guide may stand from the winner and still count as in reach. A value of 0 treats every guide as out of reach. |
| Cone Half Angle (degrees) | 5 | It sets the half angle of the cone of views around the guide's view. |
| Cone Spacing (degrees) | 5 | It sets the spacing between neighbouring views of the cone. The turns in the image plane use the same step. |
| Turn Range (degrees) | 10 | It sets the range of the turns in the image plane. The turns are zero and plus and minus each step of the spacing out to this value. |
| Fine Half Angle (degrees) | 0 | It sets the half angle of an extra fine stage that scores the views near the winner at 2.5 degrees. A value of 0 switches the stage off. |
| Window Margin (pixels) | 96 | It sets how far beyond the box of the guide's outline the search looks. A value of 0 searches the whole image. |
| Distances | 3 | It sets the number of distances around the guide's own. One is the guide's own distance, three are 0.92, 1.0 and 1.09 times it, and five add 0.85 and 1.2 times it. |
| Scale Distances | Off | When it is on, XRTracker draws each view once and makes the other distances by scaling that drawing. This draws a third of the views and starts a body a little sooner, but it can start a body a little less well. |
A larger cone, a larger window and more distances find the object from further away, and they make every search cost more. For example, a cone half angle of 10 degrees, a window margin of 0 and 5 distances make a wide search.
Reading the Search from Code¶
var body = GetComponent<TrackedBody>();
// Unknown until a winner has been measured, then InReach or Far
StartSearchPath route = body.StartSearchRoute;
// How far the guide stands from the latest winner, in pixels
float pixels = body.StartSearchGuidePixels;
if (body.TryGetStartSearchReading(out StartSearchReading reading, out StartSearchVerdict verdict))
{
// The reading holds what the latest search found, and the verdict holds what the start gate decided on it
}
Start Rules¶
A body that tracks by the edge method starts when its start score reaches QualityToStartTracking on five frames in a row.
- The score is the share of sites that find a matching edge, divided by 0.70 and clipped to 1. The default start bar is 0.65.
- The confirm score equals the start score by default, so the last of the five frames needs no more than the start score. You can set a higher Custom Confirm Score on a body to make its start surer.
- Nothing else gates the start. A frame with no matched site cannot start a body, because the score reads zero.
A body that tracks by the silhouette method keeps the start it had in 1.0.3. It starts on its own bar of 0.3, it stops when its averaged score stays under 0.4, and a stop returns the tracker to the guide.
Start Score and Hold Confidence (AR)¶
Under AR pose fusion, an edge body reads two scores. The Start Score decides the start. The Hold Confidence decides whether the body keeps or gives up its lock after the start. The Runtime Status of the Tracked Body inspector shows each one with its bar.
| Readout | What it reads | Compared with | Property |
|---|---|---|---|
| Start Score | The coverage score, which is the score that a body without AR reads | QualityToStartTracking |
StartScore |
| Hold Confidence | The registration margin, mapped to 0 to 1 | QualityToStopTracking |
TrackingQuality |
The two scores let an application tell a body that has not started yet from a body that has lost its lock. The start reads the coverage score because the registration margin judges a soft camera picture too strictly, and on a phone camera it kept right poses from starting. A body without AR pose fusion and every silhouette body show a single Tracking Quality row.
Force Start and Reset¶
While the application runs, the Lifecycle section shows three buttons.
| Button | Description |
|---|---|
| Force Start | It starts tracking at once without the start score, the same as calling ForceStartTracking(). |
| Reset | It returns the body to the pose it was placed at, stops its tracking and starts its lifecycle over. The body keeps what it learned. |
| Reset and forget | It does the same as Reset, and the body also forgets what it learned. |
Recovery After a Loss¶
A body keeps the views it tracked well in. After a loss, it tries those views against the current camera image and starts again from the one that fits. A candidate becomes tracking only through the ordinary start, so a recovery never shows a pose that the tracker has not verified. While a candidate is tried, the body stays drawn where it stood, and it moves only when tracking starts again.
A body without AR pose fusion also tries the camera-relative poses it was accepted at before.
The number of kept views is the Saved View Recovery Capacity setting on the Tracked Body Manager. It is 8 by default, and a value of 0 switches recovery off.
A body that was lost keeps trying to recover for as long as your application lets it. XRTracker does not return the body to the pose it was placed at on its own. Your application decides when a recovery will not succeed and calls ResetTracking(), or ResetTracking(true) to make the body forget what it learned. A silhouette body has no recovery.
body.OnConfidenceChanged += (confidence, reason) =>
{
if (confidence == TrackingConfidence.Lost)
{
// The tracker keeps recovering, and the application decides when to start over
}
};
body.ResetTracking(); // back to the placed pose, keeps what the body learned
body.ResetTracking(true); // back to the placed pose, forgets what the body learned
Child Bodies¶
A child body never starts on its own. It is tracked exactly while its parent is tracked, and it stops with its parent. See Parent-Child Bodies.
Tips¶
- A guide that is placed close to where the real object appears starts sooner, because the search covers an object that stands within about 90 pixels of the guide.
- The Start Score is the first number to read under AR when a body does not start, because the Hold Confidence can read well while the Start Score stays under its bar.
- An object that reads low even when it is aligned needs a lower Custom Start Threshold. See Quality Metrics.
- A body that starts with a wrong tilt or depth does better with Search Before Start switched off.