Read Live Skeletons in C# (AR51.Net)

Receive the people the AR 51 system tracks: every person's 25 body joints in meters with per-joint confidence, hands when the person wears a hand-tracking headset, plus identity and timing.

Two ways to receive skeletons

Events — subscribe to SkeletonReceived; it fires for every skeleton while the client is started, and resumes by itself after a reconnect:

ar51.Skeletons.SkeletonReceived += (_, e) =>
    Console.WriteLine($"{e.Skeleton.Id}: hips at {e.Skeleton.GetPosition(Joint.MidHip)} m");
await ar51.StartAsync();

await foreach — read StreamAsync in a loop. Each loop opens its own stream; it ends when you cancel, call StopAsync, or the connection drops (start a new loop after an AR51Exception):

await foreach (Skeleton skeleton in ar51.Skeletons.StreamAsync(cancellationToken))
    Console.WriteLine($"{skeleton.Id}: hips at {skeleton.GetPosition(Joint.MidHip)} m");

Keep event handlers short

SkeletonReceived runs on the stream's background thread, one event per skeleton, in arrival order. For example, 20 people at 120 frames per second leaves all handlers together about 0.4 ms per skeleton. Copy what you need and do heavy work elsewhere — see hand-off patterns. The examples on these pages print to the console only to keep them short.

What a skeleton contains

Member	What it is
Id	Tracking id, stable while the person stays tracked
EntityId, EntityDisplayName	The registered person's identity and name — empty when not registered
DeviceId	The headset this person wears (matches a connected device's Id) — empty when none
Positions, GetPosition(joint)	Joint positions in meters, indexed by Joint
Confidences, GetConfidence(joint)	How sure the cameras are of each joint, from 0 (not seen) to 1
TryGetPosition(joint, out position)	The position, only if the joint was detected
LeftHand, RightHand	The hands' 23 joints each, or null without a hand-tracking headset
CaptureTime, ReceivedTime	When the cameras captured it (server clock, UTC) and when you received it (local clock)

Positions are in the AR 51 world: left-handed, X right, Y up, Z forward — the same axes as Unity. A Skeleton is immutable and safe to share across threads.

Undetected joints

A joint the cameras couldn't see is reported at (0, 0, 0) with confidence 0. Use TryGetPosition to tell it apart:

if (skeleton.TryGetPosition(Joint.RightWrist, out Vector3 wrist))
    Console.WriteLine($"right wrist at {wrist} m");
else
    Console.WriteLine("right wrist not seen");

The 25 body joints

Left and right are the person's own. The value is the index into Positions.

0 Nose	1 Neck	2 RightShoulder	3 RightElbow	4 RightWrist
5 LeftShoulder	6 LeftElbow	7 LeftWrist	8 MidHip	9 RightHip
10 RightKnee	11 RightAnkle	12 LeftHip	13 LeftKnee	14 LeftAnkle
15 RightEye	16 LeftEye	17 RightEar	18 LeftEar	19 LeftBigToe
20 LeftSmallToe	21 LeftHeel	22 RightBigToe	23 RightSmallToe	24 RightHeel

Hands

Hands come from a hand-tracking headset worn by the person, not from the cameras, so LeftHand and RightHand are null without one. Each hand has 23 joints, from the wrist to each fingertip — see HandJoint.

if (skeleton.RightHand is { } hand)
    Console.WriteLine($"right index fingertip at {hand.GetPosition(HandJoint.IndexTip)} m");

People leaving, and frames

• There's no "person left" event. When someone leaves, their skeletons simply stop arriving. Treat an Id as gone after a short timeout of your choice — for example 1 second without a skeleton (see Keep the latest skeleton per person).
• There's no frame object. SkeletonReceived fires once per person per frame. To group the people of one frame, group by CaptureTime.

Timing

CaptureTime uses the server's clock; ReceivedTime uses yours. Their difference is the true delay only when both clocks are synchronized, for example with NTP.

Complete sample

Read skeletons for a few seconds with await foreach, then print each person's name, a joint with its confidence, a fingertip if hands are tracked, and the headset they wear. Run it with dotnet run -- 10 for ten seconds.

Program.cs

// Read skeletons with await foreach: several people, joint confidence, hands, the person's name and headset.
// Usage: dotnet run -- [seconds]   (default 10)
using AR51.Net;
using System.Numerics;

var seconds = args.Length > 0 ? int.Parse(args[0]) : 10;
await using IAR51Client ar51 = AR51Client.Create();
await ar51.StartAsync();

using var stop = new CancellationTokenSource(TimeSpan.FromSeconds(seconds));
var latestByPerson = new Dictionary<string, Skeleton>();
var frames = 0;

try
{
    await foreach (Skeleton skeleton in ar51.Skeletons.StreamAsync(stop.Token))
    {
        latestByPerson[skeleton.Id] = skeleton;   // one skeleton per person per frame
        frames++;
    }
}
catch (OperationCanceledException)
{
    // the time is up
}

Console.WriteLine($"{frames} skeletons in {seconds} s from {latestByPerson.Count} people:");
foreach (Skeleton skeleton in latestByPerson.Values)
{
    var name = skeleton.EntityDisplayName.Length > 0 ? skeleton.EntityDisplayName : skeleton.Id;
    Console.WriteLine($"- {name}, captured {skeleton.CaptureTime:HH:mm:ss.fff} UTC");

    // TryGetPosition is false when the cameras did not see the joint (confidence 0).
    if (skeleton.TryGetPosition(Joint.Nose, out Vector3 nose))
        Console.WriteLine($"    nose {nose} m, confidence {skeleton.GetConfidence(Joint.Nose):F2}");
    else
        Console.WriteLine("    nose not seen");

    // Hands come from a hand-tracking headset worn by this person; null otherwise.
    if (skeleton.RightHand is { } hand)
        Console.WriteLine($"    right index fingertip {hand.GetPosition(HandJoint.IndexTip)} m");

    // DeviceId names the headset this person wears; match it with the connected devices.
    DeviceInfo? headset = ar51.Devices.Devices.FirstOrDefault(device => device.Id == skeleton.DeviceId);
    if (headset is not null)
        Console.WriteLine($"    wears {headset.Name} ({headset.BatteryPercent?.ToString() ?? "?"}% battery)");
}

View this sample on Bitbucket — clone the samples repository to run it.

Reference

ISkeletonClient · Skeleton · Joint · HandPose · HandJoint