Avatar System

Skeletal animation, inverse kinematics, blend shapes, lip-sync, LOD management, GPU skinning, and performance rating for Aether avatars.

Avatars are the most complex visual objects in Aether. They combine real-time inverse kinematics, GPU-accelerated skinning, facial blend shapes, lip-sync, and distance-based LOD to deliver expressive, performant characters in VR.

The aether-avatar crate provides the full avatar pipeline, from VR tracking input to final GPU dispatch.

Key Concepts

Aether avatars are built on several interconnected subsystems:

• Skeletal animation -- A bone hierarchy drives mesh deformation through a pose evaluation pipeline.
• Inverse kinematics (IK) -- Sparse VR tracking data (headset, controllers, optional body trackers) is expanded into a full-body skeleton pose.
• Blend shapes -- Morph targets on the mesh enable facial expressions and viseme-driven lip-sync.
• GPU skinning -- Compute shaders transform vertices on the GPU using bone matrix palettes.
• LOD management -- Distance-based level-of-detail reduces rendering cost for distant avatars.
• Performance rating -- Polygon, bone, and material budgets enforce quality tiers so user-created avatars do not destroy frame rates.

Architecture

The avatar pipeline flows from tracking input through IK solving, skeleton evaluation, and finally GPU dispatch:

TrackingFrame -> IK Solver -> SkeletonPose -> BoneMatrixPalette -> GPU Skinning
VisemeWeights -> BlendShapeWeights -> GPU Blend Shape Dispatch
Camera Distance -> LOD Tier Selection -> Shader Permutation
AvatarMeshStats -> Performance Rating -> Allow / Block

Each stage is a pure function or lightweight struct with no external I/O, making the pipeline testable and composable.

Skeleton and Bone Types

The skeleton is a flat array of bones with parent indices forming a tree:

use aether_avatar::{Bone, Skeleton, IkTarget};

let bone = Bone {
    position: [0.0, 1.0, 0.0],
    rotation: [0.0, 0.0, 0.0, 1.0], // identity quaternion
    length: 0.3,
    parent: None, // root bone
};

let skeleton = Skeleton {
    bones: vec![bone],
    bone_names: vec!["hips".to_string()],
};

IkTarget represents a desired end-effector position with an optional rotation constraint:

let hand_target = IkTarget {
    position: [0.5, 1.2, 0.3],
    rotation: Some([0.0, 0.0, 0.0, 1.0]),
};

FABRIK Inverse Kinematics

Aether uses the FABRIK (Forward And Backward Reaching Inverse Kinematics) algorithm for bone chain solving. FABRIK iteratively adjusts bone positions through forward and backward passes until the end effector converges on the target.

Forward pass: Starting from the end effector, move it to the target position, then walk backward up the chain. Each parent joint is placed at child_position + direction_to_parent * bone_length.

Backward pass: Fix the root bone at its original position, then walk forward down the chain. Each child is placed at parent_position + direction_to_child * bone_length.

use aether_avatar::{FabrikSolver, FabrikConfig, IkTarget};

let config = FabrikConfig {
    max_iterations: 10,
    tolerance: 0.001,
};

let solver = FabrikSolver::new(config);
let target = IkTarget {
    position: [1.0, 0.5, 0.0],
    rotation: None,
};

// Solve a bone chain toward the target
let result = solver.solve(&mut chain, &target);

3-Point and 6-Point IK

Aether supports two tracking configurations:

3-point IK (head + 2 hands): The spine direction is estimated from head position and a default hip offset. Shoulders are derived from head rotation and arm length. Each arm is solved as a 2-bone FABRIK chain.

6-point IK (head + 2 hands + hip + 2 feet): The hip position comes directly from a tracker. The spine is solved between hip and head. Each leg is solved as a 2-bone FABRIK chain (thigh + shin).

use aether_avatar::{solve_three_point, solve_six_point, IkResult};

// 3-point: headset + two controllers
let result: IkResult = solve_three_point(&skeleton, &head, &left_hand, &right_hand);

// 6-point: full body tracking
let result: IkResult = solve_six_point(
    &skeleton, &head, &left_hand, &right_hand, &hip, &left_foot, &right_foot,
);

Joint Constraints

Per-joint angular limits prevent unnatural poses:

use aether_avatar::{JointConstraint, ConstraintSet};

let elbow_constraint = JointConstraint {
    min_angle: [0.0, -0.1, -0.1],
    max_angle: [2.6, 0.1, 0.1], // radians
    twist_limit: 0.2,
};

let constraints = ConstraintSet {
    constraints: vec![(3, elbow_constraint)], // bone index 3
};

Constraints are applied after each FABRIK pass by clamping computed angles to the allowed range per axis. Twist is separated and clamped independently.

T-Pose Calibration

Before IK begins, the system calibrates skeleton proportions from a reference T-pose:

use aether_avatar::{calibrate_from_tpose, CalibrationData};

let calibration: CalibrationData = calibrate_from_tpose(&tracking_frame, &reference_skeleton);
// calibration.arm_span, calibration.height are computed
// Bone lengths are scaled proportionally from the reference skeleton

Blend Shapes and Lip-Sync

Blend shapes (morph targets) drive facial expressions. Each target defines per-vertex position and normal deltas:

use aether_avatar::{BlendShapeTarget, BlendShapeSet, BlendShapeWeights};

let smile = BlendShapeTarget {
    name: "smile".to_string(),
    vertex_deltas: vec![], // position + normal deltas per vertex
};

let face = BlendShapeSet {
    targets: vec![smile],
    max_active: 8,
};

let mut weights = BlendShapeWeights::new();
weights.set("smile", 0.75); // 0.0 to 1.0

The VisemeEvaluator converts audio-driven LipSyncFrame data into blend shape weights, smoothing transitions between consecutive visemes:

use aether_avatar::{VisemeEvaluator, VisemeWeights};

let evaluator = VisemeEvaluator::new(0.05); // 50ms interpolation time
let weights: VisemeWeights = evaluator.evaluate(&lip_sync_frame);

GPU Skinning

Aether supports both Linear Blend Skinning and Dual Quaternion Skinning via compute shaders:

use aether_avatar::{GpuSkinningConfig, SkinningMethod, BoneMatrixPalette, SkinningDispatch};

let config = GpuSkinningConfig {
    workgroup_size: 64,
    max_bones: 256,
    max_vertices: 65536,
    method: SkinningMethod::DualQuaternion,
};

// After skeleton evaluation, upload the bone matrix palette to the GPU
let palette = BoneMatrixPalette::from_pose(&skeleton_pose);
let dispatch = SkinningDispatch::new(&config, vertex_count, &palette);

Each SkinVertex carries up to 4 bone indices and corresponding weights for smooth deformation across joints.

LOD Management

Distance-based LOD tiers reduce rendering cost when many avatars are visible:

Tier	Distance	Description
FullMesh	< 5 m	Full geometry, all blend shapes active
Simplified	5 -- 30 m	Reduced polygon mesh, limited blend shapes
Billboard	30 -- 100 m	Camera-facing billboard with baked animation
Dot	> 100 m	Colored dot indicator

Hysteresis prevents rapid tier switching when an avatar hovers near a distance threshold:

use aether_avatar::{AvatarLodConfig, AvatarLodTier, select_lod_tier};

let config = AvatarLodConfig {
    thresholds: [5.0, 30.0, 100.0],
    hysteresis: 2.0,        // 2m buffer zone
    transition_duration: 0.3, // 300ms blend
};

let tier: AvatarLodTier = select_lod_tier(distance, &previous_tier, &config);

Performance Rating

Every avatar is validated against polygon, bone, and material budgets before it can be used in a world:

Tier	Max Polygons	Description
S	10,000	Excellent -- runs on all hardware
A	25,000	Good -- suitable for most worlds
B	50,000	Medium -- may cause issues in crowded scenes
C	75,000	Poor -- world owners may block

use aether_avatar::{validate_avatar, PerformanceBudgetTable, AvatarRatingBucket};

let budget = PerformanceBudgetTable::default();
let rating: AvatarRatingBucket = validate_avatar(&mesh_stats, &budget);

// World owners set a minimum rating
if rating < world_minimum_rating {
    // Avatar is blocked from this world
}

World owners configure a minimum performance rating. Avatars that exceed the budget for that tier are blocked from entering.

Avatar Formats

Aether supports VRM format and a custom binary format optimized for the platform. The import pipeline validates format compliance, extracts skeleton data, and runs the performance rating check before the avatar is made available.

Animation State Machine

The AnimationStateMachine evaluates blend trees and state transitions, producing weighted (state, weight) pairs for the animation mixer. Transitions use configurable BlendCurve types (linear, ease-in-out, cubic) to smoothly interpolate between animation states. The state machine maintains current state plus active blend transitions and advances via update(dt, input).