COMET: Controllable Long-Term Motion Generation with Extended Joint Targets

Official PyTorch implementation of "Controllable Long-term Motion Generation with Extended Joint Targets". See the project webpage for examples and more details.

COMET builds on WANDR and includes its own inference, data preprocessing, training, evaluation, and GMM generation workflows.

Inference Quick Start

1. Install the environment

COMET supports Python >=3.10,<3.13. Install uv and create the environment:

pip install uv
uv sync

Video rendering uses aitviewer and requires OpenGL/Xvfb system dependencies. You can disable rendering with runtime.render=false and still save the motion as an .npz file.

2. Install SMPL-X

Register at the SMPL-X website, accept the license, and download SMPL-X v1.1. If models_smplx_v1_1.zip is in the repository root, install it at the default path with:

mkdir -p assets/body_models
unzip -o models_smplx_v1_1.zip -d assets/body_models
mv assets/body_models/models/smplx assets/body_models/
rm -rf assets/body_models/models

The resulting directory should contain assets/body_models/smplx/SMPLX_NEUTRAL.npz.

3. Download the pretrained checkpoint

mkdir -p outputs/checkpoints
wget https://github.com/CINEV/COMET/releases/download/v1.0.1/comet.ckpt \
  -O outputs/checkpoints/comet.ckpt

The default GMM priors and demo poses are already included under assets/; no additional inference data is required.

4. Run the bundled demos

Use the helper scripts to run every bundled example for a mode:

Mode	Included examples	Run all
Position goals	position_1, position_2, position_3	./scripts/run_position_demos.sh
Pose goals	pose_1, pose_2, pose_3	./scripts/run_pose_demos.sh
Motion in-betweening	mib_1, mib_4, mib_5	./scripts/run_mib_demos.sh
Stylization	Aeroplane, Akimbo, ArmsAboveHead, BentForward, Dinosaur	./scripts/run_style_demos.sh

For example, run all Position and Style demos with:

./scripts/run_position_demos.sh
./scripts/run_style_demos.sh

Each helper forwards additional Hydra overrides to every run. For example, this also exports the SMPL motion for all Pose demos:

./scripts/run_pose_demos.sh runtime.save_smpl=true

To run only one bundled example, select its config directly:

uv run python scripts/demo.py --config-name demo/position_2.yaml
uv run python scripts/demo.py --config-name demo/pose_3.yaml
uv run python scripts/demo_mib.py --config-name demo/mib_5.yaml

The individual stylization config uses Dinosaur by default. Select another style by overriding its bundled GMM path and use a distinct output name to avoid overwriting an earlier video:

uv run python scripts/demo.py --config-name demo/stylization.yaml \
  common.default_gmm_path=assets/priors/styles/Akimbo_FW_00_smplx.npz \
  runtime.out_name=style_Akimbo.mp4

Outputs are written to outputs/demos/. Edit the corresponding file in configs/demo/ to change targets, duration, output names, or other synthesis settings. Paths can also be overridden from the command line:

uv run python scripts/demo.py --config-name demo/position_1.yaml \
  common.model_path=/path/to/model.ckpt \
  common.default_gmm_path=/path/to/gmm.npz

Custom Inference Assets

Generate a custom GMM or extract demo poses

Generate a GMM prior from an AMASS-format motion:

uv run python scripts/compute_gmm.py \
  motion_path=/path/to/motion.npz \
  output_dir=deps/gmm \
  output_name=custom_motion.npz

For a style GMM, set output_dir=deps/style_gmm instead.

Extract a frame from an AMASS-format sequence for use as an initial or final pose:

uv run python scripts/get_pose.py \
  motion_path=deps/demo/sample_motion.npz \
  frame_idx=10 \
  output_path=deps/demo/custom_pose.npz

frame_idx is the direct frame index in the source sequence.

Data Preparation

Prepare AMASS and CIRCLE training data

AMASS, CIRCLE, and SMPL-X are subject to their respective licenses and are not distributed with this repository.

1. Download SMPL-X as described in the Quick Start.
2. Download SMPL-X neutral AMASS data from AMASS into data/raw/amass.
3. Download CIRCLE_movement.zip from CIRCLE and extract its contents into data/raw/circle. Keep each vr_data.json beside its sequence NPZ.

The raw data should have the following general structure. Dataset and sequence names vary; the paths below are representative entries from data/training_manifest.json.

data/
├── training_manifest.json
└── raw/
    ├── amass/
    │   ├── ACCAD/
    │   │   └── Female1General_c3d/
    │   │       └── A10_-_lie_to_crouch_stageii.npz
    │   └── KIT/
    │       └── 3/
    │           └── turn_left01_stageii.npz
    └── circle/
        └── s1/
            └── bathroom1_lights_1_bathroom1/
                └── 0/
                    └── 001_reaching/
                        ├── 001_reaching.npz
                        └── vr_data.json

AMASS and CIRCLE paths must match their relative path entries in the training manifest. Every CIRCLE sequence directory must contain both the motion NPZ and its corresponding vr_data.json.

4. Preprocess the datasets and calculate statistics:

uv run python scripts/preprocess_amass_circle.py
uv run python scripts/calculate_stats.py

The default outputs are:

data/processed/amass_circle.pth.tar
assets/statistics/generated/statistics_amass_circle_smplx_new.npy

Input, output, and filtering options can be changed in configs/preprocess/amass_circle.yaml and configs/calculate_stats.yaml, or with Hydra command-line overrides.

Train

Show training instructions

Training uses the preprocessed dataset and statistics file from the previous section:

uv run python scripts/train.py

Common settings can be overridden with Hydra:

uv run python scripts/train.py \
  'dataloader.load_files=[/path/to/amass_circle_final.pth.tar]' \
  statistics_path=/path/to/statistics_amass_circle_smplx_new.npy \
  trainer.devices=1 \
  batch_size=128

Resume an existing run with:

uv run python scripts/train.py resume_hash=<run_hash>

Checkpoints are written to checkpoints/, and Weights & Biases runs default to offline mode.

Evaluate

Show evaluation instructions

Evaluation requires the preprocessed dataset, statistics file, checkpoint, and GMM prior:

uv run python scripts/evaluate.py --config-name evaluate_single_goal.yaml
uv run python scripts/evaluate.py --config-name evaluate_multi_goal.yaml
uv run python scripts/evaluate.py --config-name evaluate_multi_joint.yaml

Results are written under outputs/evaluations/, with a summary.json in each Hydra run directory. Results can vary with the checkpoint, GMM prior, random seed, and evaluation configuration.

Citation

@InProceedings{Lee_2026_WACV,
    author    = {Lee, Eunjong and Kim, Eunhee and Hong, Sanghoon and Jung, Eunho and Kim, Jihoon},
    title     = {Controllable Long-term Motion Generation with Extended Joint Targets},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {March},
    year      = {2026},
    pages     = {5164-5173}
}