This repository hosts the official release plan and forthcoming code/data release for our ICLR 2026 paper.

Paper | Project Page

Our framework automatically constructs both goal states and reward functions for diverse interaction tasks in reinforcement learning. By leveraging VLM guidance, the learned motion policy drives physics-based characters to perform coherent, long-horizon interactions with static and dynamic objects, producing natural and task-consistent behaviors.

Planned release contents:
☐ Processed 3D assets (object mesh files, URDFs, and point-cloud annotations)
☐ Scene layouts and rendered top-view scene images
☐ Task text instructions and cleaned VLM-generated plans
☐ Prompt templates

Planned release contents:
☐ Training and inference code for single-task motion policies (including environment/task implementations)
☐ Training and inference code for multi-task motion policies (including environment/task implementations)

If you use this work, please cite:

@inproceedings{deng2026humanobject,
    title={Human-Object Interaction via Automatically Designed {VLM}-Guided Motion Policy},
    author={Zekai Deng and Ye Shi and Kaiyang Ji and Lan Xu and Shaoli Huang and Jingya Wang},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026}
}