Do 3D Large Language Models Really Understand 3D Spatial Relationships?

Xianzheng Ma* Tao Sun* Shuai Chen Yash Bhalgat Jindong Gu† Angel X Chang Iro Armeni Iro Laina Songyou Peng‡ Victor Adrian Prisacariu‡

International Conference on Learning Representations (ICLR) 2026

paper | project page | data | code | bibtex

We find that plain LLMs with zero 3D input can match full 3D-LLMs on existing benchmarks and propose a new benchmark and metric score for evaluating authentic 3D reasoning ability.

See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs

Yongchang Zhang* Xianzheng Ma* Tianyi Liu Guangquan Zhou Yang Chen

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026

paper | arxiv | code | bibtex

We propose an iterative, training-free, plug-and-play framework that supervises each reasoning step with visual evidence, improving reasoning accuracy while effectively reducing hallucination rates across different model scales.

Beyond Single Object: Learning 3D Relations with Large Language Models

Kohsuke Ide Ryousuke Yamada Yue Qiu Xianzheng Ma Yoshihiro Fukuhara Hirokatsu Kataoka Yutaka Satoh

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026 (Finding)

paper | bibtex

We study learning 3D spatial relations between multiple objects using Large Language Models.

Inferring Dynamic Physical Properties from Video Foundation Models

Guanqi Zhan* Xianzheng Ma* Weidi Xie Andrew Zisserman

arXiv preprint 2025

paper | arxiv | code | bibtex

We study the task of predicting dynamic physical properties (elasticity, viscosity, friction) from videos, via collecting a new dataset and contributing three methods using classical CV, video foundation models, and MLLMs.

Robotic Visual Instruction

Yanbang Li ZiYang Gong Haoyang Li Xiaoqi Huang Haolan Kang Guangpingbai Xianzheng Ma

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2025

paper | project page | code | bibtex

We propose a novel Human-Robot Interaction (HRI) paradigm designed to enable user-friendly, interpretable, and generalizable input methods for robot manipulation.

When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models

Xianzheng Ma* Brandon Smart* Yash Bhalgat* Shuai Chen Xinghui Li Jian Ding Jindong Gu Dave Zhenyu Chen Songyou Peng Jia-Wang Bian Philip H Torr Marc Pollefeys Matthias Nießner Ian D Reid Angel X. Chang Iro Laina Victor Adrian Prisacariu

IJCV under review

paper | arxiv | project page | code | bibtex

We survey the papers of 3D understanding, generation, and embodied agent tasks empowered by LLMs and other foundataion models (CLIP, SAM)

Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following

Ziyu Guo Renrui Zhang Xiangyang Zhu Yiwen Tang Xianzheng Ma Jiaming Han Kexin Chen Peng Gao Xianzhi Li Hongsheng Li Pheng-Ann Heng

paper | arxiv | project page | code | bibtex

We propose a 3D multi-modal model for general 3D learning, Point-Bind, and the first 3D large language model, Point-LLM

Matching Is Not Enough: A Two-Stage Framework for Category-Agnostic Pose Estimation

Min Shi Zihao Huang Xianzheng Ma Xiaowei Hu Zhiguo Cao

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023 Highlight

paper | arxiv | project page | code | bibtex

We propose a two-stage framework--CapeFormer for category-agnostic pose estimation.

Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation

Yulu Gan Xianzheng Ma Yan Bai Yihang Lou Renrui Zhang Nian Shi Lin Luo

AAAI Conference on Artificial Intelligence (AAAI) 2023 Outstanding Student Paper

paper | arxiv | project page | code | bibtex

We offer an alternative and new solution for continual test-time adapation by learning visual domain prompt.

Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding

Xianzheng Ma Zhixiang Wang Yacheng Zhan Yinqiang Zheng Zheng Wang Dengxin Dai Chia-Wen Lin

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022 Oral

paper | arxiv | project page | code | bibtex

We alleviate the domain gap caused by mixed fog influence and style variation without labels.

REMOTE: Reinforced Motion Transformation Network for Semi-supervised 2D Pose Estimation in Videos

Xianzheng Ma Hossein Rahmani Zhipeng Fan Bin Yang Jun Chen Jun Liu

AAAI Conference on Artificial Intelligence (AAAI) 2022

paper | bibtex

We offer an reinforcement learning based method to ultilize the temporal information in videos to train a robust pose estimator.

Rainy WCity: A Real Rainfall Dataset with Diverse Conditions for Semantic Driving Scene Understanding

Xian Zhong* Xianzheng Ma* Shidong Tu Kui Jiang Wenxin Huang Zheng Wang (* means equal contribution)

International Joint Conference on Artificial Intelligence (IJCAI) 2022

paper | arxiv | bibtex

We propose a real-world rainy driving dataset for semantic segmentation and devise an unsupervised joint optimization framework based on contrastive learning.