Yue Hu

Pragmatic Communication in Multi-Agent Collaborative Perception [TPAMI 2026]

Sun, 15 Mar 2026 00:00:00 +0000

Rate-Distortion Optimized Communication for Collaborative Perception [ICLR 2026]

Sat, 31 Jan 2026 00:00:00 +0000

Unveiling the Impact of Data and Model Scaling on High-Level Control for Humanoid Robots [ICRA 2026]

Sat, 31 Jan 2026 00:00:00 +0000

BeliefMapNav: 3D voxel-based belief map for zero-shot object navigation [NeurIPS 2025]

Thu, 18 Sep 2025 00:00:00 +0000

Imaginative world modeling with scene graphs for embodied agent navigation [ICCV 2025 Demo Track]

Thu, 18 Sep 2025 00:00:00 +0000

NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception [NeurIPS 2025]

Thu, 18 Sep 2025 00:00:00 +0000

Self-Evolving Multi-Agent Collaboration Networks for Software Development [ICLR 2025]

Sat, 01 Mar 2025 00:00:00 +0000

An Extensible Framework for Open Heterogeneous Collaborative Perception [ICLR 2024]

Mon, 01 Jan 2024 00:00:00 +0000

Communication-Efficient Collaborative Perception via Information Filling with Codebook [CVPR 2024]

Mon, 01 Jan 2024 00:00:00 +0000

Towards Collaborative Autonomous Driving Simulation Platform and End-to-End System [TPAMI 2025]

Mon, 01 Jan 2024 00:00:00 +0000

Collaboration Helps Camera Overtake LiDAR in 3D Detection [CVPR 2023]

Thu, 22 Jun 2023 00:00:00 +0000

Aerial monocular 3d object detection [RAL 2023]

Mon, 16 Jan 2023 00:00:00 +0000

Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow [NeurIPS 2023]

Sun, 01 Jan 2023 00:00:00 +0000

Category Query Learning for Human-Object Interaction Classification [CVPR 2023]

Sun, 01 Jan 2023 00:00:00 +0000

Where2comm is reported by 机器之心

Wed, 12 Oct 2022 22:17:11 +0800

Where2comm is reported by 机器之心: 《将通信带宽降低至十万分之一，NeurIPS 2022论文提出新一代协作感知方法》!

Opening positions

Wed, 12 Oct 2022 17:22:11 +0800

一、平台介绍

上海人工智能实验室联合上海交通大学面向海内外公开招聘机器学习、计算机视觉的博士后研究人员若干名，推进群体智能相关领域的研究，助力无人集群、车路协同、智慧医疗等应用的发展与落地。优厚待遇国际领先。欢迎感兴趣的研究人员应聘。

上海人工智能实验室是我国人工智能领域的新型科研机构，开展战略性、原创性、前瞻性的科学研究与技术攻关，突破人工智能的重要基础理论和关键核心技术，打造“突破型、引领型、平台型”一体化的大型综合性研究基地，支撑我国人工智能产业实现跨越式发展，目标建成国际一流的人工智能实验室，成为享誉全球的人工智能原创理论和技术的策源地。实验室网址：www.shlab.org.cn

上海交通大学是我国历史最悠久的高等学府之一，教育部直属、上海市共建、中央直管的全国重点大学，位列国家“双一流”、“985 工程”、“211 工程”，为九校联盟、环太平洋大学联盟、21 世纪学术联盟、中国大学校长联谊会、国际应用科技开发协作网、新工科教育国际联盟成员，入选“珠峰计划”、“强基计划”、“111 计划”、“2011 计划”、卓越工程师教育培养计划等。学校网址： www.sjtu.edu.cn/

二、岗位职责

研究业界前沿算法，包括但不限于多智能体系统、图机器学习、联邦学习、3D 感知、数据/模型压缩；
负责开源项目的开发和维护，开发新的算法框架，实现 SOTA 算法，提升用户体验和项目影响力；
协助指导博士研究生及实习生的研究工作；
参与申报各类科研项目。

三、申请条件

计算机、人工智能相关专业博士学位；
深入了解机器学习和计算机视觉的一个或多个方向，以一作身份在顶会或顶刊发表过高质量论文；
熟练掌握 Python 和 PyTorch，有良好的工程实现能力；
熟悉 C++、CUDA、ROS，在技术社区有影响力，曾参与贡献知名开源项目者优先

四、团队简介

王延峰教授，博士生导师，现任上海人工智能实验室主任助理、全球高校人工智能学术联盟秘书长。国家发改委人工智能专家委员会委员、科技部科技创新 2030“新一代人工智能”重大项目指南专家组成员。主要研究方向为人工智能、智慧医疗、新兴信息技术商业应用。

陈思衡，博士生导师，现任上海交通大学电子信息与电气工程学院未来媒体网络协同创新中心副教授，上海人工智能实验室双聘青年科学家，美国卡内基梅隆大学博士，入选国家级人才计划青年项目。研究方向为群体智能，图机器学习，联邦学习，自动驾驶。

五、应聘方式

将详细的个人简历，包括教育背景、工作经历、成果情况及联系方式发送至邮箱：sihengc@sjtu.edu.cn ,邮件标题注明“博士后应聘_姓名”。

Latency-aware collaborative perception [ECCV 2022]

Sat, 01 Oct 2022 00:00:00 +0000

News

Sat, 01 Oct 2022 00:00:00 +0000

[Mar. 2026] PragComm is accepted by TPAMI 2026

[Feb. 2026] Humanoid-Union is accepted by ICRA 2026

[Jan. 2026] RDcomm is accepted by ICLR 2026

[Sep. 2025] BeliefMapNav and NegoCollab is accepted by NeurIPS 2025

[Aug. 2025] SGImagineNav is accepted by ICCV Demo Track 2025

[Apr. 2025] V2XVerse is accepted by TPAMI 2025

[Mar. 2025] MAS workshop proposal is accepted by ICML 2025

[Feb. 2025] EvoMAC is accepted by ICLR 2025

[Feb. 2024] CodeFilling is accepted by CVPR 2024

[Jan. 2024] HEAL is accepted by ICLR 2024

[Nov. 2023] CoBEVFlow is accepted by NeurIPS 2023

[Apr. 2023] CoCa3D is reported by 机器之心

[Mar. 2023] Two papers are accepted to CVPR 2023

[Jan. 2023] Aerial Monocular 3D Object Detection is accepted to IEEE Robotics and Automation Letters

[Oct. 2022] Where2comm is reported by 机器之心 and 将门创投

[Sep. 2022] One paper is accepted to NeurIPS 2022 Spotlight(Top5%)

Where2comm: Communication-efficient collaborative perception via spatial confidence maps [NeurIPS 2022 Spotlight]

Thu, 15 Sep 2022 00:00:00 +0000

We are hiring !

Wed, 04 May 2022 19:53:28 +0800

MediaBrain陈思衡老师招【2023/24级】直博/直硕生啦！

陈思衡老师是未来媒体网络协同创新中心的长聘轨副教授，2019年国家重大人才工程青年项目入选者！陈老师在CMU获得博士学位，博士后完成后，在Uber Advanced Technologies Group、三菱电机实验室MERL担任Research Scientist，至今在TPAMI、TIP、NeurIPS（oral）、CVPR （oral）、AAAI （oral）、ICLR上发表了50余篇论文，Google Scholar引用2000余次，获得过IEEE信号处理协会最佳年轻作者论文奖！学界+业界影响力Max！

陈老师与海外各高校包括（CMU、Oxford、…）以及国内外各企业（Uber、滴滴、…）都有着很强的connection，可以推荐你到世界顶尖的平台学习交流！

如果你数学与编程功底扎实，踏实好学，对计算机视觉与机器学习有一定的了解；
如果你想做【一流的学术】工作、在国际顶级期刊和会议发表【高水平 & 有学术影响力】的论文；
如果你对图信号处理、图神经网络、无人系统等领域感兴趣，想要探索计算机视觉、机器学习和自动驾驶的问题；
如果对科研有热情，希望能够加入一个积极向上、开放包容、有激情有理想地探索科学的、顶级的科研氛围与团队；那么欢迎你加入我们！

加入我们，你会接受系统的科研工作练习，陈老师与学长学姐带你快速适应科研节奏，参与到我们正在开展的工作中，有机会在一年做出优质的科研工作并发表顶会论文！

请有意向的同学发送简历至：sihengc[AT]sjtu[DOT]edu[DOT]cn

也欢迎朋友们帮忙转发，期待大家共同进步！

Coperception-UAV Dataset

Tue, 19 Apr 2022 20:07:44 +0800

Table of Contents

About Coperception-UAV Dataset

Coperception-UAV is the first comprehensive dataset for UAV-based collaborative perception.

A UAV swarm has the potential to distribute tasks and achieve better, faster, and more robust performances than a single UAV. To realize this, we need to integrate collaboration ability into the entire pipeline, including perception, planning, control. Among those tasks, collaborative perception enables holistic scene understanding from multiple perspectives via the collaboration of multiple UAVs, which could fundamentally resolve the occlusion issue and the long-range issue in the traditional single-agent perception. Recently, planning and control of a UAV swarm have been intensively studied; however, the collaborative perception remains under-explored due to the lack of a comprehensive dataset. This work aims to fill this gap and proposes a collaborative perception dataset for UAV swarm.

Based on the co-simulation platform of AirSim and CARLA, our dataset consists of 131.9k synchronous images collected from 5 coordinated UAVs flying at 3 altitudes over 3 simulated towns with 2 swarm formations. Each image is fully annotated with the pixel-wise semantic segmentation labels and 2D/3D bounding boxes of vehicles. We further build a benchmark on the proposed dataset by evaluating a variety of related multi-agent collaborative methods on multiple perception tasks, including object detection, semantic segmentation, and bird’seye-view (BEV) semantic segmentation.

Simulation Setting

Our proposed dataset is collected by the co-simulation of CARLA and AirSim. We use CARLA to generate complex simulation scenes and traffic flow; and use AirSim to simulate UAV swarm flying in the scene. The flight route of UAVs is controlled by AirSim and sample data are collected randomly at about 4-second intervals.

Swarm arrangement

The UAV swarm moves and executes tasks in the three-dimensional space, where the situation could be much more complex than those of vehicles or roadside units. In the dataset, two main factors are taken into consideration that may affect the perception and collaboration patterns of UAV swarms: flight formation and altitude. Each UAV swarm consists of 5 UAVs. We arrange two types of formation modes for a UAV swarm: discipline mode, where all 5 UAVs keeps a consistent and relatively static array, and dynamic mode, where each UAV navigates independently in the scene. The former simulates the situation where the swarm of UAVs is executing a same specific task such as exploring an unknown area, search and rescue; while the latter simulates the monitoring and patrolling tasks in the city.

Sensor Setup

In the UAV swarm, Each UAV is equipped with 5 RGB cameras in 5 directions and 5 semantic cameras collecting semantic ground truth for RGB cameras.

90° horizontal FoV
1 bird’s eye view camera and 4 cameras facing forward, backward, right, and left with a pitch degree of −45°
image size: 800x450 pixels

Data

Fully-annotated data are provided in the dataset, including synchronous images with pixel-wise semantic labels, 2D & 3D bounding boxes of vehicles, and BEV semantic map.

Camera data We collect synchronous images from all cameras on 5 UAVs, which is 25 images in a sample. In total, 123.8K images are collected for the discipline swarm mode and 8.1K for the dynamic swarm mode. We provide semantic label for each image.

Bounding boxes 3D bounding boxes of vehicles are recorded at the same moment with images, including location (x, y, z), rotation (w, x, y, z in quaternion) in the global coordinate and their length, width and height. To specifically address the occlusion issue, we also provide a binary label for the occlusion status of each bounding box.

BEV semantic label We provide BEV segmentation labels of four categories: roadway, building, vehicle, and others, which are the key elements to construct the layout of a city and foreground objects. The resolution of the BEV map is 0.25m×0.25m.

Usage

The dataset is organized in a similar way with the widelyused autonomous driving dataset, nuScenes; so it can be used directly with the well-established nuScenes-devkit.

Citation

To Be Done

End-to-end human object interaction detection with hoi transformer [CVPR 2021]

Fri, 01 Jan 2021 00:00:00 +0000

Collaborative motion prediction via neural motion message passing [CVPR 2020 Oral]

Wed, 01 Jan 2020 00:00:00 +0000

Neural message passing for visual relationship detection [ICML 2019 Workshop Spotlight]

Sat, 01 Jun 2019 00:00:00 +0000

Slides

Tue, 05 Feb 2019 00:00:00 +0000

Create slides in Markdown with Wowchemy

Wowchemy | Documentation

Features

Efficiently write slides in Markdown
3-in-1: Create, Present, and Publish your slides
Supports speaker notes
Mobile friendly slides

Controls

Next: Right Arrow or Space
Previous: Left Arrow
Start: Home
Finish: End
Overview: Esc
Speaker notes: S
Fullscreen: F
Zoom: Alt + Click
PDF Export

Code Highlighting

Inline code: variable

Code block:

porridge = "blueberry"
if porridge == "blueberry":
    print("Eating...")

Math

In-line math: $x + y = z$

Block math:

$$ f\left( x \right) = ;\frac{{2\left( {x + 4} \right)\left( {x - 4} \right)}}{{\left( {x + 4} \right)\left( {x + 1} \right)}} $$

Fragments

Make content appear incrementally

{{% fragment %}} One {{% /fragment %}}
{{% fragment %}} **Two** {{% /fragment %}}
{{% fragment %}} Three {{% /fragment %}}

Press Space to play!

One Two Three

A fragment can accept two optional parameters:

class: use a custom style (requires definition in custom CSS)
weight: sets the order in which a fragment appears

Speaker Notes

Add speaker notes to your presentation

{{% speaker_note %}}

- Only the speaker can read these notes
- Press `S` key to view
  {{% /speaker_note %}}

Press the S key to view the speaker notes!

Themes

black: Black background, white text, blue links (default)
white: White background, black text, blue links
league: Gray background, white text, blue links
beige: Beige background, dark text, brown links
sky: Blue background, thin dark text, blue links

night: Black background, thick white text, orange links
serif: Cappuccino background, gray text, brown links
simple: White background, black text, blue links
solarized: Cream-colored background, dark green text, blue links

Custom Slide

Customize the slide style and background

{{< slide background-image="/media/boards.jpg" >}}
{{< slide background-color="#0000FF" >}}
{{< slide class="my-style" >}}

Custom CSS Example

Let’s make headers navy colored.

Create assets/css/reveal_custom.css with:

.reveal section h1,
.reveal section h2,
.reveal section h3 {
  color: navy;
}

Questions?

Ask

Documentation

Mon, 01 Jan 0001 00:00:00 +0000