Embodied AI involves a broad range of topics. Here, we provide an overview including:

• Simulators and environments
• Robots and controllers
• Datasets and assets
• Tasks and approaches
• Challenges

We will discuss how the visual system connects with the control and actuation system, which is often unclear to researchers in the vision community. Topics include:

• Reinforcement learning: OpenAI Gym interface
• Overview of motion planning and control

Understanding how the simulator works is crucial for leveraging its full capabilities and ensuring accurate simulations. Topics covered will include:

• Rigid-body simulation
• Visual sensor simulation (RGB and Depth)
• Simulatable 3D asset representation and construction

To study vision problems effectively, understanding the underlying simulation technologies and design choices is essential. We will explore:

• Design Dimensions: Embodiment (Sensor, Actuator), Task Specification, Metric
• Case studies: Habitat Challenge, Rearrangement Challenge, ManiSkill Challenge

Despite their benefits, virtual environments can pose challenges. We will share common issues and tips based on feedback from the SAPIEN user community.

Sim2Real bridges the gap between simulation and real-world application. This section includes:

• Case studies on how sim2real domain gaps arise in vision and robot control
• Experiences in deploying policies trained in simulators to the real world

We will summarize our findings from hosting the ManiSkill challenge, discussing:

• A comparison of imitation learning, reinforcement learning, and classic robotics
• Performance summaries by tasks and key challenges
• Analysis on generalization performance