Atomic Kitting

• 

  Scene finally setup

• 

  Calibrating Robot Arms

Inspiration

Making a valuable impact on a real world application using AI with robotics.

What it does

Effectively utilizing a VisonLanguageAction Model concept to orchestrate atomic actions of 6 DOF (degree of freedom) robotic arm to demonstrate pack and assembly task done by humans in a warehouse.

How we built it

1. Used Phospho AI recording tools, capturing the Arm actions and create a library of actions to be orchestrated by VLA, Used Phospho APIs to move the arm orchestrated with an LLM to perform the tasks. The input is a QR code that can be read by the robotic arm to initate the assembly task, we used OpenCV for this task

Challenges we ran into

1. Creating the staging environment to capture the required workflow with limited reach of the arm
2. Lack of time to train.

Accomplishments that we're proud of

1. Fully encapsulated project and working demo of a real world application

What we learned

1. Using Teleoperated actions to capture datasets that can be used by a VLA.

What's next for Atomic Kitting

1. Explore adding more autonomous capabilities to improve the dexterity of the arm for grasping objects
2. Include exception handeling capabilities

Built With

• huggingface
• openai
• opencv
• phospho
• python
• vla