This is the official codebase of the IROS 2025 paper "Action Tokenizer Matters in In-Context Imitation Learning."

Below is a brief explanation for setting up RoboCasa. For further instructions, please refer to RoboCasa.

1. Set up conda environment:

   conda create -c conda-forge -n lipvq python=3.10

2. Activate conda environment:

   conda activate lipvq

3. Clone and setup robosuite dependency (important: use the master branch!):

   git clone https://github.com/ARISE-Initiative/robosuite
   cd robosuite
   pip install -e .

4. Clone and setup this repo:

   cd ..
   git clone https://github.com/andvg3/LipVQ-VAE.git
   cd robocasa
   pip install -e .
   pip install pre-commit; pre-commit install           # Optional: set up code formatter.
   
   (optional: if running into issues with numba/numpy, run: conda install -c numba numba=0.56.4 -y)

5. Install the package and download assets:

   python robocasa/scripts/download_kitchen_assets.py   # Caution: Assets to be downloaded are around 5GB.
   python robocasa/scripts/setup_macros.py              # Set up system variables.

Please refer to the official documentation page for information about tasks and assets, downloading datasets.

Each algorithm has its own config generator script. For example for ICRT+LipVQ-VAE policy run:

robomimic/scripts/config_gen/icl_xfmr_gen.py --name <experiment-name>

After running this script you just need to run the command(s) outputted. Note: You can modify different types of action tokenizer in the outputted config in:

"observation": "modalities": { "fast_enabled": false, "bin_enabled": false, "vq_vae_enabled": true, "ln_act_enabled": false } 

Change the config to your desired tokenizers to test.

Weight are available at this link.

Similar to training, run:

python robomimic/scripts/config_gen/eval_ckpt.py --ckpt <ckpt-path> --name <experiment-name>

then execute the scripts on the screeen.

This repository is largely based on RoboCasa. If you find our code useful, please consider citing it:

@inproceedings{vuong2025action,
  title={Action Tokenizer Matters in In-Context Imitation Learning},
  author={Vuong, An Dinh and Vu, Minh Nhat and An, Dong and Reid, Ian},
  journal={IROS},
  year={2025}
}