This is the project page for paper: Universal Image Segmentation with Efficiency.
This paper expands the YOSO framework for efficient unified image segmentation, encompassing panoptic, semantic, instance, and video segmentation capabilities.
The key distinctions are:
- The "UISE/re-parameterize.ipynb" code illustrates the process of reparameterization between IFA and CFA.
- The "UISE/uise_video/" directory demonstrates the integration of UISE into video segmentation workflows.
- The "UISE/config/" directory shows the segmentation tasks that UISE newly supports.
Click to download the corresponding model from the model zoo.
| Dataset | Pan | Ins | Sem | Video |
|---|---|---|---|---|
| coco | 48.8 PQ | 32.9 mAP | 58.7 mIoU | - |
| cityscapes | 59.9 PQ | 32.4 mAP | 79.3 mIoU | - |
| ade20k | 38.3 PQ | 22.3 mAP | 44.1 mIoU | - |
| mapillary | 34.2 PQ | - | 50.1 mIoU | - |
| ytb19 | - | - | - | 40.8 mAP |
| ytb21 | - | - | - | 36.2 mAP |
We recommend to use Anaconda for installation.
conda create -n UISE python=3.8 -y
conda activate UISE
conda install pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=11.3 -c pytorch
pip install pycocotools
pip install git+https://github.com/cocodataset/panopticapi.git
pip install fvcore
pip install Pillow==8.4.0
pip install cloudpickle
pip install timm
pip install scipy
pip install opencv-python
git clone https://github.com/hujiecpp/UISE.git
cd UISE
python setup.py developSee Preparing Datasets for Mask2Former.
- Train UISE (e.g., on COCO dataset with R50 backbone).
python UISE/train_net.py --num-gpus 4 --config-file UISE/configs/coco/panoptic-segmentation/UISE-R50.yaml- Evaluate UISE (e.g., on COCO dataset with R50 backbone).
python UISE/train_net.py --num-gpus 4 --config-file UISE/configs/coco/panoptic-segmentation/UISE-R50.yaml --eval-only MODEL.WEIGHTS ./model_zoo/coco_pan_seg.pth