A novel DR visualization technique for interactive exploration of multimodal embeddings through Dynamic Kernel enhanced projection

DKMap helps you make sense of multimodal embeddings — especially those from vision-language models — by showing how well text and images align. Traditional methods like t-SNE often fail to accurately reflect alignment metrics due to projection distortion and over-averaging. DKMap solves this with a dynamic kernel approach that makes the visualizations both more accurate and more interactive. You can zoom in, explore local details, and use it in your browser or Jupyter notebook.

✨ Scalable to billions of multimodal embedding point

✨ Enables multi-resolution exploration via kernel refinement

✨ Dynamically adjusts the granularity of the contour map for more flexible exploration.

✨ Support multi-platform development -- computational notebooks (e.g., Colab, Jupyter, VS Code) and Web-based system

We provide prepared embedding files list here:

• DiffusionDB
• Pick-a-Pic

Clone or download this repository:

git clone https://github.com/DKMap-VIS/DKMap.git

Install the dependencies (in the Backend folder):

pip install -r requirements.txt

Install the dependencies (in the Frontend folder):

npm install

Please update the file paths in the following code file according to your local environment.

• File：app.py

  • line 35: (a .csv file) included position, alignment metric, image path and prompt of all data points.
  • line 53: (a .npy file) included your data embedding.

• File: Main.vue

  • line 345: (.csv files) included contour map values.
  • line 53: (a .csv file) included position, alignment metric, image path and prompt of all data points.

Then run DKMap:

npm run dev

Download the source code from the Python package file.

• Most parameters are loaded automatically from config.yaml.
• If you want to use this package in your own project, you need to manually pass the following parameters to the main function:
  • points_ds: dataset embedding (a .npy file)
  • input_dimens: input feature dimension
  • scores: alignment metric embedding (a .npy file)
• Alternatively, you can follow the steps in our provided Projection_and_mapping.ipynb, which demonstrates how to train a projection and mapping model and generate a static projection map.
• For further interactive exploration of the dataset, please refer to Interactive_contour_visualization.ipynb.

@ARTICLE{ye2026DKMap,
  author={Ye, Yilin and Ruan, Chenxi and Zhang, Yu and Deng, Zikun and Zeng, Wei},
  journal={IEEE Transactions on Visualization and Computer Graphics}, 
  title={{DKMap}: Interactive Exploration of Vision-Language Alignment in Multimodal Embeddings via Dynamic Kernel Enhanced Projection}, 
  year={2026},
  volume={32},
  number={1},
  pages={440-450},
  doi={10.1109/TVCG.2025.3642641}
}