Authors: Hyangsuk Min, Yuho Lee, Minjeong Ban, Jiaqi Deng, Nicole Hee-Yeon Kim, Taewon Yun, Hang Su, Jason Cai, Hwanjun Song
This is the official github repository for "Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages"
We provide samples in MSumBench, which is available in data.
The metadata provides basic information about UniSumEval. All attributes are included in every file. Below is a description of each attribute.
| Field | Description |
|---|---|
| uid | Dataset unique ID |
| doc_id | ID in source dataset |
| dataset | Source dataset name |
| domain | Domain |
| summary_model | Summarizer name |
| summary | Model summary |
| fv_faithfulness | faithfulness score from fact verification |
| ka_completeness | completeness score from key-fact alignment |
| ka_conciseness | conciseness score from key-fact alginment |
Please consider citation if our paper is useful in your research.
@article{min2025towards,
title={Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages},
author={Min, Hyangsuk and Lee, Yuho and Ban, Minjeong and Deng, Jiaqi and Kim, Nicole Hee-Yeon and Yun, Taewon and Su, Hang and Cai, Jason and Song, Hwanjun},
journal={arXiv preprint arXiv:2506.00549},
year={2025}
}This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP), and partly supported by the National Research Foundation of Korea (NRF).