Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Apr 21, 2026

• [Fix] SciDocBench: decouple reasoning eval from answer score. (#1519)

  Show description for e7d64cf
  yuhangzang

  authored
  e7d64cf

  Copy full SHA for e7d64cf

Commits on Apr 10, 2026

• [Fix] SciDocBench: use empty DATASET_MD5 to skip hash check on TSV updates (#1512)

  yuhangzang

  authored
  0a11caf

  Copy full SHA for 0a11caf

• [Benchmark] Add support for SciDocBench benchmark (#1511)

  Show description for 6cd39bb
  yuhangzang

  authored
  6cd39bb

  Copy full SHA for 6cd39bb

Commits on Apr 9, 2026

• [Refactor] Merge run_api.py into run.py and overhaul reuse/checkpoint mechanism (#1499)

  Show description for a9343a1
  mzr1996

  authored
  a9343a1

  Copy full SHA for a9343a1

Commits on Apr 8, 2026

• Add News for Video-MME-v2 (#1509)

  dongyh20

  authored
  637ce46

  Copy full SHA for 637ce46

• [Benchmark] Support Video-MME-v2 (#1508)

  Show description for ca905c5
  dongyh20

  authored
  ca905c5

  Copy full SHA for ca905c5

Commits on Apr 6, 2026

• [Fix] Make CCOCR KIE ground-truth parsing more robust. (#1506)

  Show description for 9e3cc84
  

  inigopm

  and

  inakiLakunza

  authored
  9e3cc84

  Copy full SHA for 9e3cc84

Commits on Mar 27, 2026

• [Benchmark] Add support for MMOral-OPG-Closed benchmark (#1483)

  Show description for a3656d4
  isjinghao

  authored
  a3656d4

  Copy full SHA for a3656d4

• [Benchmark] Add support for MMOral-OPG-Open benchmark (#1484)

  Show description for 589fe36
  

  3 peopleauthored
  589fe36

  Copy full SHA for 589fe36

Commits on Mar 25, 2026

• [Lint] Standardize code style and logger usage. (#1493)

  Show description for 401916c
  mzr1996

  authored
  401916c

  Copy full SHA for 401916c

• [Fix] Fix ChartX Evaluation Robustness. (#1489)

  Show description for f67e576
  Li-Jinsong

  authored
  f67e576

  Copy full SHA for f67e576

Commits on Mar 23, 2026

• [Benchmark] Add support for MMSafetyBench, XSTest, MMSBench, Flames, SIUO and M3oralBench. (#1488)

  Show description for 6e7e372
  

  3 peopleauthored
  6e7e372

  Copy full SHA for 6e7e372

• [Feature] add MiniMax as text LLM API provider (#1491)

  Show description for f91760e
  

  3 peopleauthored
  f91760e

  Copy full SHA for f91760e

• [Benchmark] Support SArena Benchmark (#1371)

  Show description for 7d6d81f
  

  3 peopleauthored
  7d6d81f

  Copy full SHA for 7d6d81f

• [Update] Update model settings for 2026.2 live leaderboard. (#1492)

  Show description for 655e65f
  mzr1996

  authored
  655e65f

  Copy full SHA for 655e65f

• [Benchmark] Support ChartBench. (#1490)

  Li-Jinsong

  authored
  f765748

  Copy full SHA for f765748

• [Feature] Support sequential inference accorss all datasets and parallel evaluation. (#1487)

  Show description for 4baeeee
  

  TianhaoLiang2000

  and

  mzr1996

  authored
  4baeeee

  Copy full SHA for 4baeeee

Commits on Mar 20, 2026

• [Benchmark] Add SGI-Bench 1.0 dataset (#1464)

  Show description for 161d400
  

  TianhaoLiang2000

  and

  mzr1996

  authored
  161d400

  Copy full SHA for 161d400

Commits on Mar 19, 2026

• [Fix] physics: deduplicate preds to handle repeat output, add signal timeout handler (#1470)

  Show description for 1e2b2f9
  TianhaoLiang2000

  authored
  1e2b2f9

  Copy full SHA for 1e2b2f9

Commits on Mar 18, 2026

• [Benchmark] Add OCRBench_v2_MINI dataset (#1454)

  Show description for 2e2b959
  TianhaoLiang2000

  authored
  2e2b959

  Copy full SHA for 2e2b959

• [Benchmark] XLRSBench: rewrite evaluate() with track_progress_rich and GPT-assisted scoring (#1472)

  Show description for f8e2bc1
  TianhaoLiang2000

  authored
  f8e2bc1

  Copy full SHA for f8e2bc1

• [Fix] olympiadbench: add isinstance(line, int) guard in build_prompt to handle integer row index (#1469)

  TianhaoLiang2000

  authored
  9049740

  Copy full SHA for 9049740

• [Minor] mmifeval: use __name__ for logger, downgrade score_dict log to debug (#1468)

  TianhaoLiang2000

  authored
  128c925

  Copy full SHA for 128c925

• [Minor] macbench: enable interleave by default, move images before text in non-interleave mode (#1467)

  Show description for e7d855a
  

  TianhaoLiang2000

  and

  claude

  authored
  e7d855a

  Copy full SHA for e7d855a

• [Benchmark] ChartMimic: add exception guard, 600s timeout, chart_type 120s, log to debug (#1465)

  Show description for 62f19ac
  TianhaoLiang2000

  authored
  62f19ac

  Copy full SHA for 62f19ac

Commits on Mar 17, 2026

• [Benchmark] Add AV-SpeakerBench (#1393)

  Show description for f6c9b25
  

  4 peopleauthored
  f6c9b25

  Copy full SHA for f6c9b25

• [Fix] stop poisoning EVAL_FORMAT for subsequent datasets (#1475)

  Show description for 43ce748
  

  monurcan

  and

  Copilot

  authored
  43ce748

  Copy full SHA for 43ce748

• [Model] Add support for MiniCPM-o-4_5 (#1478)

  floe20

  authored
  ee8c28a

  Copy full SHA for ee8c28a

Commits on Mar 5, 2026

• [Benchmark] Add support for NPMM-bench (#1463)

  Show description for 0080421
  

  OliverLeeXZ

  and

  FangXinyu-0913

  authored
  0080421

  Copy full SHA for 0080421

Commits on Mar 4, 2026

• [Benchmark] Add DA-2K, ERQA, and RefSpatialBench support (#1461)

  Show description for 8a904f5
  

  4 peopleauthored
  8a904f5

  Copy full SHA for 8a904f5

• chartqapro: cast prediction to str before strip to fix non-string crash (#1466)

  TianhaoLiang2000

  authored
  35ab470

  Copy full SHA for 35ab470

• [Benchmark] Add support for VenusBench-GD (#1449)

  Show description for c36dbff
  

  SunzeY

  and

  kennymckormick

  authored
  c36dbff

  Copy full SHA for c36dbff

• [EASI] merge EASI added benchmarks and models into VLMEvalKit (#1433)

  Show description for ef83c66
  

  4 peopleauthored
  ef83c66

  Copy full SHA for ef83c66

Commits on Mar 3, 2026

• [Benchmark] Support VBGD (#1448)

  Show description for bbf2457
  

  SunzeY

  and

  kennymckormick

  authored
  bbf2457

  Copy full SHA for bbf2457

• [Benchmark] Support osworld-g (#1447)

  Show description for b50c3e9
  SunzeY

  authored
  b50c3e9

  Copy full SHA for b50c3e9