Skip to content

Fix inaccuracies in ground_truth_schema.md#1

Merged
sriumcp merged 1 commit intoinference-sim:mainfrom
susiejojo:fix/ground-truth-schema-accuracy
Mar 6, 2026
Merged

Fix inaccuracies in ground_truth_schema.md#1
sriumcp merged 1 commit intoinference-sim:mainfrom
susiejojo:fix/ground-truth-schema-accuracy

Conversation

@susiejojo
Copy link
Copy Markdown
Collaborator

@susiejojo susiejojo commented Mar 6, 2026

Summary

  • Rename inference-perf-data/ to results/ to match actual on-disk directory name
  • Fix timestamp descriptions from "epoch seconds" to "monotonic clock seconds" (time.monotonic())
  • Add missing RequestError class and error field on PerRequestMetrics
  • Mark LoadSummary.send_duration/requested_rate/achieved_rate as optional (absent in summary metrics, only present in per-stage)

Note

Everything in ground_truth_schema.md is already covered by schemas.py — this doc is a simplified reference card, not the source of truth for Pydantic models.

ground_truth_schema.md schemas.py Notes
ExpConfig ExperimentConfig Same fields
SharedPrefixData SharedPrefixConfig schemas.py adds profile.yaml aliases
LoadStage LoadStage Identical
LoadConfig LoadConfig schemas.py adds interval, sweep, circuit_breakers, etc.
InferencePerfProfile InferencePerfConfig schemas.py strongly types all sub-fields
PercentileDistribution PercentileDistribution Identical, with JSON aliases
LatencyMetrics LatencyMetrics Identical
ThroughputMetrics ThroughputMetrics Identical
LoadSummary LoadSummary Identical
SuccessMetrics / FailureMetrics Same Identical
LifecycleMetrics LifecycleMetrics Identical
RequestError / PerRequestMetrics / RequestInfo Same Identical
KV events (3 types) 9 KV event types + KVEventBatch schemas.py is superset
traces.json (one-liner) Full OTEL model + parsed convenience types schemas.py is far more detailed
vllm_logging.json VLLMLoggingConfig Covered

Test plan

  • Verified directory name against all 16 experiment directories on disk
  • Cross-referenced with schemas.py Pydantic models for consistency

🤖 Generated with Claude Code

- Rename inference-perf-data/ to results/ (matches on-disk directory name)
- Fix timestamps from "epoch seconds" to "monotonic clock seconds" (time.monotonic())
- Add missing RequestError class and error field on PerRequestMetrics
- Mark LoadSummary rate/duration fields as optional (absent in summary metrics)

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@sriumcp sriumcp merged commit 94193d0 into inference-sim:main Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants