Skip to content

next-core: deduplicate output assets and detect content conflicts on emit#92292

Merged
sokra merged 3 commits intocanaryfrom
sokra/emit-conflict
Apr 5, 2026
Merged

next-core: deduplicate output assets and detect content conflicts on emit#92292
sokra merged 3 commits intocanaryfrom
sokra/emit-conflict

Conversation

@sokra
Copy link
Copy Markdown
Member

@sokra sokra commented Apr 3, 2026

What?

Adds deduplication and conflict detection to the asset emission stage in crates/next-core/src/emit.rs, and a new IssueStage::Emit variant in turbopack-core.

Before emitting, assets are grouped by their output path. If multiple assets map to the same path:

  • If their content is identical, one is silently chosen (deduplication).
  • If their content differs, both versions are written to <node_root>/<content_hash>.<ext> and an EmitConflictIssue is raised for each conflict. All assets are still emitted — conflicts do not abort the build.

Why?

Previously, duplicate output assets for the same path were emitted unconditionally — whichever write happened last silently won. This masked build graph bugs where two different modules produced conflicting output files. Reporting conflicts as issues (rather than silently overwriting) makes them visible and easy to diagnose without breaking the build.

How?

  • Collect all assets with their resolved paths via try_flat_join.
  • Bucket them into two FxIndexMap<FileSystemPath, Vec<ResolvedVc<Box<dyn OutputAsset>>>> — one for node-root assets and one for client assets.
  • For each bucket entry, call check_duplicates: compare every asset against the first using assets_diff. If content differs, emit an EmitConflictIssue as a turbo-tasks collectible — but still return the first asset so emission continues.
  • assets_diff is a #[turbo_tasks::function] that takes only (asset1, asset2, extension, node_root) — the asset_path stays out of the task key to avoid unnecessary task cardinality. When file content differs, it hashes each version with xxh3, writes them to <node_root>/<hash>.<ext>, and returns the paths in the detail message so the user can diff them.
  • EmitConflictIssue implements the Issue trait with IssueStage::Emit (new variant added to turbopack-core), IssueSeverity::Error, a descriptive title, and a detail message explaining the type of conflict.
  • Node-root and client assets are emitted in parallel via futures::join! (not try_join!) to ensure deterministic error reporting — both branches always run to completion so errors are reported in a consistent order.

@nextjs-bot nextjs-bot added created-by: Turbopack team PRs by the Turbopack team. Turbopack Related to Turbopack with Next.js. labels Apr 3, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Apr 3, 2026

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 3 skipped benchmarks1


Comparing sokra/emit-conflict (f378f4c) with canary (f65b10a)

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@nextjs-bot
Copy link
Copy Markdown
Collaborator

nextjs-bot commented Apr 3, 2026

Stats from current PR

✅ No significant changes detected

📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 455ms 455ms ▁█▁▁▁
Cold (Ready in log) 439ms 439ms ▁█▄▅▄
Cold (First Request) 1.127s 1.103s ▄▃▃▄▇
Warm (Listen) 456ms 456ms ▁█▁▁▁
Warm (Ready in log) 439ms 443ms ▃▅▂▅▂
Warm (First Request) 336ms 338ms ▄▄▅█▁

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 3.795s 3.783s ▃▄▆▃▆
Cached Build 3.823s 3.826s ▂▁▅▂▅
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
node_modules Size 487 MB 487 MB █████
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
02fkg8wfh0iju.js gzip 9.19 kB N/A -
050zwt5xh_0tx.js gzip 10.4 kB N/A -
06rvbj82bhyo0.js gzip 13 kB N/A -
087fzjd-gvlzv.js gzip 450 B N/A -
0cz1d0mv5g_q7.js gzip 39.4 kB 39.4 kB
0hmlq-poen_t-.js gzip 157 B N/A -
0jinvht928g4f.js gzip 158 B N/A -
0kkw8tg-ocubb.js gzip 155 B N/A -
0l1lzugp0ksvm.js gzip 160 B N/A -
0ppxcl_z43mad.js gzip 8.52 kB N/A -
0ux97pue-yjq1.js gzip 156 B N/A -
0xaga-wfp9zue.js gzip 70.8 kB N/A -
17zjqcbzkqr3i.js gzip 168 B N/A -
19oha6-znmkcv.js gzip 8.55 kB N/A -
1elt1qium-r2m.css gzip 115 B 115 B
1izqq-lh0tfe7.js gzip 155 B N/A -
1pguylug0spyd.js gzip 65.7 kB N/A -
2_5rjb7lqxntf.js gzip 221 B 221 B
219prxwxgaalc.js gzip 7.61 kB N/A -
26elcgxnn9zjd.js gzip 8.52 kB N/A -
2900hudr6gvm0.js gzip 2.28 kB N/A -
2dv-l_jnh4nll.js gzip 156 B N/A -
2e8e7a1ici2sg.js gzip 157 B N/A -
2e99sleuy25ku.js gzip 162 B N/A -
2lv2js3kmdeho.js gzip 8.48 kB N/A -
2rehygrd36hqv.js gzip 8.58 kB N/A -
2srwswih0m9_h.js gzip 13.3 kB N/A -
2ti84q_oc3z76.js gzip 153 B N/A -
3-p9p9mheqhzx.js gzip 8.55 kB N/A -
31030bryqpolg.js gzip 8.53 kB N/A -
31dx5nmrzzuy7.js gzip 225 B N/A -
3925v09gtu-5k.js gzip 49 kB N/A -
39x4zj5mjb4d_.js gzip 9.77 kB N/A -
3k-48b78ys_vy.js gzip 10.1 kB N/A -
3m7-5rfj0avoz.js gzip 12.9 kB N/A -
3uqce_6sa526g.js gzip 8.47 kB N/A -
3yurjqk-sjs3y.js gzip 1.46 kB N/A -
3zb6hoq6zjtmo.js gzip 156 B N/A -
40ybjx9c192n0.js gzip 13.8 kB N/A -
421vzwdt9j1b_.js gzip 5.62 kB N/A -
44ll2gsv38cvh.js gzip 157 B N/A -
turbopack-07..dmoz.js gzip 4.18 kB N/A -
turbopack-17..e6ja.js gzip 4.18 kB N/A -
turbopack-17..61z7.js gzip 4.16 kB N/A -
turbopack-1d..v4xm.js gzip 4.17 kB N/A -
turbopack-1p..59vf.js gzip 4.18 kB N/A -
turbopack-20..zwty.js gzip 4.17 kB N/A -
turbopack-2g..ta08.js gzip 4.19 kB N/A -
turbopack-2g..4wg7.js gzip 4.17 kB N/A -
turbopack-2i..lf-6.js gzip 4.18 kB N/A -
turbopack-2m..j0rz.js gzip 4.18 kB N/A -
turbopack-2p..n6lj.js gzip 4.18 kB N/A -
turbopack-3_..0fom.js gzip 4.18 kB N/A -
turbopack-37..v8dn.js gzip 4.18 kB N/A -
turbopack-3v..bsb8.js gzip 4.17 kB N/A -
03dgzoo-qf3sm.js gzip N/A 9.19 kB -
05tx5f25dlivn.js gzip N/A 8.53 kB -
0c7ez6p2qc57f.js gzip N/A 5.62 kB -
0duvj3qk5pvgn.js gzip N/A 13.8 kB -
0g2x7gvve9v2z.js gzip N/A 161 B -
0g9hwiffim6jx.js gzip N/A 155 B -
0m-34rm9w_wpm.js gzip N/A 7.6 kB -
0qnwuk92m8i7o.js gzip N/A 10.4 kB -
0r4wrn6n0ue2m.js gzip N/A 8.55 kB -
0rimus-isijl3.js gzip N/A 154 B -
0rp0fodtbt_6m.js gzip N/A 8.52 kB -
0sfck-km4dl1k.js gzip N/A 8.47 kB -
0wsggm2gh-ges.js gzip N/A 158 B -
0x0xuhmxzwkp8.js gzip N/A 8.47 kB -
1-wdvgxnzicj7.js gzip N/A 1.46 kB -
11u6nxujb2eg4.js gzip N/A 450 B -
1c5i51fexm_yj.js gzip N/A 65.7 kB -
1c9u8qch78g7q.js gzip N/A 70.8 kB -
1jv-o1_s-zmua.js gzip N/A 49 kB -
2k9ax08cjl2id.js gzip N/A 12.9 kB -
2lmcaq17omvnr.js gzip N/A 156 B -
2lms6k76q5-6m.js gzip N/A 13.3 kB -
2qx4twi9i3xus.js gzip N/A 2.28 kB -
2srnqic6tvxxd.js gzip N/A 8.52 kB -
2u1r9bcfx_7im.js gzip N/A 160 B -
2z0szsm15vh2b.js gzip N/A 155 B -
2zz_835zwahub.js gzip N/A 154 B -
307h6spznj8d7.js gzip N/A 156 B -
30l7m4nayp73a.js gzip N/A 8.55 kB -
38rr7d3kfutni.js gzip N/A 13 kB -
3ak3w7d8ttioq.js gzip N/A 158 B -
3b5uj2fcp7c5w.js gzip N/A 157 B -
3h_ecpiaatwgc.js gzip N/A 10.1 kB -
3ity0aahajapd.js gzip N/A 225 B -
3qw7x5va53jl3.js gzip N/A 169 B -
3wm9vhzaz51n2.js gzip N/A 156 B -
3wrhpuc-j1aw9.js gzip N/A 9.77 kB -
43mlw9dy_8f02.js gzip N/A 8.58 kB -
turbopack-0b..psap.js gzip N/A 4.18 kB -
turbopack-0h..bf9_.js gzip N/A 4.18 kB -
turbopack-11..bvy5.js gzip N/A 4.18 kB -
turbopack-1s..c1r4.js gzip N/A 4.18 kB -
turbopack-2a..32m2.js gzip N/A 4.18 kB -
turbopack-2d..xhfa.js gzip N/A 4.18 kB -
turbopack-2m..kbi3.js gzip N/A 4.18 kB -
turbopack-2s..zi-m.js gzip N/A 4.16 kB -
turbopack-2w..z0ye.js gzip N/A 4.18 kB -
turbopack-2z..txag.js gzip N/A 4.19 kB -
turbopack-3_..sqc_.js gzip N/A 4.17 kB -
turbopack-3-..cvdg.js gzip N/A 4.18 kB -
turbopack-3b..8bay.js gzip N/A 4.17 kB -
turbopack-42..06h6.js gzip N/A 4.18 kB -
Total 464 kB 464 kB ✅ -9 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 717 B 716 B
Total 717 B 716 B ✅ -1 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 432 B 426 B 🟢 6 B (-1%)
Total 432 B 426 B ✅ -6 B

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 342 kB 342 kB
app-page-exp..prod.js gzip 189 kB 189 kB
app-page-tur...dev.js gzip 341 kB 341 kB
app-page-tur..prod.js gzip 189 kB 189 kB
app-page-tur...dev.js gzip 338 kB 338 kB
app-page-tur..prod.js gzip 187 kB 187 kB
app-page.run...dev.js gzip 338 kB 338 kB
app-page.run..prod.js gzip 187 kB 187 kB
app-route-ex...dev.js gzip 76.6 kB 76.6 kB
app-route-ex..prod.js gzip 52.2 kB 52.2 kB
app-route-tu...dev.js gzip 76.6 kB 76.6 kB
app-route-tu..prod.js gzip 52.2 kB 52.2 kB
app-route-tu...dev.js gzip 76.2 kB 76.2 kB
app-route-tu..prod.js gzip 52 kB 52 kB
app-route.ru...dev.js gzip 76.2 kB 76.2 kB
app-route.ru..prod.js gzip 52 kB 52 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 43.8 kB 43.8 kB
pages-api-tu..prod.js gzip 33.4 kB 33.4 kB
pages-api.ru...dev.js gzip 43.8 kB 43.8 kB
pages-api.ru..prod.js gzip 33.4 kB 33.4 kB
pages-turbo....dev.js gzip 53.2 kB 53.2 kB
pages-turbo...prod.js gzip 39 kB 39 kB
pages.runtim...dev.js gzip 53.2 kB 53.2 kB
pages.runtim..prod.js gzip 39 kB 39 kB
server.runti..prod.js gzip 62.8 kB 62.8 kB
Total 3.03 MB 3.03 MB
📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/f378f4c0cce6ca57dfab527cc1146ed9e3dc2268/next

@nextjs-bot
Copy link
Copy Markdown
Collaborator

nextjs-bot commented Apr 3, 2026

Tests Passed

@sokra sokra requested a review from lukesandberg April 3, 2026 14:27
@sokra sokra marked this pull request as ready for review April 3, 2026 14:27
Comment thread crates/next-core/src/emit.rs Outdated
sokra and others added 3 commits April 4, 2026 16:52
Deduplicate assets by path before emitting, and bail with an error when
duplicate assets have different content. Cherry-picked emit.rs changes
from 576587e.

Co-Authored-By: Claude <[email protected]>
Add a dedicated IssueStage::Emit variant instead of using
IssueStage::Other("emit"). Rename check_emit_conflict back to
assets_diff which now only returns the diff description, and emit the
EmitConflictIssue from check_duplicates instead. This avoids passing
asset_path into the turbo_tasks function, reducing task key cardinality.

Co-Authored-By: Claude <[email protected]>
FileSystemPath::extension() now returns Option<&str> after upstream
dedup of extension methods. Use unwrap_or_default() to handle the
None case.

Co-Authored-By: Claude <[email protected]>
@sokra sokra force-pushed the sokra/emit-conflict branch from cbbc4b2 to f378f4c Compare April 4, 2026 16:56
@sokra sokra merged commit 81d5e07 into canary Apr 5, 2026
314 of 317 checks passed
@sokra sokra deleted the sokra/emit-conflict branch April 5, 2026 18:38
eps1lon pushed a commit that referenced this pull request Apr 7, 2026
…emit (#92292)

### What?

Adds deduplication and conflict detection to the asset emission stage in
`crates/next-core/src/emit.rs`, and a new `IssueStage::Emit` variant in
`turbopack-core`.

Before emitting, assets are grouped by their output path. If multiple
assets map to the same path:

- If their content is identical, one is silently chosen (deduplication).
- If their content differs, both versions are written to
`<node_root>/<content_hash>.<ext>` and an `EmitConflictIssue` is raised
for each conflict. All assets are still emitted — conflicts do not abort
the build.

### Why?

Previously, duplicate output assets for the same path were emitted
unconditionally — whichever write happened last silently won. This
masked build graph bugs where two different modules produced conflicting
output files. Reporting conflicts as issues (rather than silently
overwriting) makes them visible and easy to diagnose without breaking
the build.

### How?

- Collect all assets with their resolved paths via `try_flat_join`.
- Bucket them into two `FxIndexMap<FileSystemPath,
Vec<ResolvedVc<Box<dyn OutputAsset>>>>` — one for node-root assets and
one for client assets.
- For each bucket entry, call `check_duplicates`: compare every asset
against the first using `assets_diff`. If content differs, emit an
`EmitConflictIssue` as a turbo-tasks collectible — but still return the
first asset so emission continues.
- `assets_diff` is a `#[turbo_tasks::function]` that takes only
`(asset1, asset2, extension, node_root)` — the `asset_path` stays out of
the task key to avoid unnecessary task cardinality. When file content
differs, it hashes each version with xxh3, writes them to
`<node_root>/<hash>.<ext>`, and returns the paths in the detail message
so the user can diff them.
- `EmitConflictIssue` implements the `Issue` trait with
`IssueStage::Emit` (new variant added to `turbopack-core`),
`IssueSeverity::Error`, a descriptive title, and a detail message
explaining the type of conflict.
- Node-root and client assets are emitted in parallel via
`futures::join!` (not `try_join!`) to ensure deterministic error
reporting — both branches always run to completion so errors are
reported in a consistent order.

---------

Co-authored-by: Tobias Koppers <[email protected]>
Co-authored-by: Claude <[email protected]>
eps1lon pushed a commit that referenced this pull request Apr 7, 2026
…emit (#92292)

### What?

Adds deduplication and conflict detection to the asset emission stage in
`crates/next-core/src/emit.rs`, and a new `IssueStage::Emit` variant in
`turbopack-core`.

Before emitting, assets are grouped by their output path. If multiple
assets map to the same path:

- If their content is identical, one is silently chosen (deduplication).
- If their content differs, both versions are written to
`<node_root>/<content_hash>.<ext>` and an `EmitConflictIssue` is raised
for each conflict. All assets are still emitted — conflicts do not abort
the build.

### Why?

Previously, duplicate output assets for the same path were emitted
unconditionally — whichever write happened last silently won. This
masked build graph bugs where two different modules produced conflicting
output files. Reporting conflicts as issues (rather than silently
overwriting) makes them visible and easy to diagnose without breaking
the build.

### How?

- Collect all assets with their resolved paths via `try_flat_join`.
- Bucket them into two `FxIndexMap<FileSystemPath,
Vec<ResolvedVc<Box<dyn OutputAsset>>>>` — one for node-root assets and
one for client assets.
- For each bucket entry, call `check_duplicates`: compare every asset
against the first using `assets_diff`. If content differs, emit an
`EmitConflictIssue` as a turbo-tasks collectible — but still return the
first asset so emission continues.
- `assets_diff` is a `#[turbo_tasks::function]` that takes only
`(asset1, asset2, extension, node_root)` — the `asset_path` stays out of
the task key to avoid unnecessary task cardinality. When file content
differs, it hashes each version with xxh3, writes them to
`<node_root>/<hash>.<ext>`, and returns the paths in the detail message
so the user can diff them.
- `EmitConflictIssue` implements the `Issue` trait with
`IssueStage::Emit` (new variant added to `turbopack-core`),
`IssueSeverity::Error`, a descriptive title, and a detail message
explaining the type of conflict.
- Node-root and client assets are emitted in parallel via
`futures::join!` (not `try_join!`) to ensure deterministic error
reporting — both branches always run to completion so errors are
reported in a consistent order.

---------

Co-authored-by: Tobias Koppers <[email protected]>
Co-authored-by: Claude <[email protected]>
lukesandberg pushed a commit that referenced this pull request Apr 7, 2026
…emit (#92292)

Adds deduplication and conflict detection to the asset emission stage in
`crates/next-core/src/emit.rs`, and a new `IssueStage::Emit` variant in
`turbopack-core`.

Before emitting, assets are grouped by their output path. If multiple
assets map to the same path:

- If their content is identical, one is silently chosen (deduplication).
- If their content differs, both versions are written to
`<node_root>/<content_hash>.<ext>` and an `EmitConflictIssue` is raised
for each conflict. All assets are still emitted — conflicts do not abort
the build.

Previously, duplicate output assets for the same path were emitted
unconditionally — whichever write happened last silently won. This
masked build graph bugs where two different modules produced conflicting
output files. Reporting conflicts as issues (rather than silently
overwriting) makes them visible and easy to diagnose without breaking
the build.

- Collect all assets with their resolved paths via `try_flat_join`.
- Bucket them into two `FxIndexMap<FileSystemPath,
Vec<ResolvedVc<Box<dyn OutputAsset>>>>` — one for node-root assets and
one for client assets.
- For each bucket entry, call `check_duplicates`: compare every asset
against the first using `assets_diff`. If content differs, emit an
`EmitConflictIssue` as a turbo-tasks collectible — but still return the
first asset so emission continues.
- `assets_diff` is a `#[turbo_tasks::function]` that takes only
`(asset1, asset2, extension, node_root)` — the `asset_path` stays out of
the task key to avoid unnecessary task cardinality. When file content
differs, it hashes each version with xxh3, writes them to
`<node_root>/<hash>.<ext>`, and returns the paths in the detail message
so the user can diff them.
- `EmitConflictIssue` implements the `Issue` trait with
`IssueStage::Emit` (new variant added to `turbopack-core`),
`IssueSeverity::Error`, a descriptive title, and a detail message
explaining the type of conflict.
- Node-root and client assets are emitted in parallel via
`futures::join!` (not `try_join!`) to ensure deterministic error
reporting — both branches always run to completion so errors are
reported in a consistent order.

---------

Co-authored-by: Tobias Koppers <[email protected]>
Co-authored-by: Claude <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

created-by: Turbopack team PRs by the Turbopack team. locked Turbopack Related to Turbopack with Next.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants