Skip to content

fix(ecr-assets): handle Docker 27.4+ output format in TarballImageAsset#33967

Merged
mergify[bot] merged 4 commits intoaws:mainfrom
scorbiere:issue_33428
Apr 2, 2025
Merged

fix(ecr-assets): handle Docker 27.4+ output format in TarballImageAsset#33967
mergify[bot] merged 4 commits intoaws:mainfrom
scorbiere:issue_33428

Conversation

@scorbiere
Copy link
Copy Markdown
Contributor

@scorbiere scorbiere commented Mar 28, 2025

Issue # (if applicable)

Closes #33428.

Reason for this change

The TarballImageAsset class was failing to properly extract image IDs from Docker 27.4+ output due to a format change from "Loaded image: " to "Loaded image ID: ". This caused the sed command to fail to properly extract the image ID, resulting in an invalid tag format.

After reviewing Docker CLI issue docker/cli#2212 (open since 2019), I found that Docker has actually had two distinct output formats for the docker load command for some time:

  • For images with tags: "Loaded image: "
  • For images without tags: "Loaded image ID: "

See:
https://github.com/moby/moby/blob/37f866285af6910f838c762912dfd57396783b72/image/tarexport/load.go#L140-L159

While the issue was triggered by Docker 27.4, the underlying problem is that our code was only handling one of these output formats. We need a solution that handles both formats.

Description of changes

  • Introduced a more flexible regex pattern (s/Loaded image[^:]*: //g) that handles both output formats
  • Extracted the pattern to a shared constant DOCKER_LOAD_OUTPUT_REGEX for consistency and maintainability
  • Updated the code in tarball-asset.ts to use the new regex pattern
  • Added unit tests to verify compatibility with both Docker output formats
  • Ensured backward compatibility with older Docker versions

This approach makes our code more robust against variations in Docker's output format.

Describe any new or updated permissions being added

No new or updated IAM permissions are needed for this change.

Description of how you validated changes

  • Created unit tests that verify the new regex pattern works with both formats using hardcoded test strings:
    • "Loaded image: " (older Docker versions)
    • "Loaded image ID: " (Docker 27.4+)
  • Verified that the old sed expression fails with the new format
  • Confirmed all existing tests pass with the new implementation

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

scorbiere and others added 2 commits March 28, 2025 10:30
…eAsset

The TarballImageAsset class was failing to properly extract image IDs from
Docker 27.4+ output due to a format change from "Loaded image: <digest>" to
"Loaded image ID: <digest>".

This change:
- Introduces a more flexible regex pattern that handles both output formats
- Extracts the pattern to a shared constant for consistency
- Adds unit tests to verify compatibility with both formats
- Ensures backward compatibility with older Docker versions

Fixes aws#33428
@github-actions github-actions Bot added bug This issue is a bug. effort/medium Medium work item – several days of effort p1 labels Mar 28, 2025
@aws-cdk-automation aws-cdk-automation requested a review from a team March 28, 2025 20:18
@mergify mergify Bot added the contribution/core This is a PR that came from AWS. label Mar 28, 2025
Copy link
Copy Markdown
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@scorbiere scorbiere changed the title fix(aws-ecr-assets): handle Docker 27.4+ output format in TarballImageAsset fix(ecr-assets): handle Docker 27.4+ output format in TarballImageAsset Mar 28, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 28, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.98%. Comparing base (a8edf69) to head (eb5727f).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #33967   +/-   ##
=======================================
  Coverage   83.98%   83.98%           
=======================================
  Files         120      120           
  Lines        6976     6976           
  Branches     1178     1178           
=======================================
  Hits         5859     5859           
  Misses       1005     1005           
  Partials      112      112           
Flag Coverage Δ
suite.unit 83.98% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk ∅ <ø> (∅)
packages/aws-cdk-lib/core 83.98% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@scorbiere
Copy link
Copy Markdown
Contributor Author

Exemption Request: The unit test already validate the correct extraction of the image ID.

@aws-cdk-automation aws-cdk-automation added the pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. label Mar 29, 2025
@scorbiere scorbiere marked this pull request as ready for review March 29, 2025 00:13
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Mar 29, 2025
@shikha372 shikha372 self-assigned this Apr 1, 2025
@shikha372 shikha372 added pr-linter/exempt-integ-test The PR linter will not require integ test changes and removed pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. labels Apr 1, 2025
@aws-cdk-automation aws-cdk-automation dismissed their stale review April 1, 2025 22:57

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 1, 2025

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Apr 1, 2025
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 1, 2025

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: eb5727f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 2, 2025

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify Bot merged commit 009680d into aws:main Apr 2, 2025
20 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 2, 2025

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Apr 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug This issue is a bug. contribution/core This is a PR that came from AWS. effort/medium Medium work item – several days of effort p1 pr-linter/exempt-integ-test The PR linter will not require integ test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@aws-cdk/aws-ecr-assets: TarballImageAsset is not properly docker-loading image with docker 27.4

3 participants