Skip to content

MINOR: Fix shadow jar publishing for the clients module#15127

Merged
stanislavkozlovski merged 1 commit intoapache:trunkfrom
apoorvmittal10:publish-fix
Jan 8, 2024
Merged

MINOR: Fix shadow jar publishing for the clients module#15127
stanislavkozlovski merged 1 commit intoapache:trunkfrom
apoorvmittal10:publish-fix

Conversation

@apoorvmittal10
Copy link
Contributor

@apoorvmittal10 apoorvmittal10 commented Jan 4, 2024

What

The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar is recently been shadowed which does publish the artifacts to local maven successfully but errors when publishing to remote maven. The issue triggers only with publishMavenJavaPublicationToMavenRepository and signing but publishToMavenLocal works correctly with earlier changes. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.

Below is another way where the artifact details can be overridden post construction in afterEvaluate to let publish plugin know about the correct artifactId but it doesn't solve the signing plugin issue where signing plugin is unaware of kafka-client-<version>.jar and skips signing. For signing plugin the artifact should be explicitly defined as well.

            project.shadow.component(mavenJava)
        
            afterEvaluate {
              pom.withXml {
                asNode().dependencies.dependency.findAll() {
                  def projectName = it.artifactId.text()
                  if (['clients'].contains(projectName)) {
                    it.artifactId*.value = 'kafka-clients'
                  }
                }
              }
            }

Tests:

  • I created person gpg secrets locally for signing and private maven repo to upload the artifacts. I ran the release pipeline pointing to custom repo and can successfully push the kafka-clients artifacts.

Screenshot 2024-01-04 at 3 46 06 PM

  • Testing shaded dependencies generated successfully. I can see no difference in the previous and release shaded jars, and can also see shaded dependencies in the jar.

Screenshot 2024-01-04 at 3 50 11 PM

Query:

  • I compared the 3.6.1 kafka release artifacts here with generated artifacts in my personal repo, and can find module metadata artifact is missing. Debugged the issue and it seems gradle publish - from components.java actually registers both jar and module for publish which is not there in case of shadowJar hence the question is that do we require to publish module metadata information or are we good?

cc: @stanislavkozlovski @xvrl @AndrewJSchofield

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

stanislavkozlovski
stanislavkozlovski approved these changes Jan 4, 2024
View reviewed changes
Copy link
Contributor

@stanislavkozlovski stanislavkozlovski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing sleuthing! I tested the publish again and confirmed it works!

I don't have an answer to:

I compared the 3.6.1 kafka release artifacts here with generated artifacts in my personal repo, and can find module metadata artifact is missing. Debugged the issue and it seems gradle publish - from components.java actually registers both jar and module for publish which is not there in case of shadowJar hence the question is that do we require to publish module metadata information or are we good?

Who is a good person to ask about this? Maybe some of the original reviewers of the shadow PR?

@apoorvmittal10
Copy link
Contributor Author

apoorvmittal10 commented Jan 4, 2024

I tested the publish again and confirmed it works!

Awesome!

Who is a good person to ask about this? Maybe some of the original reviewers of the shadow PR?

May be @ijuma or @xvrl can help here.

@stanislavkozlovski stanislavkozlovski changed the title MINOR: Fixing shadow jar publish MINOR: Fix shadow jar publishing for the clients module Jan 8, 2024
@stanislavkozlovski stanislavkozlovski merged commit 8a76c16 into apache:trunk Jan 8, 2024
stanislavkozlovski pushed a commit that referenced this pull request Jan 8, 2024
@apoorvmittal10 @stanislavkozlovski
MINOR: Fix shadow jar publishing for the clients module (#15127)
24e41ee 
The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar was recently shadowed which would publish the artifacts to the local maven repo successfully but would throw an error when publishing to remote maven. (as part of the release process)

The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.
gaurav-narula pushed a commit to gaurav-narula/kafka that referenced this pull request Jan 24, 2024
@apoorvmittal10 @gaurav-narula
MINOR: Fix shadow jar publishing for the clients module (apache#15127)
ab9ef7e 
The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar was recently shadowed which would publish the artifacts to the local maven repo successfully but would throw an error when publishing to remote maven. (as part of the release process)

The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.
This was referenced Jan 29, 2024
Closed
Merged
yyu1993 pushed a commit to yyu1993/kafka that referenced this pull request Feb 15, 2024
@apoorvmittal10 @yyu1993
MINOR: Fix shadow jar publishing for the clients module (apache#15127)
1fd1218 
The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar was recently shadowed which would publish the artifacts to the local maven repo successfully but would throw an error when publishing to remote maven. (as part of the release process)

The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.
AnatolyPopov pushed a commit to aiven/kafka that referenced this pull request Feb 16, 2024
@apoorvmittal10 @AnatolyPopov
MINOR: Fix shadow jar publishing for the clients module (apache#15127)
2cac4e1 
The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar was recently shadowed which would publish the artifacts to the local maven repo successfully but would throw an error when publishing to remote maven. (as part of the release process)

The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.
clolov pushed a commit to clolov/kafka that referenced this pull request Apr 5, 2024
@apoorvmittal10 @clolov
MINOR: Fix shadow jar publishing for the clients module (apache#15127)
7731a54 
The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar was recently shadowed which would publish the artifacts to the local maven repo successfully but would throw an error when publishing to remote maven. (as part of the release process)

The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.
Phuc-Hong-Tran pushed a commit to Phuc-Hong-Tran/kafka that referenced this pull request Jun 6, 2024
@apoorvmittal10
MINOR: Fix shadow jar publishing for the clients module (apache#15127)
a471007 
The PR fixes the publishing of kafka-clients artifact to remote maven. The kafka-clients jar was recently shadowed which would publish the artifacts to the local maven repo successfully but would throw an error when publishing to remote maven. (as part of the release process)

The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing. Generating signed asc files error out for shadowed release artifacts as the module name (clients) differs from the artifact name (kafka-clients).

The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin. project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar though the classifier and archivesBaseName are already defined correctly in :clients and shadowJar construction.
cmccabe added a commit to cmccabe/kafka that referenced this pull request Sep 17, 2024
@cmccabe
Revert "KAFKA-16803: Change fork, update ShadowJavaPlugin to 8.1.7 (a…
04469c4 
…pache#16295)"

This reverts commit 391778b.

Unfortunately that commit re-introduced bug apache#15127 which prevented the publishing of kafka-clients
artifacts to remote maven. As that bug says:

    The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing.
    Generating signed asc files error out for shadowed release artifacts as the module name
    (clients) differs from the artifact name (kafka-clients).

    The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin.
    project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar
    though the classifier and archivesBaseName are already defined correctly in :clients and
    shadowJar construction.
@cmccabe cmccabe mentioned this pull request Sep 17, 2024
Merged
cmccabe added a commit that referenced this pull request Sep 17, 2024
@cmccabe
Revert "KAFKA-16803: Change fork, update ShadowJavaPlugin to 8.1.7 (#…
389a8d8 
…16295)" (#17218)

This reverts commit 391778b.

Unfortunately that commit re-introduced bug #15127 which prevented the publishing of kafka-clients
artifacts to remote maven. As that bug says:

    The issue triggers only with publishMavenJavaPublicationToMavenRepository due to signing.
    Generating signed asc files error out for shadowed release artifacts as the module name
    (clients) differs from the artifact name (kafka-clients).

    The fix is basically to explicitly define artifact of shadowJar to signing and publish plugin.
    project.shadow.component(mavenJava) previously outputs the name as client-<version>-all.jar
    though the classifier and archivesBaseName are already defined correctly in :clients and
    shadowJar construction.

Reviewers: David Arthur <[email protected]>
@apoorvmittal10 apoorvmittal10 mentioned this pull request Feb 24, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@stanislavkozlovski stanislavkozlovski stanislavkozlovski approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@apoorvmittal10 @stanislavkozlovski