Skip to content

Add sequence id to iceberg metadata and iceberg avro parsing refactoring#75010

Merged
divanik merged 16 commits intomasterfrom
divanik/add_sequence_id_and_implementation_refactoring
Feb 27, 2025
Merged

Add sequence id to iceberg metadata and iceberg avro parsing refactoring#75010
divanik merged 16 commits intomasterfrom
divanik/add_sequence_id_and_implementation_refactoring

Conversation

@divanik
Copy link
Member

@divanik divanik commented Jan 24, 2025

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Implemented parsing enhancements (Sequence ID parsing: Added functionality to parse sequence identifiers in manifest files AND Avro metadata parsing: Redesigned the Avro metadata parser to be easily extendable for future enhancements)

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

CI Settings (Only check the boxes if you know what you are doing)

All builds in Builds_1 and Builds_2 stages are always mandatory and will run independently of the checks below:

  • Only: Stateless tests
  • Only: Integration tests
  • Only: Performance tests

  • Skip: Style check
  • Skip: Fast test

  • Run all checks ignoring all possible failures (Resource-intensive. All test jobs execute in parallel).
  • Disable CI cache

@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-feature Pull request with new product feature label Jan 24, 2025
@robot-ch-test-poll robot-ch-test-poll added pr-improvement Pull request with some product improvements and removed pr-feature Pull request with new product feature labels Jan 28, 2025
@robot-clickhouse-ci-2
Copy link
Contributor

robot-clickhouse-ci-2 commented Jan 28, 2025

This is an automated comment for commit 1d9bab3 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check nameDescriptionStatus
Style checkRuns a set of checks to keep the code style clean. If some of tests failed, see the related log from the report❌ failure
Successful checks
Check nameDescriptionStatus
Docs checkBuilds and tests the documentation✅ success
Fast testNormally this is the first check that is ran for a PR. It builds ClickHouse and runs most of stateless functional tests, omitting some. If it fails, further checks are not started until it is fixed. Look at the report to see which tests fail, then reproduce the failure locally as described here✅ success

@hanfei1991 hanfei1991 self-assigned this Feb 20, 2025
@clickhouse-gh
Copy link
Contributor

clickhouse-gh bot commented Feb 24, 2025

Workflow [PR], commit [d314c53]

@divanik divanik marked this pull request as ready for review February 24, 2025 16:39
@divanik divanik changed the title [[Draft]] Add sequence id to iceberg metadata and iceberg avro parsing refactoring Add sequence id to iceberg metadata and iceberg avro parsing refactoring Feb 24, 2025
Copy link
Member

@hanfei1991 hanfei1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we going to add some tests?

@divanik
Copy link
Member Author

divanik commented Feb 26, 2025

Are we going to add some tests?

I don't think so. I think there is no need to add tests for refactoring part. For part with sequence_number I also don't know how to add tests without checking internal structure of Iceberg which is not desirable, imho

@divanik divanik added this pull request to the merge queue Feb 27, 2025
Merged via the queue into master with commit 495e480 Feb 27, 2025
124 checks passed
@divanik divanik deleted the divanik/add_sequence_id_and_implementation_refactoring branch February 27, 2025 12:54
@robot-clickhouse robot-clickhouse added the pr-synced-to-cloud The PR is synced to the cloud repo label Feb 27, 2025
ianton-ru pushed a commit to Altinity/ClickHouse that referenced this pull request Apr 11, 2025
…nce_id_and_implementation_refactoring

Add sequence id to iceberg metadata and iceberg avro parsing refactoring
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants