Skip to content

Avro: allow_missing_fields setting#12007

Merged
tavplubix merged 2 commits intoClickHouse:masterfrom
oandrew:avro-allow-missing-fields
Jun 29, 2020
Merged

Avro: allow_missing_fields setting#12007
tavplubix merged 2 commits intoClickHouse:masterfrom
oandrew:avro-allow-missing-fields

Conversation

@oandrew
Copy link
Contributor

@oandrew oandrew commented Jun 28, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add setting to fields not found in Avro schema

Detailed description / Documentation draft:
Add setting input_format_avro_allow_missing_fields to allow reading fields not found in Avro schema (using default value instead).
Useful for Kafka + schema evolution when old messages are missing newly added field in their schema and preventing ClickHouse from making progress.
Related to #11986

Not using input_format_defaults_for_omitted_fields because it defaults to true.

By adding documentation, you'll allow users to try your new feature immediately, not when someone else will have time to document it later. Documentation is necessary for all features that affect user experience in any way. You can add brief documentation draft above, or add documentation right into your patch as Markdown files in docs folder.

If you are doing this for the first time, it's recommended to read the lightweight Contributing to ClickHouse Documentation guide first.

@blinkov blinkov added doc-alert pr-feature Pull request with new product feature labels Jun 28, 2020
@tavplubix tavplubix self-assigned this Jun 28, 2020
@tavplubix
Copy link
Member

Could you please provide a test?

@oandrew
Copy link
Contributor Author

oandrew commented Jun 28, 2020

Failed test does not seem to be related (mysql)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature Pull request with new product feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants