This module migrates configs specified in airbyte-config to new versions.
- Update the config json schema in
airbyte-config/models. - Add the changed json schema to the main resources.
- If a migration is needed, create a migration file under
io.airbyte.migrate.migrations. - Register the migration in
Migrations.java. - If needed, write a migration unit test under
io.airbyte.migrate.migrations. - Test the migration locally in IDE or commandline (see below).
Run MigrationRunner.java with arguments (--input, --output, --target-version).
Run the following command in project root:
# Get the current version
BUILD_VERSION=$(cat .env | grep VERSION | awk -F"=" '{print $2}')
# Build the migration bundle file
SUB_BUILD=PLATFORM ./gradlew airbyte-migration:build
# Extract the bundle file
tar xf ./airbyte-migration/build/distributions/airbyte-migration-${BUILD_VERSION}.tar --strip-components=1
# Run the migration
bin/airbyte-migration \
--input <input_config_archive.tar.gz> \
--output <output_config_archive.tar.gz>See MigrationRunner for details.
BUILD_VERSION=$(cat .env | grep VERSION | awk -F"=" '{print $2}')
INPUT_PATH=<path to directory containing downloaded airbyte_archive.tar.gz>
OUTPUT_PATH=<path to where migrated archive will be written (should end in .tar.gz)>
TARGET_VERSION=<version you are migrating to or empty for latest>
docker run --rm -v ${INPUT_PATH}:/config airbyte/migration:${BUILD_VERSION} -- \
--input /config/airbyte_archive.tar.gz \
--output ${OUTPUT_PATH} \
[ --target-version ${TARGET_VERSION} ]See Upgrading Airbyte for details.