Skip to content

Support TimestampTZ in Glue catalog#83132

Merged
scanhex12 merged 9 commits intoClickHouse:masterfrom
scanhex12:timestamps_glue
Jul 5, 2025
Merged

Support TimestampTZ in Glue catalog#83132
scanhex12 merged 9 commits intoClickHouse:masterfrom
scanhex12:timestamps_glue

Conversation

@scanhex12
Copy link
Member

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Support TimestampTZ in Glue catalog. This closes #81654

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@scanhex12 scanhex12 marked this pull request as draft July 3, 2025 10:51
@clickhouse-gh
Copy link
Contributor

clickhouse-gh bot commented Jul 3, 2025

Workflow [PR], commit [f8d2d84]

Summary:

job_name test_name status info comment
Stress test (amd_tsan) failure
Server died FAIL
Hung check failed, possible deadlock found (see hung_check.log) FAIL
Killed by signal (in clickhouse-server.log) FAIL
Fatal message in clickhouse-server.log (see fatal_messages.txt) FAIL
Killed by signal (output files) FAIL
Found signal in gdb.log FAIL

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Jul 3, 2025
@scanhex12 scanhex12 marked this pull request as ready for review July 3, 2025 12:07
@alesapin alesapin self-assigned this Jul 3, 2025
Copy link
Member

@alesapin alesapin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we need to fix build for specific platforms.

[7811/10308] Building CXX object src/CMakeFiles/dbms.dir/Databases/DataLake/GlueCatalog.cpp.o
FAILED: src/CMakeFiles/dbms.dir/Databases/DataLake/GlueCatalog.cpp.o 
prlimit --as=20000000000 --data=10000000000 --cpu=1800 /usr/bin/sccache /usr/bin/clang++-19 --target=loongarch64-linux-gnu --sysroot=/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/cmake/linux/../../contrib/sysroot/linux-loongarch64 -DBOOST_ASIO_HAS_STD_INVOKE_RESULT=1 -DBOOST_ASIO_STANDALONE=1 -DBOOST_TIMER_ENABLE_DEPRECATED=1 -DBSONCXX_STATIC -DCARES_STATICLIB -DCONFIGDIR=\"\" -DDUMMY_BACKTRACE -DENABLE_MULTITARGET_CODE=1 -DFIU_ENABLE -DHAVE_BZLIB_H=1 -DHAVE_CONFIG_H -DHAVE_FUTIMESAT=1 -DHAVE_ICONV=1 -DHAVE_LIBLZMA=1 -DHAVE_LIBZSTD=1 -DHAVE_LINUX_FS_H=1 -DHAVE_LINUX_TYPES_H=1 -DHAVE_LZMA_H=1 -DHAVE_STRUCT_STAT_ST_MTIM_TV_NSEC=1 -DHAVE_SYS_STATFS_H=1 -DHAVE_ZLIB_H=1 -DHAVE_ZSTD_H=1 -DHAVE_ZSTD_compressStream=1 -DHAVE_ZSTD_minCLevel=1 -DINCBIN_SILENCE_BITCODE_WARNING -DLIBSASL_EXPORTS=1 -DLZ4_FAST_DEC_LOOP=1 -DMAJOR_IN_SYSMACROS=1 -DMONGOCXX_STATIC -DOBSOLETE_CRAM_ATTR=1 -DOBSOLETE_DIGEST_ATTR=1 -DPLUGINDIR=\"\" -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSASLAUTHD_CONF_FILE_DEFAULT=\"\" -DSNAPPY_CODEC_AVAILABLE -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUSEARCH_USE_FP16LIB -DUSE_CLICKHOUSE_THREADS=1 -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DZLIB_COMPAT -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -D_LIBUNWIND_IS_NATIVE_ONLY -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/includes/configs -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/src -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/src -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/llvm-project/libcxx/include/c++/v1 -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/base/.. -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/base/base/.. -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/cctz/include -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/re2 -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/pcg-random/. -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libfiu/libfiu -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libssh/include -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/libssh/include -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/miniselect/include -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/zstd/lib -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/pocketfft -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libarchive-cmake -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libarchive/libarchive -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/cyrus-sasl-cmake -I/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/lz4/lib -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/llvm-project/libcxx/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/llvm-project/libcxxabi/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/llvm-project/libunwind/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libdivide-cmake/. -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libdivide -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/numactl -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/cityhash102/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/boost -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/Net/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/Foundation/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/NetSSL_OpenSSL/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/Util/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/JSON/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/XML/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/openssl-cmake/linux_loongarch64/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/openssl/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/replxx/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/incbin -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/fmtlib-cmake/../fmtlib/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/magic_enum/include/magic_enum -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/abseil-cpp -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/croaring/cpp -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/croaring/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/sparsehash-c11 -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/llvm-project/compiler-rt/lib -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/double-conversion -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/dragonbox/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/zlib-ng -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/zlib-ng-cmake -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/pdqsort -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/xz/src/liblzma/api -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/google-protobuf/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws/src/aws-cpp-sdk-core/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/aws-cmake/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws/generated/src/aws-cpp-sdk-s3/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws-c-auth/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws-c-common/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws-c-io/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws-crt-cpp/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws-c-mqtt/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws-c-sdkutils/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/aws/generated/src/aws-cpp-sdk-glue/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/azure/sdk/core/azure-core/inc -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/azure/sdk/identity/azure-identity/inc -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/azure/sdk/storage/azure-storage-common/inc -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/azure/sdk/storage/azure-storage-blobs/inc -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/snappy -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/snappy-cmake -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libbcrypt -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/msgpack-c/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/liburing/src/include-compat -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/liburing/src/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/liburing/src/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/fast_float/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/simdjson/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/librdkafka-cmake/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/librdkafka/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/librdkafka-cmake/auxdir -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/cppkafka/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/nats-io/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/nats-io/src/adapters -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/nats-io/src/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/nats-io/src/unix -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libuv/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/krb5/src/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/krb5-cmake/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/NuRaft/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/base/poco/Redis/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-c-driver/src/libbson/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/mongo-c-driver/src/libbson/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-c-driver/src/common/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/mongo-c-driver/src/common/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-c-driver/src/libmongoc/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/mongo-c-driver/src/libmongoc/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-c-driver/src/libmongoc/src/mongoc -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/mongo-c-driver/src/libmongoc/src/mongoc -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-c-driver/src/utf8proc-2.8.0 -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-c-driver/src/uthash -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/c-ares/src/lib/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/c-ares/src/lib -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/c-ares/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/c-ares-cmake/linux -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-cxx-driver/src/bsoncxx/include/bsoncxx/v_noabi -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-cxx-driver/src/bsoncxx/lib/bsoncxx/v_noabi -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/mongo-cxx-driver/src/bsoncxx/lib/bsoncxx/v_noabi -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-cxx-driver/src/mongocxx/include/mongocxx/v_noabi -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/mongo-cxx-driver/src/mongocxx/lib/mongocxx/v_noabi -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/mongo-cxx-driver/src/mongocxx/lib/mongocxx/v_noabi -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/icu/icu4c/source/i18n -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/icu/icu4c/source/common -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/capnproto/c++/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/avro/lang/c++/api -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/s2geometry/src -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/s2geometry-cmake -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/AMQP-CPP/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/AMQP-CPP -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/sqlite-amalgamation -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/rocksdb/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/datasketches-cpp/common/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/datasketches-cpp/count/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/datasketches-cpp/theta/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/libstemmer_c/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/wordnet-blast -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/lemmagen-c/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/ulid-c/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/rapidjson/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/consistent-hashing -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/usearch/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/FP16/include -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/contrib/prometheus-protobufs-cmake -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/contrib/SHA3IUF -isystem /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/build/src/Client/BuzzHouse/Proto -mcmodel=extreme -mcmodel=extreme --gcc-toolchain=/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/cmake/linux/../../contrib/sysroot/linux-loongarch64/usr -fdiagnostics-color=always -Xclang -fuse-ctor-homing -fsized-deallocation  -gdwarf-aranges -pipe -fasynchronous-unwind-tables -ffile-prefix-map=/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse=. -ftime-trace -falign-functions=32 -ffp-contract=off  -fdiagnostics-absolute-paths -fstrict-vtable-pointers -Wall -Wextra -Wframe-larger-than=65536 -Weverything -Wpedantic -Wvla-cxx-extension -Wno-return-type-c-linkage -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -Wno-unsafe-buffer-usage -Wno-switch-default -O2 -g -DNDEBUG -O3 -g  -fno-pie -std=c++23   -D OS_LINUX -Werror -Wno-deprecated-declarations -Wno-poison-system-directories -nostdinc++ -MD -MT src/CMakeFiles/dbms.dir/Databases/DataLake/GlueCatalog.cpp.o -MF src/CMakeFiles/dbms.dir/Databases/DataLake/GlueCatalog.cpp.o.d -o src/CMakeFiles/dbms.dir/Databases/DataLake/GlueCatalog.cpp.o -c /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/src/Databases/DataLake/GlueCatalog.cpp
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/src/Databases/DataLake/GlueCatalog.cpp:435:51: error: no member named 'StorageS3DeltaLakeConfiguration' in namespace 'DB'
  435 |         auto configuration = std::make_shared<DB::StorageS3DeltaLakeConfiguration>(storage_settings);
      |                                               ~~~~^
1 error generated.

@scanhex12 scanhex12 requested a review from alesapin July 3, 2025 18:58
@scanhex12 scanhex12 added this pull request to the merge queue Jul 5, 2025
Merged via the queue into ClickHouse:master with commit 711b95b Jul 5, 2025
119 of 122 checks passed
@scanhex12 scanhex12 deleted the timestamps_glue branch July 5, 2025 11:09
@robot-clickhouse robot-clickhouse added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 5, 2025
Enmk pushed a commit to Altinity/ClickHouse that referenced this pull request Aug 29, 2025
Enmk added a commit to Altinity/ClickHouse that referenced this pull request Sep 3, 2025
…Z_in_glue

Antalya 25.6: Backport of ClickHouse#83132 - Support TimestampTZ in Glue catalog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Timezone info lost when reading DateTime columns from Glue DataLakeCatalog

4 participants