…3d9c5450be967f7fed646d0ce3129

Cherry pick #12165 to 20.5: Fix splitting table name of dictionary source

#12259)

* simple changelog script

* bump CI

* Fix tests.

* Add explicit test for a case where AST hashes collide for different prepared sets

* [blog] add RSS feed (#12064)

* [blog] add rss feed

* better title

* Update CHANGELOG.md

Co-authored-by: Ivan Blinkov <[email protected]>

* bump ci

* Moved useless S3 logging to TRACE level.

* Add a test to cover non-const tuple elemenets (just in case)

* Bump

* Update query-complexity.md

Remove a note about read limits applied on threads level.

* Update query-complexity.md

* Update query-complexity.md

* Update query-complexity.md

* DOCS-609: max_server_memory_usage (#11771)

* Revolg DOCSUP-1000 add max server memory usage setting (#125)

* Add max_server_memory_usage setting, delete max_memory_usage_for_all_queries setting.

* Syntax fixed

* Apply suggestions from code review

Co-authored-by: BayoNet <[email protected]>

* Doc for the max_server_memory_usage setting. Updates.

Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-609: Minor fixes.

* CLICKHOUSEDOCS-609: Actualized position of the setting.

Co-authored-by: olgarev <[email protected]>
Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>

* Add force_primary_key to a pk in tuple test

* DOCS-510: runningAccumulate (#12061)

* asiana21-DOCSUP-797 (#117)

* docs(runningAccumulate): the function description is added

* docs(runningAccumulate): the function description is modified

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* docs(runningAccumulate): some changes

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* docs(runningAccumulate): added ru translation

Co-authored-by: asiana21 <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-510: Minor fix.

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: Ivan Blinkov <[email protected]>

* CLICKHOUSEDOCS-510: Fixed links.

Co-authored-by: AsiaKorushkina <[email protected]>
Co-authored-by: asiana21 <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: Ivan Blinkov <[email protected]>

* Remove const specifier to allow auto-move (clangtidy)

* Rewrite Set lookup to make it more readable

* [docs] Sync zh/development/build-osx.md from EN (#12071)

* ISSUES-4006 support first for ALTER ADD|MODIFY COLUMN

* Update deb image

* Don't download image twice

* Remove garbage from images

* bump ci

* Changelog for 20.1, 20.4

* fixup

* DOCS-635: Translated the EN version of the AvroConfluent format description (#11930)

* DOCSUP-1350 (#128)

* edited EN version

* add EN and RU translation

* minor changes

* CLICKHOUSEDOCS-635: Updated the description.

Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: Evgenia Sudarikova <[email protected]>

* Fix tests.

* Better PVS image

* DOCS-605: Description for the always_fetch_merged_part setting (#11921)

* Revolg DOCSUP-998 Document the always_fetch_merged_part setting (#123)

* Add always_fetch_merged_part setting

* revolg-DOCSUP-998-add_always_fetch_merged_part_setting link fixed

* Apply suggestions from code review

Co-authored-by: BayoNet <[email protected]>

* Add always_fetch_merged_part setting. Updates.

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* Add always_fetch_merged_part setting. Updates.

Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-605: Minor fixes.

* CLICKHOUSEDOCS-605: Added Plausible to Adopters.

* Update docs/ru/operations/settings/settings.md

Co-authored-by: alesapin <[email protected]>

* Update docs/en/operations/settings/settings.md

Co-authored-by: alesapin <[email protected]>

* CLICKHOUSEDOCS-605: Fixed access rights description.

Co-authored-by: olgarev <[email protected]>
Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: alesapin <[email protected]>

* Update Dockerfile

* revert e9d3c48

* [blog] 'Package Repository Behind CDN' post (#12082)

* support iframes from DataLens

* initial blog post text

* Improve REVOKE command: now it requires only grant/admin option for only
access which will be revoked.
REVOKE ALL FROM user1 now revokes all granted roles.

* Fix limiting the number of threads for VIEW.

* Update tips.md

* Added test.

* Remove pvs studio from images list

* Better shutdown and conversion

* Reverse arguments

* Fix result_rows and result_bytes metrics for selects.

* Fix result_rows and result_bytes metrics for selects.

* improve breadcrumbs markup

* Fix tests.

* fix segfault with -StateResample combinators

* Less race conditions

* fix test with buffer table

* [docker] install ca-certificates before the first apt-get update (#12095)

* [docker] install ca-certificates before first apt-get update

* Update Dockerfile

* DOCS-522: max_parser_depth (#12097)

* asiana21-DOCSUP-925-max_parser_depth (#132)

* docs(max_parser_depth): added the setting description

* docs(max_parser_depth): some changes

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* docs(max_parser_depth): added ru translation

* docs(max_parser_depth): removed quotation marks

Co-authored-by: asiana21 <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-522: Fixed the link.

Co-authored-by: AsiaKorushkina <[email protected]>
Co-authored-by: asiana21 <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>

* Rewrite curl dependency in a more ch compatible way

- add support of unbundled curl
- add CURL::libcurl libraries
- avoid explicit linkage of daemon with curl (added with sentry)
- set CACHE variables for non-direct users:
  - mariadb-connector-c
  - aws-s3-cmake
  - sentry-native

Cc: @alexey-milovidov
Cc: @alesapin (requires docker image update)
Cc: @abyss7

Refs: #11300
Refs: #8011
Refs: #8905

v2: replace cmake/find/curl.cmake with proper contrib/curl-cmake (as
pointed by @abyss7, cmake/find/*.cmake is deprecated)

* Added results for AWS Lightsail

* (typo) in doc (#12099)

* Update polymorphic_parts_l.xml

* Update polymorphic_parts_m.xml

* Update polymorphic_parts_s.xml

* kafka: fix SIGSEGV on DROP TABLE

After #11599 it is possible that messages of the
ReadBufferFromKafkaConsumer will be cleaned-up right in
read_kafka_message callback (from KafkaBlockInputStream) if the stop
flag isset (i.e. DROP TABLE is waiting the consumer), and if
read_kafka_message already processed some rows it will not return 0 and
the loop after will try to get current topic from the buffer, which uses
messages in the underlying and this will got SIGSEGV:

    12:14:56.173262 [ 55421 ] {f7930856-d478-4e41-af56-24ce7b693e95} <Debug> executeQuery: (from 0.0.0.0:0, user: ) DROP TABLE IF EXISTS data.queue
    12:14:56.173285 [ 55421 ] {f7930856-d478-4e41-af56-24ce7b693e95} <Trace> StorageKafka (newly_queue): Waiting for cleanup
    12:14:56.180016 [ 55390 ] {} <Trace> BaseDaemon: Received signal 11
    12:14:56.180267 [ 4914 ] {} <Fatal> BaseDaemon: ########################################
    12:14:56.181879 [ 4914 ] {} <Fatal> BaseDaemon: (version 20.6.1.1, build id: 4CE0298F08583658) (from thread 55468) (no query) Received signal Segmentation fault (11)
    12:14:56.181900 [ 4914 ] {} <Fatal> BaseDaemon: Address: 0x8 Access: read. Address not mapped to object.
    12:14:56.181909 [ 4914 ] {} <Fatal> BaseDaemon: Stack trace:
    12:14:56.184676 [ 4914 ] {} <Fatal> BaseDaemon: 3. /ch/contrib/cppkafka/include/cppkafka/message.h:111: DB::KafkaBlockInputStream::readImpl() @ 0xe343f1c in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.185553 [ 4914 ] {} <Fatal> BaseDaemon: 4. /ch/contrib/libcxx/include/vector:1003: DB::IBlockInputStream::read() @ 0xd9d95bd in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.188238 [ 4914 ] {} <Fatal> BaseDaemon: 5. /ch/src/DataStreams/copyData.cpp:26: DB::copyData() @ 0xd9f712a in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.188780 [ 4914 ] {} <Fatal> BaseDaemon: 6. /ch/contrib/libcxx/include/vector:1532: DB::StorageKafka::streamToViews() @ 0xe335e73 in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.189331 [ 4914 ] {} <Fatal> BaseDaemon: 7. /ch/src/Storages/Kafka/StorageKafka.cpp:491: DB::StorageKafka::threadFunc() @ 0xe336738 in /usr/lib/debug/usr/bin/clickhouse

55421 thread (shows that it still waiting for deactivation):

    5  std::__1::lock_guard<>::lock_guard () at ../contrib/libcxx/include/__mutex_base:90
    6  DB::BackgroundSchedulePoolTaskInfo::deactivate (this=0x7fc7e4465f20) at ../src/Core/BackgroundSchedulePool.cpp:59
    7  DB::StorageKafka::shutdown (this=0x7fc7e45e4600) at ../contrib/libcxx/include/memory:3821

And just in case thread where read_kafka_message is called:

    0  DB::ReadBufferFromKafkaConsumer::nextImpl (this=0x7fd4901d4118) at ../contrib/libcxx/include/atomic:1491
    1  DB::ReadBuffer::next (this=0x7fd4901d4118) at ../src/IO/ReadBuffer.h:59
    2  DB::ReadBuffer::eof (this=0x7fd4901d4118) at ../src/IO/ReadBuffer.h:81
    3  DB::skipWhitespaceIfAny (buf=...) at ../src/IO/ReadHelpers.h:945
    4  DB::JSONEachRowRowInputFormat::readRow (ext=..., columns=..., this=0x7fd499a7a020) at ../src/Processors/Formats/Impl/JSONEachRowRowInputFormat.cpp:222
    5  DB::JSONEachRowRowInputFormat::readRow (this=0x7fd499a7a020, columns=..., ext=...) at ../src/Processors/Formats/Impl/JSONEachRowRowInputFormat.cpp:218
    6  DB::IRowInputFormat::generate (this=0x7fd499a7a020) at ../src/Processors/Formats/IRowInputFormat.cpp:64
    7  DB::ISource::work (this=0x7fd499a7a020) at ../src/Processors/ISource.cpp:48
    8  DB::KafkaBlockInputStream::<lambda()>::operator()(void) const () at ../contrib/libcxx/include/memory:3826
    9  DB::KafkaBlockInputStream::readImpl (this=0x7fd46e718820) at ../contrib/libcxx/include/new:340

Cc: @filimonov

* kafka: check that the data is still usable after parsing

* kafka: improve logging during engine shutdown

This will help with tracking possible issues, when you need to know was
buffer released or not.

* kafka: avoid superior polling after DROP/DETACH TABLE

Before this patch isStalled() was checked before polledDataUnusable(),
and after DROP TABLE isStalled() == true (although this looks tricky).

* kafka: remove outdated comment

As stated by @filimonov it is not relevant (after #11599)

* Added test.

* Create codeql-analysis.yml

* Tiny fixes

* Create anchore-analysis.yml

* Update polymorphic_parts_s.xml

* Update polymorphic_parts_m.xml

* Update polymorphic_parts_l.xml

* Changelog for 20.5

* Update CHANGELOG.md

* Update README.md

* Fix test (#12088)

* Test for a fixed issue #10668, related to input_format_allow_errors_num in CSV

* Add unbundled mode flag

* Remove LC converting to Arrow.

* Remove LC converting to Arrow.

* Move skip lists to clickhouse-test

* Remove LC converting to Arrow.

* More verbose message about skip

* Make skip-list optional

* Added test.

* Create initializeAggregation to initialize an aggregation function based on a value

* Add type column in system.disks

* Update clickhouse-test

* fix test with UBSan

* Add missing query context for system logs

Needed to allow attaching materialized views with joins or with
subqueries to system.logs.

* Fix handling dependency of table with ENGINE=Dictionary on dictionary.

* Add test.

* Update 01355_CSV_input_format_allow_errors.sh

* Update CHANGELOG.md

* Update Dockerfile

* Added a showcase of minimal Docker image

* Added a comment

* Added a comment

* Added a comment

* Formatting

* contrib/unixodbc-cmake: Fix build when UNBUNDLED

target_compile_definitions may only set INTERFACE properties on IMPORTED targets

* Fix version column in replicated version collapsing merge tree (#12121)

* poco-cmake: Fix Poco::Data::ODBC target when UNBUNDLED

By default IMPORTED target has a scope in the directory in which it is created
and below. This leads to the following issues when building UNBUNDLED:

Target "clickhouse" links to target "Poco::Data::ODBC" but the target was not
found.  Perhaps a find_package() call is missing for an IMPORTED target, or an
ALIAS target is missing?

* Add missed <atomic> include in ProxyListConfiguration

* Improve performace of reading in order of sorting key. (#11696)

* simplify reading in order of sorting key

* add perf test for reading many parts

* Revert "simplify reading in order of sorting key"

This reverts commit 7267d7c.

* add threshold for preliminary merge for reading in order

* better threshold

* limit threads in test

* Do not enable sentry if ENABLE_LIBRARIES is not set

* Fail if curl library was enabled and was not found in case of unbundled build

* Normalize "pid" file handling #3501

* Update StatusFile.cpp

* New ISO8601 year modificators for formatDateTime

* Added a test

* Include libcurl4-openssl-dev into yandex/clickhouse-deb-builder

* Remove harmful code from "geoDistance" #12117

* Update formatDateTime.cpp

* Fix warnings from CodeQL

* Added a test

* Minor modification

* Update libdivide to the latest master

* Replace exit to abort in libdivide

* Fix potential overflow in integer division #12119

* Added a test

* Whitespace

* Added a test for #4211

* Attempt to fix "Arcadia" build

* Rename test

* Update arraySum.cpp

* Run perf tests with memory sampling (for allocations >1M)

This is to know the memory allocation size distribution, that can be
obtained later from left-metric-log.tsv.

This is an attempt to tune tcmalloc (new CPP version by google) to use
lock-free part of the allocator for typical allocations (and it is a bad
idea just to increase kMaxSize there, since number of allocation for
each size class is also important).

P.S. hope that this file will be applied, if no, then the same effect
can be reached by tunning defaults in Settings.h

Refs: #11590
Cc: @akuzm

* Make code clearer: use enum instead of `bool internal`.

* Update StorageDictionary.h

* Update StorageDictionary.h

* ILIKE operator (#12125)

* Integrated CachingAllocator into MarkCache

* fixed build errors

* reset func hotfix

* upd: Fixing build

* updated submodules links

* fix 2

* updating grabber allocator proto

* updating lost work

* updating CMake to use concepts

* some other changes to get it building (integration into MarkCache)

* further integration into caches

* updated Async metrics, fixed some build errors

* and some other errors revealing

* added perfect forwarding to some functions

* fix: forward template

* fix: constexpr modifier

* fix: FakePODAllocator missing member func

* updated PODArray constructor taking alloc params

* fix: PODArray overload with n restored

* fix: FakePODAlloc duplicating alloc() func

* added constexpr variable for alloc_tag_t

* split cache values by allocators, provided updates

* fix: memcpy

* fix: constexpr modifier

* fix: noexcept modifier

* fix: alloc_tag_t for PODArray constructor

* fix: PODArray copy ctor with different alloc

* fix: resize() signature

* updating to lastest working master

* syncing with 273267

* first draft version

* fix: update Searcher to case-insensitive

* added ILIKE test

* fixed style errors, updated test, split like and ilike,  added notILike

* replaced inconsistent comments

* fixed show tables ilike

* updated missing test cases

* regenerated ya.make

* Update 01355_ilike.sql

Co-authored-by: myrrc <[email protected]>
Co-authored-by: alexey-milovidov <[email protected]>

* add parseDateTimeBestEffortUS function (#12028)

* add function parseDateTimeBestEffortUS
* add test
* add doc

Co-authored-by: alexey-milovidov <[email protected]>

* Cleanup changelog (half done). Now it is acceptable #12104

* Cleanup changelog (half done). Now it is more acceptable #12104

* Cleanup changelog (half done). Now it is more acceptable #12104

* Cleanup changelog (half done). Additions requested by @filimonov #12104

* Remove underscore as word-break character. This partially reverts #11975

* Fix bad test number

* Added a test

* Change exception code from LOGICAL_ERROR to BAD_ARGUMENTS when the name of remote table is empty

* fix segfault with -StateResample combinators

* Fix bad code in redundant ORDER BY optimization #10067

* Add a test

* Support KILL QUERY [connection_id] for MySQL

* Fix error

* Autocomplete does not have to work in "Unbundled" build

* Whitespace

* Add MySQL to ClickHouse query replacement mapping table

* Style

* update ya.make

* Added function "hasThreadFuzzer"

* Fix test under ThreadFuzzer

* Test for issue #9088

ALTER DELETE unexpectedly deletes NULL rows

* Remove unused potentially dangerous function

* Fix mutations interpreter #9088

* Added a test

* Added another test just in case

* Added yet another test just in case

* Update 01358_mutation_delete_null_rows.sql

* Fix test

* Fix flaky test

* Attempt to fix flaky test 00721_force_by_identical_result_after_merge_zookeeper

* [anchore] more detailed Dockerfile scan reports (#12159)

* Update Dockerfile

* Update array-functions.md (#12130)

修改编码错误

* Add integration test for mysql replacement query

* Update array-functions.md (#12129)

修改编码错误

* fix ubsan final

* [docs] improve redirects destination

* Update SECURITY.md (#12161)

* Update Dockerfile

* add cluster() adn clusterAllReplicas() table functions description, add signatures to remoteSecure() table function (#12156)

* add cluster() adn clusterAllReplicas() table functions description, add signatures to remoteSecure() table function

Signed-off-by: Slach <[email protected]>

* small russian fixes

Signed-off-by: Slach <[email protected]>

* Update docs/en/sql-reference/table-functions/cluster.md

Co-authored-by: Ivan Blinkov <[email protected]>

* Update docs/en/sql-reference/table-functions/cluster.md

Co-authored-by: Ivan Blinkov <[email protected]>

* Update docs/en/sql-reference/table-functions/cluster.md

Co-authored-by: Ivan Blinkov <[email protected]>

* Update docs/en/sql-reference/table-functions/cluster.md

Co-authored-by: Ivan Blinkov <[email protected]>

* Update docs/en/sql-reference/table-functions/cluster.md

Co-authored-by: Ivan Blinkov <[email protected]>

* Update docs/en/sql-reference/table-functions/cluster.md

Co-authored-by: Ivan Blinkov <[email protected]>

Co-authored-by: Ivan Blinkov <[email protected]>

* Remove -v from ninja

* [docs] introduction for integration table engines (#12167)

* [docs] introduction for integration table engines

* Update jdbc.md

* Update odbc.md

* Update mysql.md

* Update kafka.md

* Update hdfs.md

* Fix #10437, CR fixes

* Same change for Kafka - just in case, and to make it conform.

* [docs] add redirect from an introduction index page (#12176)

* [website] add apple-touch-icon (#12164)

* Better assert

* Don't split dictionary source's table name into schema and table name itself
if ODBC driver doesn't support schema.

* Sync reference file with changes in sql file

* [docs] introduction for third-party interfaces (#12175)

* [docs] introduction for third-party interfaces

* Update index.md

* Update index.md

* [docs] introduction for special table engines (#12170)

* [docs] introduction for integration table engines

* Update jdbc.md

* Update odbc.md

* Update mysql.md

* Update kafka.md

* Update hdfs.md

* [docs] introduction for special table engines

* Update index.md

* Update index.md

Co-authored-by: Alexander Kuzmenkov <[email protected]>
Co-authored-by: alesapin <[email protected]>
Co-authored-by: Nikolai Kochetov <[email protected]>
Co-authored-by: Nicolae Vartolomei <[email protected]>
Co-authored-by: alexey-milovidov <[email protected]>
Co-authored-by: Ivan Blinkov <[email protected]>
Co-authored-by: Alexander Kuzmenkov <[email protected]>
Co-authored-by: Vladimir Chebotarev <[email protected]>
Co-authored-by: filimonov <[email protected]>
Co-authored-by: BayoNet <[email protected]>
Co-authored-by: olgarev <[email protected]>
Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: AsiaKorushkina <[email protected]>
Co-authored-by: asiana21 <[email protected]>
Co-authored-by: BohuTANG <[email protected]>
Co-authored-by: Vitaly Baranov <[email protected]>
Co-authored-by: zhang2014 <[email protected]>
Co-authored-by: Evgenia Sudarikova <[email protected]>
Co-authored-by: Alexander Kazakov <[email protected]>
Co-authored-by: Anton Popov <[email protected]>
Co-authored-by: Azat Khuzhin <[email protected]>
Co-authored-by: manmitya <[email protected]>
Co-authored-by: Nikolai Kochetov <[email protected]>
Co-authored-by: Mikhail Filimonov <[email protected]>
Co-authored-by: Guillaume Tassery <[email protected]>
Co-authored-by: Anton Ivashkin <[email protected]>
Co-authored-by: Ilya Yatsishin <[email protected]>
Co-authored-by: Nikita Mikhaylov <[email protected]>
Co-authored-by: tavplubix <[email protected]>
Co-authored-by: tavplubix <[email protected]>
Co-authored-by: Matwey V. Kornilov <[email protected]>
Co-authored-by: vivarum <[email protected]>
Co-authored-by: myrrc <[email protected]>
Co-authored-by: myrrc <[email protected]>
Co-authored-by: flynn <[email protected]>
Co-authored-by: yhgcn <[email protected]>
Co-authored-by: Eugene Klimov <[email protected]>

…able-name

Fix splitting table name of dictionary source

(cherry picked from commit be85f9f)