…cb812b60c7080a607d008ba24c8e4

Cherry pick #12133 to 20.5: Normalize "pid" file handling #3501

* Update replxx submodule

* Add a test for #10102

* disable tests in arcadia

* Added a test from #5131

* Try to enable long perf tests

* perf test: report queries with new functions separately

* performance comparison

* longer

* muzzy decay 10 ms

* Support Enums type for MySQL engine #3985

* performance comparison

* Add integration test for MySQL enums type

* Fix bad log message at server startup

* Insert enum column values for test

* Better diagnostics of "Replica {} appears to be already active" message

* Fix typos

* Added a test for #8550

* Update extended-roadmap.md

* Update test

* Fix FPE, step 1

* Make it more correct

* Add a test

* Avoid floating point error in geohashesInBox #11369

* Avro: allow missing fields

* update

* Fix build

* Revert "[experiment] maybe fix warnings in integration tests"

* Add xeus-clickhouse (#12010)

xeus-clickhouse is a Jupyter kernal for ClickHouse

* Fix test

* Fix array size overflow in generateRandom

* Added a test

* Update ErrorCodes.cpp

* More hardening

* More hardening

* Move reading from RabbitMQ into background task

* Fix access rights: cannot grant DDL when allow_ddl=0

* Add tests

* Add test test_allow_ddl.

* Added failing test

* Remove the usage of obsolete setting

* Whitespace

* Added a test

* Allow to ALTER partition key in some cases

* Slightly more safe

* Only allow Enum extension within the same width

* Update test

* Added failing test for Replicated table

* Fix clang-tidy

* Whitespace

* Write partition key changes to ZooKeeper

* Typos

* Implement for ReplicatedMergeTree

* Added test reference

* Added missing ORDER BY in test

* Fix integration test

* Fix error in addMonths with floating point argument

* Fix error in addMonths with floating point argument

* Fix overflow in function "neighbor"

* Whitespace

* Whitespace

* Fix ORDER BY tuple with COLLATE on const column

* clickhouse-local: logging flags fixes

* clickhouse-local: use std::filesystem::equivalent

* Limit /proc/cpuinfo metrics to linux only

* Feature 8074 (#12014)

* refs #8074 1: start to stream

* refs final check format and spell

* refs final check format and spell

* refs #8074 fix up link to sql-reference/functions/other-functions

* Bump idna from 2.9 to 2.10 in /docs/tools

Bumps [idna](https://github.com/kjd/idna) from 2.9 to 2.10.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](kjd/idna@v2.9...v2.10)

Signed-off-by: dependabot-preview[bot] <[email protected]>

* make a short test for DateTime

* Fixup

* DOCS-636 Synchronized some edits from EN to RU (#11941)

* [review][clickhousedocs][translate] DOCSUP-1351: Edit and translate to Russian (#129)

* DOCSUP-1351: Add blockSerializedSize (RU)

* DOCSUP-1351: Fix blockSerializedSize description by function description template.

* DOCSUP-1351: PR8756- Update Distributed engine section.

* DOCSUP-1351: PR9076-Show Queries more like MySQL did.

* DOCSUP-1351: PR#9919-system.tables improvements (total_rows/total_bytes/storage_policy)

* DOCSUP-1351: (ru) blockSerializedSize remake by template.

* DOCSUP-1351: PR#9598-Add function arrayReduceInRange and update arrayReduce function.

* DOCSUP-1351: PR#9598 Fix arrayReduce and arrayReduceInRanges sections by template.

* DOCSUP-1351: Some fixes.

* DOCSUP-1351: Some fixes.

Co-authored-by: romanzhukov <[email protected]>

* CLICKHOUSEDOCS-636: Minor changes.

Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: Roman Bug <[email protected]>
Co-authored-by: romanzhukov <[email protected]>

* Use ConcurentBoundedQueue instead of vector

* Temporary enable debug logs when releasing docs

* Better docker images list

* [docs] improve stable releases discovery

* Update contrib/jemalloc-cmake/CMakeLists.txt

* Remove libevent library

* Automated backporting machinery (#12029)

* Add automated cherry-pick machinery

* Added script for backporting machinery

* Implement dry-run mode

* Comment old methods

* Fix python2

* Add ya.make for sandbox tasks

* Remove old stuff

* Move writing to RabbitMQ into background task

* Add TLB misses perf counters.

* Update ThreadProfileEvents.cpp

* Fake change in dockerfile

* Fix tuple nullable comparison.

* Fix access rights: cannot grant INTROSPECTION when allow_introspection_functions=0.

* fixup

* Update ThreadProfileEvents.cpp

* Fix test.

* Added test.

* Fix and simplify code

* Fix test.

* Update ITransformingStep.h

* Fix "Sharding key is not deterministic" message

* Do not check *optimize_skip_unused_shards_nesting w/o *optimize_skip_unused_shards

This will avoid supperior log message in case of
*optimize_skip_unused_shards_nesting already disables it.
And also it is logical.

* Remove test that is not supported by "Arcadia" build system

* Make topK return Enum for Enum types

* parse metadata in parallel when loading tables

* Fix build

* fixup

* Update ThreadProfileEvents.cpp

* Update browse-code.md (#12047)

* Update browse-code.md

纠正了一些翻译错误。

* Update browse-code.md

Co-authored-by: Ivan Blinkov <[email protected]>

* Bump python-slugify from 1.2.6 to 4.0.1 in /docs/tools

Bumps [python-slugify](https://github.com/un33k/python-slugify) from 1.2.6 to 4.0.1.
- [Release notes](https://github.com/un33k/python-slugify/releases)
- [Changelog](https://github.com/un33k/python-slugify/blob/master/CHANGELOG.md)
- [Commits](un33k/python-slugify@1.2.6...4.0.1)

Signed-off-by: dependabot-preview[bot] <[email protected]>

* Update ThreadProfileEvents.cpp

* Update README.md

* Fix typo (#12046)

* Fix test_allow_introspection.

* Try fix labda tuple(LC) argument.

* Test image changes

* Try fix labda tuple(LC) argument.

* Add pvs studio to images

* Added test.

* fix low card types in merge join (#12035)

* Update contrib/poco to ClickHouse-Extras/poco #22 (#12037)

* Updated ClickHouse-Extras/poco #22

* Fix defaultValueOfArgumentType

* Added test.

* Update cpu_synthetic.xml

* Moves task shall be started if new storage policy needs them.

* Added test for automatic start of background move task.

* Add ability to run any image version from packager and runner

* Try fix pk in tuple performance

Possible approach for fixing #10574

The problem is that prepared sets are built correctly, it is a hash map of key -> set
where key is a hash of AST and list of data types (when we a list of
tuples of literals).

However, when the key is built from the index to try and find if there
exists a prepared set that would match it looks for data types of the
primary key (see how data_types is populated) because the primary key
has only one field (v in my example) it can not find the prepared set.

The patch looks for any prepared indexes where data types match for the
subset of fields found in primary key, we are not interested in other
fields anyway for the purpose of primary key pruning.

* Fix partial revokes (complex cases).

* Changelog for 20.3.11, 12

* simple changelog script

* bump CI

* Add explicit test for a case where AST hashes collide for different prepared sets

* [blog] add RSS feed (#12064)

* [blog] add rss feed

* better title

* Update CHANGELOG.md

Co-authored-by: Ivan Blinkov <[email protected]>

* bump ci

* Moved useless S3 logging to TRACE level.

* Add a test to cover non-const tuple elemenets (just in case)

* Bump

* Update query-complexity.md

Remove a note about read limits applied on threads level.

* Update query-complexity.md

* Update query-complexity.md

* Update query-complexity.md

* DOCS-609: max_server_memory_usage (#11771)

* Revolg DOCSUP-1000 add max server memory usage setting (#125)

* Add max_server_memory_usage setting, delete max_memory_usage_for_all_queries setting.

* Syntax fixed

* Apply suggestions from code review

Co-authored-by: BayoNet <[email protected]>

* Doc for the max_server_memory_usage setting. Updates.

Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-609: Minor fixes.

* CLICKHOUSEDOCS-609: Actualized position of the setting.

Co-authored-by: olgarev <[email protected]>
Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>

* Add force_primary_key to a pk in tuple test

* DOCS-510: runningAccumulate (#12061)

* asiana21-DOCSUP-797 (#117)

* docs(runningAccumulate): the function description is added

* docs(runningAccumulate): the function description is modified

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* docs(runningAccumulate): some changes

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: BayoNet <[email protected]>

* docs(runningAccumulate): added ru translation

Co-authored-by: asiana21 <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-510: Minor fix.

* Update docs/en/sql-reference/functions/other-functions.md

Co-authored-by: Ivan Blinkov <[email protected]>

* CLICKHOUSEDOCS-510: Fixed links.

Co-authored-by: AsiaKorushkina <[email protected]>
Co-authored-by: asiana21 <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: Ivan Blinkov <[email protected]>

* Remove const specifier to allow auto-move (clangtidy)

* Rewrite Set lookup to make it more readable

* [docs] Sync zh/development/build-osx.md from EN (#12071)

* ISSUES-4006 support first for ALTER ADD|MODIFY COLUMN

* Update deb image

* Don't download image twice

* Remove garbage from images

* bump ci

* fixup

* DOCS-635: Translated the EN version of the AvroConfluent format description (#11930)

* DOCSUP-1350 (#128)

* edited EN version

* add EN and RU translation

* minor changes

* CLICKHOUSEDOCS-635: Updated the description.

Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: Evgenia Sudarikova <[email protected]>

* Better PVS image

* DOCS-605: Description for the always_fetch_merged_part setting (#11921)

* Revolg DOCSUP-998 Document the always_fetch_merged_part setting (#123)

* Add always_fetch_merged_part setting

* revolg-DOCSUP-998-add_always_fetch_merged_part_setting link fixed

* Apply suggestions from code review

Co-authored-by: BayoNet <[email protected]>

* Add always_fetch_merged_part setting. Updates.

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* Add always_fetch_merged_part setting. Updates.

Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-605: Minor fixes.

* CLICKHOUSEDOCS-605: Added Plausible to Adopters.

* Update docs/ru/operations/settings/settings.md

Co-authored-by: alesapin <[email protected]>

* Update docs/en/operations/settings/settings.md

Co-authored-by: alesapin <[email protected]>

* CLICKHOUSEDOCS-605: Fixed access rights description.

Co-authored-by: olgarev <[email protected]>
Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: alesapin <[email protected]>

* Update Dockerfile

* revert e9d3c48

* [blog] 'Package Repository Behind CDN' post (#12082)

* support iframes from DataLens

* initial blog post text

* Improve REVOKE command: now it requires only grant/admin option for only
access which will be revoked.
REVOKE ALL FROM user1 now revokes all granted roles.

* Fix limiting the number of threads for VIEW.

* Update tips.md

* Added test.

* Remove pvs studio from images list

* Better shutdown and conversion

* Reverse arguments

* improve breadcrumbs markup

* Less race conditions

* [docker] install ca-certificates before the first apt-get update (#12095)

* [docker] install ca-certificates before first apt-get update

* Update Dockerfile

* DOCS-522: max_parser_depth (#12097)

* asiana21-DOCSUP-925-max_parser_depth (#132)

* docs(max_parser_depth): added the setting description

* docs(max_parser_depth): some changes

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* Update docs/en/operations/settings/settings.md

Co-authored-by: BayoNet <[email protected]>

* docs(max_parser_depth): added ru translation

* docs(max_parser_depth): removed quotation marks

Co-authored-by: asiana21 <[email protected]>
Co-authored-by: BayoNet <[email protected]>

* CLICKHOUSEDOCS-522: Fixed the link.

Co-authored-by: AsiaKorushkina <[email protected]>
Co-authored-by: asiana21 <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>

* Added results for AWS Lightsail

* (typo) in doc (#12099)

* Create codeql-analysis.yml

* Tiny fixes

* Create anchore-analysis.yml

* Update README.md

* Fix test (#12088)

* Test for a fixed issue #10668, related to input_format_allow_errors_num in CSV

* Add unbundled mode flag

* Move skip lists to clickhouse-test

* More verbose message about skip

* Make skip-list optional

* Update clickhouse-test

* Update 01355_CSV_input_format_allow_errors.sh

* Added a showcase of minimal Docker image

* Added a comment

* Added a comment

* Added a comment

* Formatting

* Fix version column in replicated version collapsing merge tree (#12121)

* Improve performace of reading in order of sorting key. (#11696)

* simplify reading in order of sorting key

* add perf test for reading many parts

* Revert "simplify reading in order of sorting key"

This reverts commit 7267d7c.

* add threshold for preliminary merge for reading in order

* better threshold

* limit threads in test

* Normalize "pid" file handling #3501

* Update StatusFile.cpp

Co-authored-by: Andrew Onyshchuk <[email protected]>
Co-authored-by: Alexey Milovidov <[email protected]>
Co-authored-by: Nikita Mikhaylov <[email protected]>
Co-authored-by: Alexander Kuzmenkov <[email protected]>
Co-authored-by: BohuTANG <[email protected]>
Co-authored-by: feng lv <[email protected]>
Co-authored-by: alesapin <[email protected]>
Co-authored-by: Wang Fenjin <[email protected]>
Co-authored-by: kssenii <[email protected]>
Co-authored-by: Vitaly Baranov <[email protected]>
Co-authored-by: Mikhail Gaidamaka <[email protected]>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Nikolai Kochetov <[email protected]>
Co-authored-by: Ivan Blinkov <[email protected]>
Co-authored-by: BayoNet <[email protected]>
Co-authored-by: Sergei Shtykov <[email protected]>
Co-authored-by: Roman Bug <[email protected]>
Co-authored-by: romanzhukov <[email protected]>
Co-authored-by: tavplubix <[email protected]>
Co-authored-by: Alexander Kuzmenkov <[email protected]>
Co-authored-by: Ivan <[email protected]>
Co-authored-by: Azat Khuzhin <[email protected]>
Co-authored-by: chengy8934 <[email protected]>
Co-authored-by: MicrochipQ <[email protected]>
Co-authored-by: Artem Zuikov <[email protected]>
Co-authored-by: Alexander Kazakov <[email protected]>
Co-authored-by: Vladimir Chebotarev <[email protected]>
Co-authored-by: Nicolae Vartolomei <[email protected]>
Co-authored-by: filimonov <[email protected]>
Co-authored-by: olgarev <[email protected]>
Co-authored-by: Olga Revyakina <[email protected]>
Co-authored-by: AsiaKorushkina <[email protected]>
Co-authored-by: asiana21 <[email protected]>
Co-authored-by: zhang2014 <[email protected]>
Co-authored-by: Evgenia Sudarikova <[email protected]>
Co-authored-by: Anton Popov <[email protected]>
Co-authored-by: manmitya <[email protected]>
Co-authored-by: Nikolai Kochetov <[email protected]>
Co-authored-by: Mikhail Filimonov <[email protected]>
Co-authored-by: Ilya Yatsishin <[email protected]>
Co-authored-by: tavplubix <[email protected]>

Normalize "pid" file handling #3501

(cherry picked from commit f1151ed)