Integration tests

Broken in #11903

Functional stateful tests (address) — No output log

An error with Docker registry.

Unable to find image 'yandex/clickhouse-stateful-test:latest' locally
docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
See 'docker run --help'.

40% expected variability for query no. 0 from questdb_sum_32 -- rather bad, but why? Let's open 'output.7z' and look at the variability of metrics for this query (questdb_sum_32_0_SELECT...right.metrics.rep). The second column is array [min, med, max] for metric value among query runs:

8	[4000,4000,36000]	SystemTimeMicroseconds
3.029	[144499493,151033285,602019377]	PerfMinEnabledRunningTime
2.27	[3,170,389]	OSCPUWaitMicroseconds
1.25	[2,4,7]	SoftPageFaults
1.178	[23,28,56]	PerfContextSwitches
1.151	[72,86,171]	NetworkSendElapsedMicroseconds
0.943	[67674059,68195397,132009544]	PerfBranchInstructions
0.597	[11877,15112,20905]	PerfInstructionTLBReferences

We see SystemTime go from 4 ms to the maximum of 36 ms, which is about 25% of the full run time of 150ms. This must be the profiler again... I think I'll have to disable it for measured runs, and only run it for prewarm runs.

omg, the memory profiling now takes 25% of query no.2 in questdb_sum_int32. These results won't make sense, I've reverted the profiling patch, will rerun the tests.

Do I understand correctly, that performance test creates and fills all tables first and then runs all the tests with all substitutions?
Looks like there is a room for improvement...