Ok.

BTW, do you consider the option to use existing storage_configuration and just to refer to some policy to store temporary data?

BTW, do you consider the option to use existing storage_configuration and just to refer to some policy to store temporary data?

At first I went this way (and just has tmp_policy), but now I don't have any ideas why, using existing storage_configuration looks indeed more convenient. Should be rebased...

but now I don't have any ideas why

Got it, I want to have separate default disk path (since default selector uses <path> and I want to use <tmp_path>)

@alexey-milovidov So, the question is, what is preferable?

I did the same thing for distributed engine (did not submitted yet), by adding another separate distributed_storage_configuration, to leave storage_configuration for MergeTree engine only, but I don't have any objective reasons,

Got it, I want to have separate default disk path (since default selector uses and I want to use <tmp_path>)

Not sure that I understand this completely, but looks like it's not an issue:

• if tmp_policy is not specified, tmp_path is used;
• if tmp_policy is default, then default policy is used that equals to path.

Probably one issue is that tmp subdirectory will not be created under path.
Maybe change the meaning of tmp_path so that it will be equal to path by default but tmp subdirectory is always used inside tmp_path?

Probably one issue is that tmp subdirectory will not be created under path.
Maybe change the meaning of tmp_path so that it will be equal to path by default but tmp subdirectory is always used inside tmp_path?

The issue is that there is tmp_path and path, so if storage_policy will be used instead and no tmp_policy will be specified then it will use path (but at least it should be changed to path+/tmp)

Always using tmp subdirectory is Ok.

Ok

Looks like the following is failing in upstream too:

• test_ttl_move/test.py::test_double_move_while_select[test_double_move_while_select_positive-1]
• test_multiple_disks/test.py::test_concurrent_alter_move[concurrently_altering_replicated_mt-ReplicatedMergeTree('/clickhouse/concurrently_altering_replicated_mt', '1')]
• test_multiple_disks/test.py::test_concurrent_alter_modify[alter_modifying_mt-MergeTree()]
• test_ttl_move/test.py::test_concurrent_alter_with_ttl_move[concurrently_altering_ttl_replicated_mt-ReplicatedMergeTree('/clickhouse/concurrently_altering_ttl_replicated_mt', '1')]

And test_storage_kafka cannot start docker.

Always using tmp subdirectory is Ok.

Rebased (<tmp_path> behavior left unchanged)

Looks like test failures are not relevant to this changes, or am I missing something?

Hm, tmp subdirectory is missing, will fix it

Always using tmp subdirectory is Ok.

@alexey-milovidov so tmp_path directive can go away?

Hm, tmp subdirectory is missing, will fix it

So after looking into this again, I'm not sure which way this should goes:

• adding tmp/ in all callers looks error-prone
• adding some interface for <disk> is not that useful, create the disk with that <path> is simpler (the directory will not be created but it is okay, if this will useful then <create>true</create> can be parsed and call mkdir, and also note that currently tmp_path also will not be created and fail if the directory will not exist)

IOW maybe this should left as-is? @alexey-milovidov

adding tmp/ in all callers looks error-prone

So, the issue is that after we selected disk, we have to append tmp to its path but it's too error prone because we have to add it in multiple places: external aggregation, external sorting, joins, distributed tables... Do I understand correctly?

IOW maybe this should left as-is?

Ok, now it looks reasonable.

Do I understand correctly?

Yes

Ok. Let's proceed with the first variant of implementation.

Rebased to include #8790 (to make the tests pass)