Rework total memory tracker#10362
Conversation
alexey-milovidov
added
pr-improvement
alexey-milovidov
force-pushed
the
rework-total-memory-tracker
branch
from
566c79b to
1eb3080
Compare
src/Core/Settings.h
Outdated
There was a problem hiding this comment.
Maybe Only has meaning at server startup. will be better (like other settings has)?
|
(And just a note it includes #10361) |
alexey-milovidov
force-pushed
the
rework-total-memory-tracker
branch
from
a080e15 to
3755a5b
Compare
| \ | ||
| M(SettingUInt64, max_memory_usage, 0, "Maximum memory usage for processing of single query. Zero means unlimited.", 0) \ | ||
| M(SettingUInt64, max_memory_usage_for_user, 0, "Maximum memory usage for processing all concurrently running queries for the user. Zero means unlimited.", 0) \ | ||
| M(SettingUInt64, max_memory_usage_for_all_queries, 0, "Maximum memory usage for processing all concurrently running queries on the server. Zero means unlimited.", 0) \ |
There was a problem hiding this comment.
Actually max_memory_usage_for_all_queries can be useful to (to limit the memory usage for all queries except specific, by overwriting max_memory_usage_for_all_queries to 0), but I guess max_memory_usage_for_user can be used instead (but will require more attention, since there is default user).
|
My collegue @velom from Yandex.Metrica requested (he is too shy to post a comment on GitHub) that |
Maybe reintroducing |
|
This PR makes ClickHouseMetrics_MemoryTracking grows on and on (after some time it >10TB, and it does not stops growing), while ClickHouseAsyncMetrics_MemoryResident is ok (after some time it is in between 300-600GB), any thoughts? |
|
We cannot longer calculate this metric in incremental fashion - should reset it in the same way as total_memory_tracker. |
|
Why it grows? Because memory is deallocated in a context where there is no CurrentThread and total_memory_tracker. I planned try to make it used in every context. It should be a simple change but care should be taken... Maybe you can also try to make this change? Also worth to note that we cannot calculate 100% consistently. |
Sure, I will take a look if you will not fix it by that time |
|
By reading this topic, I understood follwing; please correct me if I am wrong:
I am interested in this topic because we are using CH as a library in third-party application, and it would definitely be great to be able to explicitly measure CH runtime memory separately from the rest of RSS for monitoring purposes. Expressing it as (RSS) minus (the rest of 'known' memory) may hide memory-related issues in ClickHouse code. |
3 participants
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added a setting
max_server_memory_usageto limit total memory usage of the server. The metricMemoryTrackingis now calculated without a drift. The settingmax_memory_usage_for_all_queriesis now obsolete and does nothing. This closes #10293. Note: on servers with very low memory amount and swap you may have to addmax_server_memory_usage_to_ram_ratioin config.xml and set it to a value larger than one.