Skip to content

Trim size of CachedOnDiskReadBufferFromFile structure 50x#96098

Merged
azat merged 3 commits intoClickHouse:masterfrom
azat:cached-reads-pull
Feb 5, 2026
Merged

Trim size of CachedOnDiskReadBufferFromFile structure 50x#96098
azat merged 3 commits intoClickHouse:masterfrom
azat:cached-reads-pull

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Feb 5, 2026

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Trim size of CachedOnDiskReadBufferFromFile structure 50x

With 64 byte alignment and 1144 counters (as of now) the Counters object took ~73KiB, and I saw that CachedOnDiskReadBufferFromFile eats ~90GiB of RAM because of this.

And those counters are used only for debugging (filesystem_cache_log), but as @kssenii told she hasn't been used them for a long time

azat added 2 commits February 5, 2026 09:14
@azat
Minor cleanup in CachedOnDiskReadBufferFromFile
37d8cfc
@azat
Use explicit capture for buffer creator callback for CachedOnDiskRead…
f38b195 
…BufferFromFile

Though compiler should be smart enought, better to list them explicitly
to avoid doubts
@azat azat requested a review from kssenii February 5, 2026 17:39
@azat azat mentioned this pull request Feb 5, 2026
Merged
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Feb 5, 2026 •
edited
Loading

Workflow [PR], commit [a729564]

Summary: ❌

job_name test_name status info comment
Integration tests (amd_binary, 5/5) failure
test_restore_db_replica/test.py::test_query_after_restore_db_replica[alter table-with exists table-no restart] FAIL cidb, issue
Stress test (amd_tsan) failure
Logical error: Block structure mismatch in A stream: different number of columns: (STID: 0993-2da1) FAIL cidb, issue

@clickhouse-gh clickhouse-gh bot added the pr-performance Pull request with some performance improvements label Feb 5, 2026
@alexey-milovidov alexey-milovidov self-assigned this Feb 5, 2026
@azat
Trim size of CachedOnDiskReadBufferFromFile structure 50x
a729564 
With 64 byte alignment and 1144 counters (as of now) the Counters object
took ~73KiB, and I saw that CachedOnDiskReadBufferFromFile eats ~90GiB
of RAM because of this.
@azat azat force-pushed the cached-reads-pull branch from caa4e77 to a729564 Compare February 5, 2026 18:04
@kssenii kssenii self-assigned this Feb 5, 2026
@azat azat enabled auto-merge February 5, 2026 19:51
@azat azat added this pull request to the merge queue Feb 5, 2026
Merged via the queue into ClickHouse:master with commit 3561bc3 Feb 5, 2026
130 of 133 checks passed
@azat azat deleted the cached-reads-pull branch February 5, 2026 21:24
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Feb 5, 2026
@azat azat added the pr-must-backport Pull request should be backported intentionally. Use this label with great care! label Feb 5, 2026
@robot-clickhouse robot-clickhouse added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label Feb 5, 2026
This was referenced Feb 5, 2026
Closed
Merged
Merged
Merged
Merged
@Algunenano Algunenano added the memory When memory usage is higher than expected label Feb 5, 2026
robot-clickhouse added a commit that referenced this pull request Feb 6, 2026
@robot-clickhouse
Backport #96098 to 25.8: Trim size of CachedOnDiskReadBufferFromFile …
349bdf3 
…structure 50x
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Feb 6, 2026
Merged
robot-clickhouse added a commit that referenced this pull request Feb 6, 2026
@robot-clickhouse
Backport #96098 to 25.11: Trim size of CachedOnDiskReadBufferFromFile…
f7c5cbb 
… structure 50x
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Feb 6, 2026
Merged
robot-clickhouse added a commit that referenced this pull request Feb 6, 2026
@robot-clickhouse
Backport #96098 to 25.12: Trim size of CachedOnDiskReadBufferFromFile…
81969b7 
… structure 50x
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Feb 6, 2026
Merged
robot-clickhouse added a commit that referenced this pull request Feb 6, 2026
@robot-clickhouse
Backport #96098 to 26.1: Trim size of CachedOnDiskReadBufferFromFile …
a3a1946 
…structure 50x
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Feb 6, 2026
Merged
@robot-clickhouse robot-clickhouse added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Feb 6, 2026
clickhouse-gh bot added a commit that referenced this pull request Feb 6, 2026
@clickhouse-gh
Merge pull request #96177 from ClickHouse/backport/26.1/96098
ee44de4 
Backport #96098 to 26.1: Trim size of CachedOnDiskReadBufferFromFile structure 50x
clickhouse-gh bot added a commit that referenced this pull request Feb 6, 2026
@clickhouse-gh
Merge pull request #96175 from ClickHouse/backport/25.11/96098
48278ad 
Backport #96098 to 25.11: Trim size of CachedOnDiskReadBufferFromFile structure 50x
azat added a commit that referenced this pull request Feb 6, 2026
@azat
Merge pull request #96176 from ClickHouse/backport/25.12/96098
fe69819 
Backport #96098 to 25.12: Trim size of CachedOnDiskReadBufferFromFile structure 50x
clickhouse-gh bot added a commit that referenced this pull request Feb 6, 2026
@clickhouse-gh
Merge pull request #96174 from ClickHouse/backport/25.8/96098
703ac6d 
Backport #96098 to 25.8: Trim size of CachedOnDiskReadBufferFromFile structure 50x
@alexey-milovidov alexey-milovidov mentioned this pull request Apr 8, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexey-milovidov alexey-milovidov alexey-milovidov approved these changes

@kssenii kssenii kssenii approved these changes

Assignees

@alexey-milovidov alexey-milovidov

@kssenii kssenii

Labels

memory When memory usage is higher than expected pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-performance Pull request with some performance improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@azat @alexey-milovidov @kssenii @Algunenano @robot-clickhouse @pnovitskiy @robot-clickhouse-ci-2