Skip to content

Fix error in "extractAllGroups" function#14889

Merged
alexey-milovidov merged 1 commit intomasterfrom
extract-all-groups-empty-match
Sep 17, 2020
Merged

Fix error in "extractAllGroups" function#14889
alexey-milovidov merged 1 commit intomasterfrom
extract-all-groups-empty-match

Conversation

@alexey-milovidov
Copy link
Member

@alexey-milovidov alexey-milovidov commented Sep 16, 2020

Changelog category (leave one):

  • Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Sep 16, 2020
@alesapin alesapin self-assigned this Sep 17, 2020
@alesapin
Copy link
Member

alesapin commented Sep 17, 2020

AST Fuzzer:

 56 Aborted                 (core dumped) ./clickhouse-server --config-file db/config.xml -- --path db 2>&1

But no errors in the server log and in the fuzzer log.

@akuzm
Copy link
Contributor

akuzm commented Sep 17, 2020

AST Fuzzer:

 56 Aborted                 (core dumped) ./clickhouse-server --config-file db/config.xml -- --path db 2>&1

But no errors in the server log and in the fuzzer log.

No errors would be bad, better investigate. But I found the error in server log:

2020.09.17 03:44:13.654828 [ 71 ] {} <Trace> SystemLog (system.trace_log): Flushing system log, 24 entries to flush
<jemalloc>: ../contrib/jemalloc/include/jemalloc/internal/arena_inlines_a.h:45: Failed assertion: "newarena != NULL"
2020.09.17 03:44:13.657126 [ 63 ] {} <Trace> BaseDaemon: Received signal 6

It also looks very bad...

@akuzm
Copy link
Contributor

akuzm commented Sep 17, 2020

It also looks very bad...

  1. jemalloc shouldn't have an assertion in this place -- a creation of new arena can return null in case we are out of memory
  2. there are a lot of messages like these:
It looks like that the process is near the limit on number of virtual memory mappings.
Current number of mappings (/proc/self/maps): 1048566.
Limit on number of mappings (/proc/sys/vm/max_map_count): 1048576.
You should increase the limit for vm.max_map_count in /etc/sysctl.conf

But why would we be out of memory mappings on a server that is basically idle? Looks like a memory leak.

@alexey-milovidov alexey-milovidov merged commit 2413caa into master Sep 17, 2020
@alexey-milovidov alexey-milovidov deleted the extract-all-groups-empty-match branch September 17, 2020 13:02
This was referenced Sep 17, 2020
Merged
Merged
Merged
Merged
robot-clickhouse pushed a commit that referenced this pull request Sep 17, 2020
Backport #14889 to 20.7: Fix error in \"extractAllGroups\" function
b1197fd
@robot-clickhouse robot-clickhouse mentioned this pull request Sep 17, 2020
Merged
robot-clickhouse pushed a commit that referenced this pull request Sep 17, 2020
Backport #14889 to 20.8: Fix error in \"extractAllGroups\" function
341b23a
@robot-clickhouse robot-clickhouse mentioned this pull request Sep 17, 2020
Merged
robot-clickhouse pushed a commit that referenced this pull request Sep 17, 2020
Backport #14889 to 20.9: Fix error in \"extractAllGroups\" function
287d0a6
@robot-clickhouse robot-clickhouse mentioned this pull request Sep 17, 2020
Merged
robot-clickhouse pushed a commit that referenced this pull request Sep 17, 2020
Backport #14889 to 20.6: Fix error in \"extractAllGroups\" function
ce969a7
@robot-clickhouse robot-clickhouse mentioned this pull request Sep 17, 2020
Merged
alexey-milovidov added a commit that referenced this pull request Sep 17, 2020
@alexey-milovidov
Merge pull request #14942 from ClickHouse/backport/20.7/14889
1fc6fc7 
Backport #14889 to 20.7: Fix error in "extractAllGroups" function
alexey-milovidov added a commit that referenced this pull request Sep 17, 2020
@alexey-milovidov
Merge pull request #14943 from ClickHouse/backport/20.8/14889
3371a62 
Backport #14889 to 20.8: Fix error in "extractAllGroups" function
alexey-milovidov added a commit that referenced this pull request Sep 17, 2020
@alexey-milovidov
Merge pull request #14944 from ClickHouse/backport/20.9/14889
978f6d2 
Backport #14889 to 20.9: Fix error in "extractAllGroups" function
alexey-milovidov added a commit that referenced this pull request Sep 17, 2020
@alexey-milovidov
Merge pull request #14945 from ClickHouse/backport/20.6/14889
7d99c67 
Backport #14889 to 20.6: Fix error in "extractAllGroups" function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alesapin alesapin alesapin approved these changes

Assignees

@alesapin alesapin

Labels

no-docs-needed pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

extactAllGroups[Horizonta|Vertical] stuck and high memory usage with specific regexp expression

5 participants

@alexey-milovidov @alesapin @akuzm @qoega @robot-clickhouse