Skip to content

Excessive use of GC #32

@greenwoodma

Description

@greenwoodma

Currently the code does an explicit GC both before and after loading the gazetteer so that it can report on memory consumption. It does this even when duplicating a PR in which no extra memory is allocated. We've seen each GC take as much as 5s, so when using this on cloud or GCP it can add quite an overhead: 40s for 4 threads is just GC. On cloud this is possibly what's causing the prometheus warnings as the full load time is high.

The suggestion is to add a new debug or profiling option to the PR, so that in normal usage we don't report the memory consumption and don't explicitly run the GC.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions