Schmil Monderer (7e642d03) at 18 Mar 21:54

... and 2399 more commits

Schmil Monderer (7a83727e) at 18 Mar 13:36

... and 1 more commit

@srushik Makes sense

ok thanks

Schmil Monderer (2ae0f869) at 18 Mar 10:54

Schmil Monderer (8b26685d) at 18 Mar 10:45

@bwill We should also check that customers aren't using the REST endpoint directly. It is possible that some use the endpoint in their pipeline for a conditional workflow

Schmil Monderer (e198728e) at 18 Mar 08:28

@minac What do you by "missing". The transaction was redundant because we are updating a single table only once

done

Schmil Monderer (710b4f00) at 17 Mar 21:31

... and 1 more commit

Schmil Monderer (c69338f7) at 17 Mar 21:31

Schmil Monderer (a1cc19b3) at 17 Mar 15:47

Schmil Monderer (b76e72e9) at 17 Mar 15:03

What does this MR do and why?

Implements dual-write support for vulnerability data to both Elasticsearch indices: the existing vulnerabilities index and the new vulnerability_reads index.

This is part of the Vulnerabilities Across Contexts (VAC) initiative. The new vulnerability_reads index uses vulnerability_reads.id as primary key (instead of vulnerability_id), which is needed to support multiple context-specific read entries per vulnerability.

Changes

• Vulnerabilities::Read#elastic_reference — When the vulnerability_read_es_dual_write feature flag is enabled and the create_vulnerability_reads_index ES migration has completed, returns an array of two serialized references (one for each index) instead of a single reference.
• Search::Elastic::Reference.serialize — Handles Array return values from elastic_reference (pass-through).
• Elastic::ProcessBookkeepingService#track! — Flattens the serialized items array after mapping, so multi-ref arrays are enqueued as individual items.
• Feature flag vulnerability_read_es_dual_write — Ops flag (default off) to gate the dual-write behavior.

How it works

Vulnerabilities::Read record
  → elastic_reference
    → [Vulnerability ref string, Vulnerabilities::Read ref string]
  → ProcessBookkeepingService.track!
    → serialize + flatten
    → both refs enqueued independently in Redis
  → ProcessBookkeepingService#execute
    → each ref deserialized and indexed to its respective ES index

Dependencies

This MR is chained on:

• !225631 (preloaders)
• !225822 (reference class) ← current target branch

Once those merge to master, this MR should be retargeted to master.

Feature flag

Name: vulnerability_read_es_dual_write Type: ops

The flag is additionally gated on the create_vulnerability_reads_index ES migration having completed, to prevent writes to a non-existent index.

Closes #592362

Schmil Monderer (6397ba23) at 17 Mar 14:46

... and 5 more commits

Schmil Monderer (afd153ca) at 17 Mar 14:23

Schmil Monderer (30339e17) at 17 Mar 13:21

Schmil Monderer (fef3ed84) at 17 Mar 12:21

Schmil Monderer (8215c44c) at 17 Mar 11:54