Marius Bobin activity https://gitlab.com/mbobin 2026-03-18T15:18:07Z tag:gitlab.com,2026-03-18:5218181289 Marius Bobin commented on merge request !227141 at GitLab.org / GitLab 2026-03-18T15:18:07Z mbobin Marius Bobin

Looks good! 👍

 tag:gitlab.com,2026-03-18:5218170832 Marius Bobin approved merge request !227141: Use plain retry_lock in AtomicProcessingService (behind FF) at GitLab.org / GitLab 2026-03-18T15:15:55Z mbobin Marius Bobin

What does this MR do and why?

Ci::ProcessBuildService#execute is just an elaborate routing helper to different state machine events. All state machine event callbacks are already in transactions, so the outer transaction is redundant. By switching to a plain retry_lock, we can benefit from the optimizations introduced in !225133

This change is disabled by default, behind the feature flag ci_pipeline_processing_atomic_processing_service_plain_retry_lock.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. In rails console enable the feature flag 
    Feature.enable(:ci_pipeline_processing_atomic_processing_service_plain_retry_lock)
  2. Run a pipeline
  3. Now, enable precompute_pending_build_args flag, as this is a related flag that we are also rolling out 
    Feature.enable(:precompute_pending_build_args)
  4. Run a pipeline

Expected results: Both pipelines should run to completion

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

 tag:gitlab.com,2026-03-18:5217963268 Marius Bobin closed merge request !226893: Update Pipeline-CommitStatus/Build relations with Rails composite keys at GitLab.org / GitLab 2026-03-18T14:36:41Z mbobin Marius Bobin

What does this MR do?

Replaces the custom in_partition lambda + partition_foreign_key pattern with Rails 7.1 native composite foreign keys for all relations between Ci::Pipeline and CommitStatus/Ci::Build/Ci::Processable/Ci::Bridge.

This is MR 1 of a series to address #431734.

Conversion pattern

Before:

has_many :statuses, ->(pipeline) { in_partition(pipeline) },
  class_name: 'CommitStatus', foreign_key: :commit_id,
  inverse_of: :pipeline, partition_foreign_key: :partition_id

After:

has_many :statuses,
  class_name: 'CommitStatus',
  foreign_key: [:commit_id, :partition_id],
  primary_key: [:id, :partition_id],
  inverse_of: :pipeline

For relations with additional scopes (e.g. .latest, .order(...)), the lambda is retained for the extra scoping only, with the in_partition call removed.

Files changed

File Changes
app/models/commit_status.rb belongs_to :pipeline updated
app/models/ci/build.rb belongs_to :pipeline updated
app/models/ci/pipeline.rb ~25 has_many relations updated

Related issues

Closes #431734 (partially)

Part of &11812 maintenance::refactor,CI data partitioning,Category:Continuous Integration

 tag:gitlab.com,2026-03-18:5217960221 Marius Bobin closed merge request !226895: Update Pipeline-Stage and Stage-CommitStatus partitioned relations to use Rails 7.1 composite keys at GitLab.org /... 2026-03-18T14:36:09Z mbobin Marius Bobin

What does this MR do?

Replaces the custom in_partition lambda + partition_foreign_key pattern with Rails 7.1 native composite foreign keys for all relations between Ci::Pipeline ↔️ Ci::Stage and Ci::Stage ↔️ CommitStatus/Ci::Build/Ci::Bridge.

This is MR 2 of a series to address #431734. Depends on !226893.

Relations updated

File Relation Type
app/models/ci/pipeline.rb has_many :stages Pipeline → Stage
app/models/ci/stage.rb belongs_to :pipeline Stage → Pipeline
app/models/ci/stage.rb has_many :statuses Stage → CommitStatus
app/models/ci/stage.rb has_many :latest_statuses Stage → CommitStatus
app/models/ci/stage.rb has_many :retried_statuses Stage → CommitStatus
app/models/ci/stage.rb has_many :processables Stage → Processable
app/models/ci/stage.rb has_many :builds Stage → Build
app/models/ci/stage.rb has_many :bridges Stage → Bridge
app/models/ci/stage.rb has_many :generic_commit_statuses Stage → GenericCommitStatus
app/models/commit_status.rb belongs_to :ci_stage CommitStatus → Stage

Related issues

Closes #431734 (partially)

Part of &11812

 tag:gitlab.com,2026-03-18:5217959672 Marius Bobin closed merge request !226905: Draft: Remove ActiveRecord partition_foreign_key and query_constraints patches at GitLab.org / GitLab 2026-03-18T14:36:04Z mbobin Marius Bobin

What does this MR do?

Removes the custom ActiveRecord patches in gems/activerecord-gitlab that were used to support the partition_foreign_key option on associations and the query_constraints backport.

These patches are no longer needed because:

  1. All CI partitioned relations now use Rails 7.1 native composite foreign keys (foreign_key: [:fk, :partition_id], primary_key: [:id, :partition_id]) instead of the custom in_partition lambda + partition_foreign_key pattern.
  2. Rails 7.2 natively supports query_constraints, making the Base patch redundant.

This is MR 7 (final cleanup) of a series to address #431734.

Dependencies

This MR must be merged AFTER all 6 model MRs:

  1. !226893 - Pipeline to CommitStatus/Build
  2. !226895 - Pipeline to Stage, Stage to CommitStatus
  3. !226896 - Pipeline to Variable/ExecutionConfig/Workload
  4. !226898 - Build to JobArtifact chain
  5. !226899 - Build to trace/metadata/state models
  6. !226901 - Build/Processable to job-level models

Files removed

  • gems/activerecord-gitlab/lib/active_record/gitlab_patches/partitioning.rb (entry point)
  • gems/activerecord-gitlab/lib/active_record/gitlab_patches/partitioning/base.rb (query_constraints backport)
  • gems/activerecord-gitlab/lib/active_record/gitlab_patches/partitioning/associations/builder/association.rb
  • gems/activerecord-gitlab/lib/active_record/gitlab_patches/partitioning/reflection/abstract_reflection.rb
  • gems/activerecord-gitlab/lib/active_record/gitlab_patches/partitioning/reflection/association_reflection.rb
  • gems/activerecord-gitlab/lib/active_record/gitlab_patches/partitioning/reflection/macro_reflection.rb
  • All associated spec files under gems/activerecord-gitlab/spec/active_record/gitlab_patches/partitioning/

Files modified

  • gems/activerecord-gitlab/lib/active_record/gitlab_patches.rb (removed require)

What is NOT removed

The in_partition scope in app/models/concerns/ci/partitionable.rb is kept because it is still used in non-association contexts.

Related issues

Closes #431734

Part of &11812

 tag:gitlab.com,2026-03-18:5217959282 Marius Bobin closed merge request !226901: Update Build/Processable job-level partitioned relations to Rails 7.1 composite keys at GitLab.org / GitLab 2026-03-18T14:36:00Z mbobin Marius Bobin

What does this MR do?

Replaces the custom in_partition lambda + partition_foreign_key pattern with Rails 7.1 native composite foreign keys for all remaining job-level relations on Ci::Build and Ci::Processable, and their corresponding child models.

This is MR 6 (final) of a series to address #431734.

Depends on !226893.

Relations updated

File Relation Type
app/models/ci/build.rb has_many :inputs Build → JobInput
app/models/ci/build.rb has_many :job_annotations Build → JobAnnotation
app/models/ci/build.rb has_many :taggings Build → BuildTag
app/models/ci/processable.rb has_one :job_definition_instance Processable → JobDefinitionInstance
app/models/ci/processable.rb has_one :job_definition (through) Processable → JobDefinition
app/models/ci/processable.rb has_many :job_messages Processable → JobMessage
app/models/ci/processable.rb has_many :error_job_messages Processable → JobMessage
app/models/ci/processable.rb has_one :job_source Processable → BuildSource
app/models/ci/build_need.rb belongs_to :build BuildNeed → Processable
app/models/ci/build_source.rb belongs_to :job BuildSource → Processable
app/models/ci/build_tag.rb belongs_to :build BuildTag → Build
app/models/ci/build_name.rb belongs_to :build BuildName → Build
app/models/ci/job_annotation.rb belongs_to :job JobAnnotation → Build
app/models/ci/job_message.rb belongs_to :job JobMessage → Processable
app/models/ci/job_input.rb belongs_to :job JobInput → Build
app/models/ci/job_definition_instance.rb belongs_to :job JobDefinitionInstance → Processable
app/models/ci/job_definition_instance.rb belongs_to :job_definition JobDefinitionInstance → JobDefinition
app/models/ci/unit_test_failure.rb belongs_to :build UnitTestFailure → Build

Full MR series

  1. !226893 - Pipeline ↔️ CommitStatus/Build
  2. !226895 - Pipeline ↔️ Stage, Stage ↔️ CommitStatus
  3. !226896 - Pipeline ↔️ Variable/ExecutionConfig/Workload
  4. !226898 - Build ↔️ JobArtifact chain
  5. !226899 - Build ↔️ trace/metadata/state models
  6. This MR - Build/Processable ↔️ job-level models

Related issues

Closes #431734

Part of &11812

 tag:gitlab.com,2026-03-18:5217958767 Marius Bobin closed merge request !226899: Update Build trace, metadata, and state partitioned relations to Rails 7.1 composite keys at GitLab.org / GitLab 2026-03-18T14:35:54Z mbobin Marius Bobin

What does this MR do?

Replaces the custom in_partition lambda + partition_foreign_key pattern with Rails 7.1 native composite foreign keys for all relations between Ci::Build and its trace/metadata/state child models.

This is MR 5 of a series to address #431734. Depends on !226893.

Relations updated

File Relation Type
app/models/ci/build.rb has_many :trace_chunks Build → BuildTraceChunk
app/models/ci/build.rb has_one :runner_manager_build Build → RunnerManagerBuild
app/models/ci/build_trace_chunk.rb belongs_to :build BuildTraceChunk → Build
app/models/ci/build_trace_metadata.rb belongs_to :build BuildTraceMetadata → Build
app/models/ci/build_trace_metadata.rb belongs_to :trace_artifact BuildTraceMetadata → JobArtifact
app/models/ci/build_metadata.rb belongs_to :build BuildMetadata → CommitStatus
app/models/ci/build_pending_state.rb belongs_to :build BuildPendingState → Build
app/models/ci/build_report_result.rb belongs_to :build BuildReportResult → Build
app/models/ci/running_build.rb belongs_to :build RunningBuild → Build
app/models/ci/pending_build.rb belongs_to :build PendingBuild → Build

Related issues

Closes #431734 (partially)

Part of &11812

 tag:gitlab.com,2026-03-18:5217894355 Marius Bobin closed merge request !226898: Update Build-JobArtifact partitioned relations to Rails 7.1 composite keys at GitLab.org / GitLab 2026-03-18T14:23:17Z mbobin Marius Bobin

What does this MR do?

Replaces the custom in_partition lambda + partition_foreign_key pattern with Rails 7.1 native composite foreign keys for all relations in the Build ↔️ JobArtifact axis, including Ci::JobArtifactReport and the EE Geo::JobArtifactState.

This is MR 4 of a series to address #431734. Depends on !226893.

Relations updated

File Relation Type
app/models/ci/build.rb has_many :job_artifacts Build → JobArtifact
app/models/ci/build.rb has_one :job_artifacts_* (dynamic, per file type) Build → JobArtifact
app/models/ci/job_artifact.rb belongs_to :job JobArtifact → Build
app/models/ci/job_artifact.rb has_one :artifact_report JobArtifact → JobArtifactReport
app/models/ci/job_artifact_report.rb belongs_to :job_artifact JobArtifactReport → JobArtifact
ee/app/models/ee/ci/job_artifact.rb has_one :job_artifact_state JobArtifact → Geo::JobArtifactState

Related issues

Closes #431734 (partially)

Part of &11812

 tag:gitlab.com,2026-03-18:5217892775 Marius Bobin closed merge request !226896: Update Pipeline-Variable, BuildExecutionConfig, and Workload partitioned relations to Rails 7.1 composite keys at ... 2026-03-18T14:22:57Z mbobin Marius Bobin

What does this MR do?

Replaces the custom in_partition lambda + partition_foreign_key pattern with Rails 7.1 native composite foreign keys for relations between Ci::Pipeline ↔️ Ci::PipelineVariable, Ci::Pipeline/Ci::Build ↔️ Ci::BuildExecutionConfig, Ci::Pipeline ↔️ Ci::Workloads::Workload, and Ci::Workloads::Workload ↔️ Ci::Workloads::VariableInclusions.

This is MR 3 of a series to address #431734. Depends on !226893.

Relations updated

File Relation Type
app/models/ci/pipeline.rb has_many :variables Pipeline → PipelineVariable
app/models/ci/pipeline.rb has_one :workload Pipeline → Workload
app/models/ci/pipeline_variable.rb belongs_to :pipeline PipelineVariable → Pipeline
app/models/ci/build_execution_config.rb belongs_to :pipeline BuildExecutionConfig → Pipeline
app/models/ci/build_execution_config.rb has_many :builds BuildExecutionConfig → Build
app/models/ci/build.rb belongs_to :execution_config Build → BuildExecutionConfig
app/models/ci/workloads/workload.rb has_many :variable_inclusions Workload → VariableInclusions

Related issues

Closes #431734 (partially)

Part of &11812

 tag:gitlab.com,2026-03-18:5217027891 Marius Bobin commented on issue #592819 at GitLab.org / GitLab 2026-03-18T11:22:55Z mbobin Marius Bobin

@stejacks-gitlab I don't think we can't answer any of those questions(reliably) with our current capabilities, but it will be easier with https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/ci_job_telemetry/

/cc @golnazs @pedropombeiro

 tag:gitlab.com,2026-03-18:5216999967 Marius Bobin opened merge request !227881: Revert "Merge branch 'kerrizor/add-find_by_id_through_partition' into 'master'" at GitLab.org / GitLab 2026-03-18T11:15:58Z mbobin Marius Bobin

What does this MR do and why?

Revert "Merge branch 'kerrizor/add-find_by_id_through_partition' into 'master'"

This reverts merge request !227539 because is no longer needed thanks to !227689

References

#593701

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

 tag:gitlab.com,2026-03-18:5216996381 Marius Bobin pushed new project branch revert-e02244fa at GitLab.org / GitLab 2026-03-18T11:15:08Z mbobin Marius Bobin

Marius Bobin (52535665) at 18 Mar 11:15

Revert "Merge branch 'kerrizor/add-find_by_id_through_partition' in...

 tag:gitlab.com,2026-03-18:5216985784 Marius Bobin opened merge request !227879: Revert "Merge branch 'mb-partition-query' into 'master'" at GitLab.org / GitLab 2026-03-18T11:12:41Z mbobin Marius Bobin

What does this MR do and why?

Revert "Merge branch 'mb-partition-query' into 'master'"

This reverts merge request !227525 because is no longer needed thanks to !227689

References

#593701

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

 tag:gitlab.com,2026-03-18:5216981034 Marius Bobin pushed new project branch revert-4dc20a1a at GitLab.org / GitLab 2026-03-18T11:11:35Z mbobin Marius Bobin

Marius Bobin (cefac2ec) at 18 Mar 11:11

Revert "Merge branch 'mb-partition-query' into 'master'"

 tag:gitlab.com,2026-03-18:5216819958 Marius Bobin pushed to project branch 593701-partition-pruning at GitLab.org / GitLab 2026-03-18T10:36:50Z mbobin Marius Bobin

Marius Bobin (7f77c523) at 18 Mar 10:36

Add feature flags and limit scope

 tag:gitlab.com,2026-03-17:5214085749 Marius Bobin commented on merge request !227689 at GitLab.org / GitLab 2026-03-17T17:36:28Z mbobin Marius Bobin

@stomlinson @mfanGitLab I had to push !227689 (62a3e279) to fix the tests.

 tag:gitlab.com,2026-03-17:5214080445 Marius Bobin pushed to project branch 593701-partition-pruning-ci-method at GitLab.org / GitLab 2026-03-17T17:35:04Z mbobin Marius Bobin

Marius Bobin (62a3e279) at 17 Mar 17:35

Fix tests and apply reviewer feedback

 tag:gitlab.com,2026-03-17:5213852326 Marius Bobin commented on merge request !227689 at GitLab.org / GitLab 2026-03-17T16:38:53Z mbobin Marius Bobin

@pedropombeiro I'm going to merge this as is because we need it to mitigate the incident.

 tag:gitlab.com,2026-03-17:5213845537 Marius Bobin commented on merge request !227689 at GitLab.org / GitLab 2026-03-17T16:37:07Z mbobin Marius Bobin

We're not updating the records, so what's the point? 🤔

 tag:gitlab.com,2026-03-17:5213845497 Marius Bobin commented on merge request !227689 at GitLab.org / GitLab 2026-03-17T16:37:07Z mbobin Marius Bobin

I don't think it's needed because we do use it in the app and should be hit by those tests. Pipeline here is just an example of a partitionable resource, it doesn't mean that we want to test it on all of them.