Thanks @pshutsin, I had some questions and suggestions.
I noticed the rspec:undercoverage test is showing a warning https://gitlab.com/gitlab-org/gitlab/-/jobs/13532575386, this will hard fail when the tier 3 pipeline runs.
I went through the validation steps, and the query works as described.
suggestion: Same as below, it doesn't look like there's any test coverage for an unauthorized user.
suggestion: Add test coverage for an unauthorized user.
question: Do you know if there are any current consumers of this? I can see this is experimental so we don't need to go through the formal deprecation process, but wondering if we should play it safe and add the new field, add deprecated to the old one and then remove in the next milestone?
suggestion: There isn't any check that enforces this, could we add a guard to the mount_aggregation_engine method to enforce it?
question: Why is there a difference in how authorize is passed, AgentPlatformSessions passes as a keyword arg, CodeSuggestions is within the block?
Thanks @10io, could you set MWPS, pipeline is green now.
Also noticed that the undercoverage job is showing some coverage gaps that need to be addressed. https://gitlab.com/gitlab-org/gitlab/-/jobs/13507341867
@radbatnag I'm having issues validating. I get a 404 Project not found when I try to run the pipeline as the developer.
The user is a member, and the project exists. Not sure what i'm missing or doing wrong?
Here are all the details of what i'm doing.
Troubleshooting, by checking project and membership
gitlab git:(rfh-4244-return-403-instead-of-500) echo $PROJECT_ID
36
➜ gitlab git:(rfh-4244-return-403-instead-of-500) curl -s "http://gdk.test:3000/api/v4/projects/$PROJECT_ID/members" \
-H "PRIVATE-TOKEN: $ROOT_PAT" | python3 -m json.tool
[
{
"id": 22,
"username": "i-user-0-1769046775",
"public_email": null,
"name": "I User0",
"state": "active",
"locked": false,
"avatar_url": "https://www.gravatar.com/avatar/7b21d4d3108a1f9136940a12f1b77a92087f31a3a103b8fd3cf07fbba30cde58?s=80&d=identicon",
"web_url": "http://gdk.test:3000/i-user-0-1769046775",
"access_level": 40,
"created_at": "2026-03-17T04:02:16.470Z",
"created_by": {
"id": 1,
"username": "root",
"public_email": null,
"name": "Administrator",
"state": "active",
"locked": false,
"avatar_url": "https://www.gravatar.com/avatar/df6a7653faf47b4a08264217861507dc6abf07369a07803ebeb34d9a212cf6f9?s=80&d=identicon",
"web_url": "http://gdk.test:3000/root"
},
"expires_at": null,
"email": "[email protected]",
"membership_state": "active"
}
]
➜ gitlab git:(rfh-4244-return-403-instead-of-500) curl -s "http://gdk.test:3000/api/v4/projects/$PROJECT_ID" \
-H "PRIVATE-TOKEN: $ROOT_PAT" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['id'],
d['path_with_namespace'])"
36 ip-restricted-group/registry-ip-test
I confirmed the dev pat is working
gitlab git:(rfh-4244-return-403-instead-of-500) curl -s "http://gdk.test:3000/api/v4/user" \
-H "PRIVATE-TOKEN: $DEV_PAT" | python3 -m json.tool
{
"id": 22,
"username": "i-user-0-1769046775",
"public_email": null,
"name": "I User0",
"state": "active",
"locked": false,
"avatar_url": "https://www.gravatar.com/avatar/7b21d4d3108a1f9136940a12f1b77a92087f31a3a103b8fd3cf07fbba30cde58?s=80&d=identicon",
"web_url": "http://gdk.test:3000/i-user-0-1769046775",
"created_at": "2026-01-22T01:52:55.860Z",
"bio": "",
"location": "",
"linkedin": "",
"twitter": "",
"discord": "",
"website_url": "",
"github": "",
"job_title": "",
"pronouns": null,
"organization": "",
"bot": false,
"work_information": null,
"local_time": null,
"last_sign_in_at": "2026-03-17T03:29:46.929Z",
"confirmed_at": "2026-01-22T01:52:55.845Z",
"last_activity_on": "2026-03-17",
"email": "[email protected]",
"theme_id": 3,
"color_scheme_id": 1,
"projects_limit": 100000,
"current_sign_in_at": "2026-03-17T03:29:46.929Z",
"identities": [],
"can_create_group": true,
"can_create_project": true,
"two_factor_enabled": false,
"external": false,
"private_profile": false,
"commit_email": "[email protected]",
"preferred_language": "en",
"shared_runners_minutes_limit": null,
"extra_shared_runners_minutes_limit": null,
"scim_identities": []
}
Validation steps I went through
gitlab git:(rfh-4244-return-403-instead-of-500) curl -s -X POST "http://gdk.test:3000/api/v4/groups" \
-H "PRIVATE-TOKEN: $ROOT_PAT" \
-H "Content-Type: application/json" \
-d '{"name": "ip-restricted-group", "path": "ip-restricted-group", "visibility": "private"}'
{"id":135,"web_url":"http://gdk.test:3000/groups/ip-restricted-group","name":"ip-restricted-group","path":"ip-restricted-group","description":null,"visibility":"private","share_with_group_lock":false,"require_two_factor_authentication":false,"two_factor_grace_period":48,"project_creation_level":"developer","auto_devops_enabled":null,"subgroup_creation_level":"maintainer","emails_disabled":false,"emails_enabled":true,"show_diff_preview_in_email":true,"mentions_disabled":null,"lfs_enabled":true,"archived":false,"math_rendering_limits_enabled":true,"lock_math_rendering_limits_enabled":false,"default_branch":null,"default_branch_protection":2,"default_branch_protection_defaults":{"allowed_to_push":[{"access_level":40}],"allow_force_push":false,"allowed_to_merge":[{"access_level":40}],"developer_can_initial_push":false},"avatar_url":null,"request_access_enabled":true,"full_name":"ip-restricted-group","full_path":"ip-restricted-group","created_at":"2026-03-17T04:01:21.968Z","parent_id":null,"organization_id":1,"shared_runners_setting":"enabled","max_artifacts_size":null,"marked_for_deletion_on":null,"ldap_cn":null,"ldap_access":null,"wiki_access_level":"enabled","repository_storage":null,"duo_core_features_enabled":null,"duo_features_enabled":true,"lock_duo_features_enabled":false,"duo_namespace_access_rules":[],"shared_with_groups":[],"prevent_sharing_groups_outside_hierarchy":false,"shared_runners_minutes_limit":null,"extra_shared_runners_minutes_limit":null,"prevent_forking_outside_group":false,"service_access_tokens_expiration_enforced":true,"membership_lock":false,"ip_restriction_ranges":null,"allowed_email_domains_list":null,"only_allow_merge_if_pipeline_succeeds":false,"allow_merge_on_skipped_pipeline":false,"only_allow_merge_if_all_discussions_are_resolved":false,"unique_project_download_limit":0,"unique_project_download_limit_interval_in_seconds":0,"unique_project_download_limit_allowlist":[],"unique_project_download_limit_alertlist":[1],"auto_ban_user_on_excessive_projects_download":false}%
➜ gitlab git:(rfh-4244-return-403-instead-of-500) export GROUP_ID=135
➜ gitlab git:(rfh-4244-return-403-instead-of-500) curl -s -X PUT "http://gdk.test:3000/api/v4/groups/$GROUP_ID" \
-H "PRIVATE-TOKEN: $ROOT_PAT" \
-H "Content-Type: application/json" \
-d '{"ip_restriction_ranges": "127.0.0.1/32"}'
{"id":135,"web_url":"http://gdk.test:3000/groups/ip-restricted-group","name":"ip-restricted-group","path":"ip-restricted-group","description":null,"visibility":"private","share_with_group_lock":false,"require_two_factor_authentication":false,"two_factor_grace_period":48,"project_creation_level":"developer","auto_devops_enabled":null,"subgroup_creation_level":"maintainer","emails_disabled":false,"emails_enabled":true,"show_diff_preview_in_email":true,"mentions_disabled":null,"lfs_enabled":true,"archived":false,"math_rendering_limits_enabled":true,"lock_math_rendering_limits_enabled":false,"default_branch":null,"default_branch_protection":2,"default_branch_protection_defaults":{"allowed_to_push":[{"access_level":40}],"allow_force_push":false,"allowed_to_merge":[{"access_level":40}],"developer_can_initial_push":false},"avatar_url":null,"request_access_enabled":true,"full_name":"ip-restricted-group","full_path":"ip-restricted-group","created_at":"2026-03-17T04:01:21.968Z","parent_id":null,"organization_id":1,"shared_runners_setting":"enabled","max_artifacts_size":null,"marked_for_deletion_on":null,"ldap_cn":null,"ldap_access":null,"wiki_access_level":"enabled","repository_storage":null,"duo_core_features_enabled":null,"duo_features_enabled":true,"lock_duo_features_enabled":false,"duo_namespace_access_rules":[],"shared_with_groups":[],"runners_token":"GR1348941WqGajdTWGSvtSkwBTZMg","enabled_git_access_protocol":"all","prevent_sharing_groups_outside_hierarchy":false,"projects":[],"shared_projects":[],"shared_runners_minutes_limit":null,"extra_shared_runners_minutes_limit":null,"prevent_forking_outside_group":false,"service_access_tokens_expiration_enforced":true,"membership_lock":false,"ip_restriction_ranges":"127.0.0.1/32","allowed_email_domains_list":null,"only_allow_merge_if_pipeline_succeeds":false,"allow_merge_on_skipped_pipeline":false,"only_allow_merge_if_all_discussions_are_resolved":false,"unique_project_download_limit":0,"unique_project_download_limit_interval_in_seconds":0,"unique_project_download_limit_allowlist":[],"unique_project_download_limit_alertlist":[1],"auto_ban_user_on_excessive_projects_download":false}%
➜ gitlab git:(rfh-4244-return-403-instead-of-500) curl -s -X POST "http://gdk.test:3000/api/v4/projects" \
-H "PRIVATE-TOKEN: $ROOT_PAT" \
-H "Content-Type: application/json" \
-d "{\"name\": \"registry-ip-test\", \"namespace_id\": $GROUP_ID, \"container_registry_enabled\":
true, \"initialize_with_readme\": true}"
{"id":36,"description":null,"name":"registry-ip-test","name_with_namespace":"ip-restricted-group / registry-ip-test","path":"registry-ip-test","path_with_namespace":"ip-restricted-group/registry-ip-test","created_at":"2026-03-17T04:01:48.928Z","default_branch":"main","tag_list":[],"topics":[],"ssh_url_to_repo":"ssh://[email protected]:2222/ip-restricted-group/registry-ip-test.git","http_url_to_repo":"http://gdk.test:3000/ip-restricted-group/registry-ip-test.git","web_url":"http://gdk.test:3000/ip-restricted-group/registry-ip-test","readme_url":"http://gdk.test:3000/ip-restricted-group/registry-ip-test/-/blob/main/README.md","forks_count":0,"avatar_url":null,"star_count":0,"last_activity_at":"2026-03-17T04:01:48.807Z","visibility":"private","namespace":{"id":135,"name":"ip-restricted-group","path":"ip-restricted-group","kind":"group","full_path":"ip-restricted-group","parent_id":null,"avatar_url":null,"web_url":"http://gdk.test:3000/groups/ip-restricted-group"},"repository_storage":"default","container_registry_image_prefix":"registry.test:5100/ip-restricted-group/registry-ip-test","_links":{"self":"http://gdk.test:3000/api/v4/projects/36","issues":"http://gdk.test:3000/api/v4/projects/36/issues","merge_requests":"http://gdk.test:3000/api/v4/projects/36/merge_requests","repo_branches":"http://gdk.test:3000/api/v4/projects/36/repository/branches","labels":"http://gdk.test:3000/api/v4/projects/36/labels","events":"http://gdk.test:3000/api/v4/projects/36/events","members":"http://gdk.test:3000/api/v4/projects/36/members","cluster_agents":"http://gdk.test:3000/api/v4/projects/36/cluster_agents"},"marked_for_deletion_at":null,"marked_for_deletion_on":null,"packages_enabled":true,"empty_repo":false,"archived":false,"resolve_outdated_diff_discussions":false,"container_expiration_policy":{"cadence":"1d","enabled":false,"keep_n":10,"older_than":"90d","name_regex":".*","name_regex_keep":null,"next_run_at":"2026-03-18T04:01:48.959Z"},"repository_object_format":"sha1","issues_enabled":true,"merge_requests_enabled":true,"wiki_enabled":true,"jobs_enabled":true,"snippets_enabled":true,"container_registry_enabled":true,"service_desk_enabled":false,"service_desk_address":null,"can_create_merge_request_in":true,"issues_access_level":"enabled","repository_access_level":"enabled","merge_requests_access_level":"enabled","forking_access_level":"enabled","wiki_access_level":"enabled","builds_access_level":"enabled","snippets_access_level":"enabled","pages_access_level":"private","analytics_access_level":"enabled","container_registry_access_level":"enabled","security_and_compliance_access_level":"private","releases_access_level":"enabled","environments_access_level":"enabled","feature_flags_access_level":"enabled","infrastructure_access_level":"enabled","monitor_access_level":"enabled","model_experiments_access_level":"enabled","model_registry_access_level":"enabled","package_registry_access_level":"enabled","emails_disabled":false,"emails_enabled":true,"show_diff_preview_in_email":true,"shared_runners_enabled":true,"lfs_enabled":true,"creator_id":1,"import_url":null,"import_type":null,"import_status":"none","import_error":null,"open_issues_count":0,"description_html":"","updated_at":"2026-03-17T04:01:48.928Z","ci_default_git_depth":20,"ci_delete_pipelines_in_seconds":null,"ci_forward_deployment_enabled":true,"ci_forward_deployment_rollback_allowed":true,"ci_job_token_scope_enabled":false,"ci_separated_caches":true,"ci_allow_fork_pipelines_to_run_in_parent_project":true,"ci_id_token_sub_claim_components":["project_path","ref_type","ref"],"build_git_strategy":"fetch","keep_latest_artifact":true,"restrict_user_defined_variables":false,"ci_pipeline_variables_minimum_override_role":"developer","runner_token_expiration_interval":null,"group_runners_enabled":true,"resource_group_default_process_mode":"unordered","auto_cancel_pending_pipelines":"enabled","build_timeout":3600,"auto_devops_enabled":true,"auto_devops_deploy_strategy":"continuous","ci_push_repository_for_job_token_allowed":false,"protect_merge_request_pipelines":true,"ci_display_pipeline_variables":false,"runners_token":"GR1348941XQZxc3vzVizp_8VJ8TVh","ci_config_path":null,"public_jobs":true,"shared_with_groups":[],"only_allow_merge_if_pipeline_succeeds":false,"allow_merge_on_skipped_pipeline":null,"request_access_enabled":true,"only_allow_merge_if_all_discussions_are_resolved":false,"remove_source_branch_after_merge":true,"printing_merge_request_link_enabled":true,"merge_method":"merge","squash_option":"default_off","enforce_auth_checks_on_uploads":true,"suggestion_commit_message":null,"merge_commit_template":null,"squash_commit_template":null,"issue_branch_template":null,"warn_about_potentially_unwanted_characters":true,"autoclose_referenced_issues":true,"max_artifacts_size":null,"approvals_before_merge":0,"mirror":false,"external_authorization_classification_label":null,"requirements_enabled":true,"requirements_access_level":"enabled","security_and_compliance_enabled":true,"secret_push_protection_enabled":false,"pre_receive_secret_detection_enabled":false,"compliance_frameworks":[],"issues_template":null,"merge_requests_template":null,"ci_restrict_pipeline_cancellation_role":"developer","merge_pipelines_enabled":false,"merge_trains_enabled":false,"merge_trains_skip_train_allowed":false,"only_allow_merge_if_all_status_checks_passed":false,"allow_pipeline_trigger_approve_deployment":false,"prevent_merge_without_jira_issue":false,"duo_remote_flows_enabled":true,"duo_foundational_flows_enabled":true,"duo_sast_fp_detection_enabled":false,"duo_secret_detection_fp_enabled":true,"duo_sast_vr_workflow_enabled":false,"spp_repository_pipeline_access":true,"merge_request_title_regex":null,"merge_request_title_regex_description":null}%
➜ gitlab git:(rfh-4244-return-403-instead-of-500) export PROJECT_ID=36
➜ gitlab git:(rfh-4244-return-403-instead-of-500) curl -s -X POST "http://gdk.test:3000/api/v4/projects/$PROJECT_ID/members" \
-H "PRIVATE-TOKEN: $ROOT_PAT" \
-H "Content-Type: application/json" \
-d "{\"user_id\": $DEV_USER_ID, \"access_level\": 40}"
{"id":22,"username":"i-user-0-1769046775","public_email":null,"name":"I User0","state":"active","locked":false,"avatar_url":"https://www.gravatar.com/avatar/7b21d4d3108a1f9136940a12f1b77a92087f31a3a103b8fd3cf07fbba30cde58?s=80\u0026d=identicon","web_url":"http://gdk.test:3000/i-user-0-1769046775","access_level":40,"created_at":"2026-03-17T04:02:16.470Z","created_by":{"id":1,"username":"root","public_email":null,"name":"Administrator","state":"active","locked":false,"avatar_url":"https://www.gravatar.com/avatar/df6a7653faf47b4a08264217861507dc6abf07369a07803ebeb34d9a212cf6f9?s=80\u0026d=identicon","web_url":"http://gdk.test:3000/root"},"expires_at":null,"email":"[email protected]","membership_state":"active"}%
I added the gitlab yml file to the project
Triggered pipeline as developer and get a 404:
gitlab git:(rfh-4244-return-403-instead-of-500) curl -s -X POST "http://gdk.test:3000/api/v4/projects/$PROJECT_ID/pipeline" \
-H "PRIVATE-TOKEN: $DEV_PAT" \
-H "Content-Type: application/json" \
-d '{"ref": "main"}'
{"message":"404 Project Not Found"}%
Thanks @radbatnag, taking a look!
Thanks @jdrpereira, i've fixed this up !10271 (e2e62415)
Fiona McCawley (e2e62415) at 16 Mar 23:02
Update manifest limit size
Fiona McCawley (7baee13a) at 16 Mar 23:00
Update manifest limit size
There was an incident relating to a failed pipeline that was escalated to the container registry channel.
While there wasn't specifically an issue with the container registry, the registry sets a limit on manifest sizes (currently 256kb).
Thought I would add a runbook in case this issue happens again.