Skip to content

Commit 1b9a8ab

Browse files
brendan-kellamclaude
brendan-kellam
and
claude
authored
feat(backend): add Bitbucket Server (Data Center) permission syncing (#938)
* feat(backend): add Bitbucket Server permission syncing Adds account-driven and repo-driven permission sync support for Bitbucket Server (Data Center), mirroring the existing GitHub, GitLab, and Bitbucket Cloud implementations. - Extend BitbucketServerIdentityProviderConfig to support purpose: "account_linking" and accountLinkingRequired field - Request REPO_READ OAuth scope when permission syncing is enabled - Add bitbucket-server token refresh support via /rest/oauth2/latest/token - Add bitbucketServer/bitbucket-server to permission sync constants - Add getReposForAuthenticatedBitbucketServerUser and getUserPermissionsForServerRepo to bitbucket.ts - Add bitbucket-server branch to accountPermissionSyncer - Add bitbucketServer branch to repoPermissionSyncer - Update docs to reflect new account_linking purpose support Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * chore: update CHANGELOG for Bitbucket Server permission syncing Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * feat(backend): store Bitbucket Server projectKey/repoSlug in codeHostMetadata Instead of deriving projectKey and repoSlug from the displayName at permission sync time, write them into codeHostMetadata.bitbucketServer during connection compile, and read from there in repoPermissionSyncer. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(backend): instrument all Bitbucket API calls with fetchWithRetry Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * docs: update Bitbucket Data Center authentication docs with token type tabs Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * docs: update permission syncing and IDP docs for Bitbucket Data Center - Add Bitbucket Data Center to permission-syncing.mdx with prerequisites, partial coverage warning, and notes - Update platform support table to show 🟠 Partial for Bitbucket Data Center - Add connection prerequisites to all code host sections - Update IDP links to point to specific subsections per code host - Update idp.mdx so all code hosts consistently mention additional OAuth scopes needed for permission syncing (matching GitLab's style) - Update bitbucket-data-center.mdx auth section with tabs for user account vs project/repository tokens, and add Note about admin permissions required Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(backend): remove project-level permission call from Bitbucket Server repo-driven sync Repo-driven syncing for Bitbucket Server now only covers users with direct repo-level grants. Project-level and group-level access remains covered by account-driven syncing, consistent with the Bitbucket Cloud approach. This avoids redundant API calls (one per repo for the same project) that could cause rate limiting issues at scale. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(web): preserve context path in OAuth token refresh URL construction Using absolute paths (e.g. '/rest/...') with new URL() drops any pathname from the base URL. Normalize the base to end with '/' and use relative paths so deployments with a context path (e.g. https://example.com/bitbucket) resolve correctly. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(backend): serialize error objects with JSON.stringify in bitbucket error messages Replace error.type interpolation with JSON.stringify(error) for consistent and complete error serialization. Also remove stray space before parenthesis in one throw expression. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]>
1 parent da5317b commit 1b9a8ab

19 files changed

Lines changed: 391 additions & 93 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1515
- Added `wa_askgh_login_wall_prompted` PostHog event fired when an unauthenticated user attempts to ask a question on Ask GitHub. [#933](https://github.com/sourcebot-dev/sourcebot/pull/933)
1616
- Added Bitbucket Server (Data Center) OAuth 2.0 SSO identity provider support (`provider: "bitbucket-server"`). [#934](https://github.com/sourcebot-dev/sourcebot/pull/934)
1717
- Added Bitbucket Server (Data Center) sync all repositories support. [#927](https://github.com/sourcebot-dev/sourcebot/pull/927)
18+
- Added permission syncing support for Bitbucket Server (Data Center), including account-driven and repo-driven sync. [#938](https://github.com/sourcebot-dev/sourcebot/pull/938)
1819

1920
### Changed
2021
- Hide version upgrade toast for askgithub deployment (`EXPERIMENT_ASK_GH_ENABLED`). [#931](https://github.com/sourcebot-dev/sourcebot/pull/931)

‎docs/docs/configuration/idp.mdx‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -74,9 +74,11 @@ in the GitHub identity provider config.
7474
- `"Metadata" repository permissions (read)` (only needed if using permission syncing)
7575
</Tab>
7676
<Tab title="GitHub OAuth App">
77-
Follow [this guide](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/creating-an-oauth-app) by GitHub to create an OAuth App.
78-
77+
Follow [this guide](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/creating-an-oauth-app) by GitHub to create an OAuth App.
78+
7979
When asked to provide a callback url, provide `<sourcebot_url>/api/auth/callback/github` (ex. https://sourcebot.coolcorp.com/api/auth/callback/github)
80+
81+
If [permission syncing](/docs/features/permission-syncing#github) is enabled, also enable the `repo` scope.
8082
</Tab>
8183
</Tabs>
8284
</Step>
@@ -128,7 +130,7 @@ in the GitLab identity provider config.
128130
When configuring your application:
129131
- Set the callback URL to `<sourcebot_url>/api/auth/callback/gitlab` (ex. https://sourcebot.coolcorp.com/api/auth/callback/gitlab)
130132
- Enable the `read_user` scope
131-
- If using for permission syncing, also enable the `read_api` scope
133+
- If [permission syncing](/docs/features/permission-syncing#gitlab) is enabled, also enable the `read_api` scope
132134

133135
The result of registering an OAuth application is an `APPLICATION_ID` (`CLIENT_ID`) and `SECRET` (`CLIENT_SECRET`) which you'll provide to Sourcebot.
134136
</Step>
@@ -182,7 +184,7 @@ in the Bitbucket Cloud identity provider config.
182184
When configuring your consumer:
183185
- Set the callback URL to `<sourcebot_url>/api/auth/callback/bitbucket-cloud` (ex. https://sourcebot.coolcorp.com/api/auth/callback/bitbucket-cloud)
184186
- Enable **Account: Read**
185-
- If using for permission syncing, also enable **Repositories: Read**
187+
- If [permission syncing](/docs/features/permission-syncing#bitbucket-cloud) is enabled, also enable **Repositories: Read**
186188

187189
The result of creating an OAuth consumer is a `Key` (`CLIENT_ID`) and `Secret` (`CLIENT_SECRET`) which you'll provide to Sourcebot.
188190
</Step>
@@ -220,7 +222,8 @@ in the Bitbucket Cloud identity provider config.
220222

221223
### Bitbucket Server
222224

223-
A Bitbucket Server (Data Center) connection can be used for [authentication](/docs/configuration/auth).
225+
A Bitbucket Server (Data Center) connection can be used for [authentication](/docs/configuration/auth) and/or [permission syncing](/docs/features/permission-syncing). This is controlled using the `purpose` field
226+
in the Bitbucket Server identity provider config.
224227

225228
<Accordion title="instructions">
226229
<Steps>
@@ -231,6 +234,7 @@ A Bitbucket Server (Data Center) connection can be used for [authentication](/do
231234

232235
When configuring your application:
233236
- Set the redirect URL to `<sourcebot_url>/api/auth/callback/bitbucket-server` (ex. https://sourcebot.coolcorp.com/api/auth/callback/bitbucket-server)
237+
- If [permission syncing](/docs/features/permission-syncing#bitbucket-data-center) is enabled, also enable the `REPO_READ` scope
234238

235239
The result of creating the application is a `CLIENT_ID` and `CLIENT_SECRET` which you'll provide to Sourcebot.
236240
</Step>
@@ -247,7 +251,10 @@ A Bitbucket Server (Data Center) connection can be used for [authentication](/do
247251
"identityProviders": [
248252
{
249253
"provider": "bitbucket-server",
250-
"purpose": "sso",
254+
// "sso" for auth + perm sync, "account_linking" for only perm sync
255+
"purpose": "account_linking",
256+
// if purpose == "account_linking" this controls if a user must connect to the IdP
257+
"accountLinkingRequired": true,
251258
"baseUrl": "https://bitbucket.example.com",
252259
"clientId": {
253260
"env": "YOUR_CLIENT_ID_ENV_VAR"

‎docs/docs/connections/bitbucket-data-center.mdx‎

Lines changed: 69 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -80,35 +80,75 @@ If you're not familiar with Sourcebot [connections](/docs/connections/overview),
8080

8181
## Authenticating with Bitbucket Data Center
8282

83-
In order to index private repositories, you'll need to provide an access token to Sourcebot via a [token](/docs/configuration/config-file#tokens).
84-
85-
Create an access token for the desired scope (repo, project, or workspace). Visit the official [Bitbucket Data Center docs](https://confluence.atlassian.com/bitbucketserver/http-access-tokens-939515499.html)
86-
for more info.
87-
88-
1. Add the `token` property to your connection config:
89-
90-
```json
91-
{
92-
"type": "bitbucket",
93-
"deploymentType": "server",
94-
"url": "https://mybitbucketdeployment.com",
95-
"token": {
96-
// note: this env var can be named anything. It
97-
// doesn't need to be `BITBUCKET_TOKEN`.
98-
"env": "BITBUCKET_TOKEN"
99-
}
100-
// .. rest of config ..
101-
}
102-
```
103-
104-
2. Pass this environment variable each time you run Sourcebot:
105-
106-
```bash
107-
docker run \
108-
-e BITBUCKET_TOKEN=<ACCESS_TOKEN> \
109-
/* additional args */ \
110-
ghcr.io/sourcebot-dev/sourcebot:latest
111-
```
83+
In order to index private repositories, you'll need to provide a [HTTP Access Token](https://confluence.atlassian.com/bitbucketserver/http-access-tokens-939515499.html). Tokens can be scoped to a user account, a project, or an individual repository. Only repositories visible to the token will be able to be indexed by Sourcebot.
84+
85+
<Note>
86+
If [permission syncing](/docs/features/permission-syncing#bitbucket-data-center) is enabled, the token must have **Repository Admin** permissions so Sourcebot can read repository-level user permissions.
87+
</Note>
88+
89+
<Tabs>
90+
<Tab title="User account token">
91+
User account tokens grant access to all repositories the user can access. Because these are tied to a specific user account, you must also set the `user` field to that user's username.
92+
93+
1. In Bitbucket Data Center, navigate to your profile → **Manage account** → **HTTP access tokens** and click **Create token**. Give it a name and grant it **Project read** and **Repository read** permissions.
94+
95+
2. Add the `user` (your Bitbucket username) and `token` properties to your connection config:
96+
97+
```json
98+
{
99+
"type": "bitbucket",
100+
"deploymentType": "server",
101+
"url": "https://mybitbucketdeployment.com",
102+
"user": "myusername",
103+
"token": {
104+
// note: this env var can be named anything. It
105+
// doesn't need to be `BITBUCKET_TOKEN`.
106+
"env": "BITBUCKET_TOKEN"
107+
}
108+
// .. rest of config ..
109+
}
110+
```
111+
112+
3. Pass this environment variable each time you run Sourcebot:
113+
114+
```bash
115+
docker run \
116+
-e BITBUCKET_TOKEN=<ACCESS_TOKEN> \
117+
/* additional args */ \
118+
ghcr.io/sourcebot-dev/sourcebot:latest
119+
```
120+
</Tab>
121+
<Tab title="Project / repository token">
122+
Project and repository tokens are scoped to a specific project or repository.
123+
124+
1. In Bitbucket Data Center, navigate to the project or repository → **Settings** → **HTTP access tokens** and click **Create token**. Give it a name and grant it **Repository read** and **Project read** permissions.
125+
126+
2. Add the `token` property to your connection config:
127+
128+
```json
129+
{
130+
"type": "bitbucket",
131+
"deploymentType": "server",
132+
"url": "https://mybitbucketdeployment.com",
133+
"token": {
134+
// note: this env var can be named anything. It
135+
// doesn't need to be `BITBUCKET_TOKEN`.
136+
"env": "BITBUCKET_TOKEN"
137+
}
138+
// .. rest of config ..
139+
}
140+
```
141+
142+
3. Pass this environment variable each time you run Sourcebot:
143+
144+
```bash
145+
docker run \
146+
-e BITBUCKET_TOKEN=<ACCESS_TOKEN> \
147+
/* additional args */ \
148+
ghcr.io/sourcebot-dev/sourcebot:latest
149+
```
150+
</Tab>
151+
</Tabs>
112152

113153
## Troubleshooting
114154
If you're seeing errors like `TypeError: fetch failed` when fetching repo info, it may be that Sourcebot is refusing to connect to your self-hosted Bitbucket instance due to unrecognized SSL certs. Try setting the `NODE_TLS_REJECT_UNAUTHORIZED=0` environment variable or providing Sourcebot your certs through the `NODE_EXTRA_CA_CERTS` environment variable.

‎docs/docs/features/permission-syncing.mdx‎

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ We are actively working on supporting more code hosts. If you'd like to see a sp
4040
| [GitHub (GHEC & GHEC Server)](/docs/features/permission-syncing#github) | ✅ |
4141
| [GitLab (Self-managed & Cloud)](/docs/features/permission-syncing#gitlab) | ✅ |
4242
| [Bitbucket Cloud](/docs/features/permission-syncing#bitbucket-cloud) | 🟠 Partial |
43-
| Bitbucket Data Center | 🛑 |
43+
| [Bitbucket Data Center](/docs/features/permission-syncing#bitbucket-data-center) | 🟠 Partial |
4444
| Gitea | 🛑 |
4545
| Gerrit | 🛑 |
4646
| Generic git host | 🛑 |
@@ -50,7 +50,8 @@ We are actively working on supporting more code hosts. If you'd like to see a sp
5050
## GitHub
5151

5252
Prerequisites:
53-
- Configure GitHub as an [external identity provider](/docs/configuration/idp).
53+
- Configure a [GitHub connection](/docs/connections/github).
54+
- Configure GitHub as an [external identity provider](/docs/configuration/idp#github).
5455
- **If you are using a self-hosted GitHub instance**, you must also set the `baseUrl` property of the `github` identity provider in the [config file](/docs/configuration/config-file) to the base URL of your GitHub instance (e.g. `https://github.example.com`).
5556

5657
Permission syncing works with **GitHub.com**, **GitHub Enterprise Cloud**, and **GitHub Enterprise Server**. For organization-owned repositories, users that have **read-only** access (or above) via the following methods will have their access synced to Sourcebot:
@@ -61,27 +62,29 @@ Permission syncing works with **GitHub.com**, **GitHub Enterprise Cloud**, and *
6162
- Organization owners.
6263

6364
**Notes:**
64-
- A GitHub [external identity provider](/docs/configuration/idp) must be configured to (1) correlate a Sourcebot user with a GitHub user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
65+
- A GitHub [external identity provider](/docs/configuration/idp#github) must be configured to (1) correlate a Sourcebot user with a GitHub user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
6566
- OAuth tokens must assume the `repo` scope in order to use the [List repositories for the authenticated user API](https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#list-repositories-for-the-authenticated-user) during [User driven syncing](/docs/features/permission-syncing#how-it-works). Sourcebot **will only** use this token for **reads**.
6667

6768
## GitLab
6869

6970
Prerequisites:
70-
- Configure GitLab as an [external identity provider](/docs/configuration/idp).
71+
- Configure a [GitLab connection](/docs/connections/gitlab).
72+
- Configure GitLab as an [external identity provider](/docs/configuration/idp#gitlab).
7173
- **If you are using a self-hosted GitLab instance**, you must also set the `baseUrl` property of the `gitlab` identity provider in the [config file](/docs/configuration/config-file) to the base URL of your GitLab instance (e.g. `https://gitlab.example.com`).
7274

7375
Permission syncing works with **GitLab Self-managed** and **GitLab Cloud**. Users with **Guest** role or above with membership to a group or project will have their access synced to Sourcebot. Both direct and indirect membership to a group or project will be synced with Sourcebot. For more details, see the [GitLab docs](https://docs.gitlab.com/user/project/members/#membership-types).
7476

7577

7678
**Notes:**
77-
- A GitLab [external identity provider](/docs/configuration/idp) must be configured to (1) correlate a Sourcebot user with a GitLab user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
79+
- A GitLab [external identity provider](/docs/configuration/idp#gitlab) must be configured to (1) correlate a Sourcebot user with a GitLab user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
7880
- OAuth tokens require the `read_api` scope in order to use the [List projects for the authenticated user API](https://docs.gitlab.com/ee/api/projects.html#list-all-projects) during [User driven syncing](/docs/features/permission-syncing#how-it-works).
7981
- [Internal GitLab projects](https://docs.gitlab.com/user/public_access/#internal-projects-and-groups) are **not** enforced by permission syncing and therefore are visible to all users. Only [private projects](https://docs.gitlab.com/user/public_access/#private-projects-and-groups) are enforced.
8082

8183
## Bitbucket Cloud
8284

8385
Prerequisites:
84-
- Configure Bitbucket Cloud as an [external identity provider](/docs/configuration/idp).
86+
- Configure a [Bitbucket Cloud connection](/docs/connections/bitbucket-cloud).
87+
- Configure Bitbucket Cloud as an [external identity provider](/docs/configuration/idp#bitbucket-cloud).
8588

8689
Permission syncing works with **Bitbucket Cloud**. OAuth tokens must assume the `account` and `repository` scopes.
8790

@@ -98,9 +101,33 @@ If your workspace relies heavily on group or project-level permissions rather th
98101
</Warning>
99102

100103
**Notes:**
101-
- A Bitbucket Cloud [external identity provider](/docs/configuration/idp) must be configured to (1) correlate a Sourcebot user with a Bitbucket Cloud user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
104+
- A Bitbucket Cloud [external identity provider](/docs/configuration/idp#bitbucket-cloud) must be configured to (1) correlate a Sourcebot user with a Bitbucket Cloud user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
102105
- OAuth tokens require the `account` and `repository` scopes. The `repository` scope is required to list private repositories during [User driven syncing](/docs/features/permission-syncing#how-it-works).
103106

107+
## Bitbucket Data Center
108+
109+
Prerequisites:
110+
- Configure a [Bitbucket Data Center connection](/docs/connections/bitbucket-data-center).
111+
- Configure Bitbucket Data Center as an [external identity provider](/docs/configuration/idp#bitbucket-server).
112+
113+
Permission syncing works with **Bitbucket Data Center**. OAuth tokens must assume the `PUBLIC_REPOS` and `REPO_READ` scopes.
114+
115+
<Warning>
116+
**Partial coverage for repo-driven syncing.** Repo-driven syncing only captures users who have been **directly and explicitly** granted access to the repository. Users who have access via any of the following are **not** captured by repo-driven syncing:
117+
118+
- Project-level permissions (inherited by all repos in the project)
119+
- Group membership
120+
121+
These users **will** still gain access via [user-driven syncing](/docs/features/permission-syncing#how-it-works), which fetches all repositories accessible to each authenticated user using the `REPO_READ` scope. However, there may be a delay between when access is granted and when affected users see the repository in Sourcebot (up to the `experiment_userDrivenPermissionSyncIntervalMs` interval, which defaults to 24 hours).
122+
123+
If your instance relies heavily on project or group-level permissions, we recommend reducing the `experiment_userDrivenPermissionSyncIntervalMs` interval to limit the window of delay.
124+
</Warning>
125+
126+
**Notes:**
127+
- A Bitbucket Data Center [external identity provider](/docs/configuration/idp#bitbucket-server) must be configured to (1) correlate a Sourcebot user with a Bitbucket Data Center user, and (2) to list repositories that the user has access to for [User driven syncing](/docs/features/permission-syncing#how-it-works).
128+
- The connection token must have **Repository Read** permissions so Sourcebot can read repository-level user permissions for [Repo driven syncing](/docs/features/permission-syncing#how-it-works).
129+
- OAuth tokens require the `REPO_READ` scope to list accessible repositories during [User driven syncing](/docs/features/permission-syncing#how-it-works).
130+
104131
# How it works
105132

106133
Permission syncing works by periodically syncing ACLs from the code host(s) to Sourcebot to build an internal mapping between Users and Repositories. This mapping is hydrated in two directions:

‎docs/snippets/schemas/v3/identityProvider.schema.mdx‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -850,7 +850,10 @@
850850
"const": "bitbucket-server"
851851
},
852852
"purpose": {
853-
"const": "sso"
853+
"enum": [
854+
"sso",
855+
"account_linking"
856+
]
854857
},
855858
"clientId": {
856859
"anyOf": [
@@ -919,6 +922,10 @@
919922
"https://bitbucket.example.com"
920923
],
921924
"pattern": "^https?:\\/\\/[^\\s/$.?#].[^\\s]*$"
925+
},
926+
"accountLinkingRequired": {
927+
"type": "boolean",
928+
"default": false
922929
}
923930
},
924931
"required": [
@@ -1777,7 +1784,10 @@
17771784
"const": "bitbucket-server"
17781785
},
17791786
"purpose": {
1780-
"const": "sso"
1787+
"enum": [
1788+
"sso",
1789+
"account_linking"
1790+
]
17811791
},
17821792
"clientId": {
17831793
"anyOf": [
@@ -1846,6 +1856,10 @@
18461856
"https://bitbucket.example.com"
18471857
],
18481858
"pattern": "^https?:\\/\\/[^\\s/$.?#].[^\\s]*$"
1859+
},
1860+
"accountLinkingRequired": {
1861+
"type": "boolean",
1862+
"default": false
18491863
}
18501864
},
18511865
"required": [

‎docs/snippets/schemas/v3/index.schema.mdx‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5395,7 +5395,10 @@
53955395
"const": "bitbucket-server"
53965396
},
53975397
"purpose": {
5398-
"const": "sso"
5398+
"enum": [
5399+
"sso",
5400+
"account_linking"
5401+
]
53995402
},
54005403
"clientId": {
54015404
"anyOf": [
@@ -5464,6 +5467,10 @@
54645467
"https://bitbucket.example.com"
54655468
],
54665469
"pattern": "^https?:\\/\\/[^\\s/$.?#].[^\\s]*$"
5470+
},
5471+
"accountLinkingRequired": {
5472+
"type": "boolean",
5473+
"default": false
54675474
}
54685475
},
54695476
"required": [
@@ -6322,7 +6329,10 @@
63226329
"const": "bitbucket-server"
63236330
},
63246331
"purpose": {
6325-
"const": "sso"
6332+
"enum": [
6333+
"sso",
6334+
"account_linking"
6335+
]
63266336
},
63276337
"clientId": {
63286338
"anyOf": [
@@ -6391,6 +6401,10 @@
63916401
"https://bitbucket.example.com"
63926402
],
63936403
"pattern": "^https?:\\/\\/[^\\s/$.?#].[^\\s]*$"
6404+
},
6405+
"accountLinkingRequired": {
6406+
"type": "boolean",
6407+
"default": false
63946408
}
63956409
},
63966410
"required": [

0 commit comments

Comments
 (0)