Skip to content

Presence api#11886

Draft
ArnabChatterjee20k wants to merge 38 commits into1.9.xfrom
presence-api
Draft

Presence api#11886
ArnabChatterjee20k wants to merge 38 commits into1.9.xfrom
presence-api

Conversation

@ArnabChatterjee20k
Copy link
Copy Markdown
Member

What does this PR do?

(Provide a description of what this PR does and why it's needed.)

Test Plan

(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work. Screenshots may also be helpful.)

Related PRs and Issues

  • (Related PR or issue)

Checklist

  • Have you read the Contributing Guidelines on issues?
  • If the PR includes a change to an API's metadata (desc, label, params, etc.), does it also include updated API specs and example docs?

@ArnabChatterjee20k ArnabChatterjee20k marked this pull request as draft April 13, 2026 15:41
->setHttpPath('/v1/presences/:presenceId')
->desc('Delete presence')
->groups(['api', 'presences'])
->label('scope', 'documents.write')
Copy link
Copy Markdown
Member Author

@ArnabChatterjee20k ArnabChatterjee20k Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding document scopes for now for easy start, will change it to presence scope

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 13, 2026

Greptile Summary

This PR introduces a new Presence API module (/v1/presences) with upsert, get, list, update, and delete endpoints backed by a new presenceLogs collection. There are two blocking bugs in Upsert.php and one in Update.php that will cause runtime failures on the privileged/API-key code paths:

  • Update.php is missing the use HTTP; trait, so the PATCH /v1/presences/:presenceId endpoint is never registered by the Utopia framework.
  • In Upsert.php, $user is overwritten with a plain Document returned by getDocument('users', ...), then passed to setPermission() which type-hints User $user — a fatal TypeError on every API-key upsert that provides permissions.
  • userInternalId is populated with $user->getId() instead of $user->getSequence(), storing the external ID in an internal-ID field (contradicting every other module: Account MFA, Teams, VCS).

Confidence Score: 2/5

Not safe to merge — two of the five endpoints have bugs that cause fatal PHP errors or silently corrupt data on the privileged code path.

Three P0/P1 findings affect the core write paths: a missing HTTP trait that prevents endpoint registration, a PHP type error in setPermission, and a wrong method call that corrupts userInternalId for every new presence record.

src/Appwrite/Platform/Modules/Presences/HTTP/Update.php and src/Appwrite/Platform/Modules/Presences/HTTP/Upsert.php require fixes before this can ship.

Important Files Changed

Filename Overview
src/Appwrite/Platform/Modules/Presences/HTTP/Update.php Missing use HTTP; trait causes the PATCH endpoint to never be registered; also has an overly restrictive userId requirement on update.
src/Appwrite/Platform/Modules/Presences/HTTP/Upsert.php Two critical bugs: $user overwritten with plain Document causing a fatal TypeError in setPermission, and userInternalId set via getId() instead of getSequence().
app/config/collections/projects.php Adds presenceLogs collection with indexes; schema contains multiple unresolved TODOs about type choices, defaults, and index strategy.
src/Appwrite/Platform/Modules/Presences/HTTP/Action.php New base action providing setPermission helper; logic is sound and mirrors the pattern used by other modules.
src/Appwrite/Platform/Modules/Presences/HTTP/XList.php List endpoint with cursor pagination and query filtering; correctly restricts allowed attributes.
src/Appwrite/Utopia/Response/Model/Presence.php Response model for Presence; exposes userInternalId publicly which is typically considered an internal implementation detail.
tests/e2e/Services/Presence/PresenceBase.php E2E tests cover upsert, get, list, update, delete, and access-control cases; client-side tests only assert 401 (no positive client-side path tested).

Reviews (1): Last reviewed commit: "Refactor presence tests to handle unauth..." | Re-trigger Greptile

use Utopia\Validator\Nullable;
use Utopia\Validator\Text;

class Update extends PresenceAction
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 Missing use HTTP; trait — endpoint will not be registered

Update extends PresenceAction (which extends PlatformAction) but never declares use HTTP;. Every other HTTP action in the codebase that calls setHttpMethod / setHttpPath uses this trait (e.g. Teams/Http/Teams/Create.php, Presences/HTTP/Upsert.php). Without it the Utopia framework will not register PATCH /v1/presences/:presenceId as an HTTP route, so all update calls will silently 404 or be ignored.

Suggested change
class Update extends PresenceAction
class Update extends PresenceAction
{
use HTTP;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed its alreay present

Comment on lines +98 to +105
$user = $dbForProject->getDocument('users', $userId);
if ($user->isEmpty()) {
throw new Exception(Exception::USER_NOT_FOUND, params: [$userId]);
}

$userInternalId = $user->getId();
$resolvedUserId = $user->getId();
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 $user variable overwritten — fatal type error on API-key path

In the else branch, $user (typed as Appwrite\Utopia\Database\Documents\User) is overwritten with the return value of getDocument(...), which is a plain Utopia\Database\Document. This overwritten value is then passed to setPermission(Document $document, ?array $permissions, User $user, ...) on line 116, which type-hints the third argument as User. PHP 8 will throw a fatal TypeError on every API-key / privileged upsert that provides permissions.

// Store the fetched document separately; keep $user intact for setPermission
$fetchedUser = $dbForProject->getDocument('users', $userId);
if ($fetchedUser->isEmpty()) {
    throw new Exception(Exception::USER_NOT_FOUND, params: [$userId]);
}
$userInternalId = $fetchedUser->getSequence();
$resolvedUserId = $fetchedUser->getId();

Comment on lines +93 to +105
$resolvedUserId = $userId;
if (!$isAPIKey && !$isPrivilegedUser) {
$userInternalId = $user->getId();
$resolvedUserId = $user->getId();
} else {
$user = $dbForProject->getDocument('users', $userId);
if ($user->isEmpty()) {
throw new Exception(Exception::USER_NOT_FOUND, params: [$userId]);
}

$userInternalId = $user->getId();
$resolvedUserId = $user->getId();
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 userInternalId set with getId() instead of getSequence()

Every other module that writes a userInternalId field (Account MFA, Teams memberships, VCS installations) uses $user->getSequence(), not $user->getId(). getId() returns the external $id, making userInternalId identical to userId — the field stores the wrong identifier and indexes on _key_userInternal become useless.

Suggested change
$resolvedUserId = $userId;
if (!$isAPIKey && !$isPrivilegedUser) {
$userInternalId = $user->getId();
$resolvedUserId = $user->getId();
} else {
$user = $dbForProject->getDocument('users', $userId);
if ($user->isEmpty()) {
throw new Exception(Exception::USER_NOT_FOUND, params: [$userId]);
}
$userInternalId = $user->getId();
$resolvedUserId = $user->getId();
}
if (!$isAPIKey && !$isPrivilegedUser) {
$userInternalId = $user->getSequence();
$resolvedUserId = $user->getId();
} else {
$fetchedUser = $dbForProject->getDocument('users', $userId);
if ($fetchedUser->isEmpty()) {
throw new Exception(Exception::USER_NOT_FOUND, params: [$userId]);
}
$userInternalId = $fetchedUser->getSequence();
$resolvedUserId = $fetchedUser->getId();
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its already updated

Comment on lines +2757 to +2760

// Naming it presenceLogs as later it might be only be used as a presence events table only and not for the actual presence
// TODO: check between var_string and var_id => userInternalId, userId(they can be null as well in case of guest login)
'presenceLogs' => [
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Multiple TODO comments indicate unresolved design decisions

The collection definition contains at least five // TODO: comments (type choice for userId/userInternalId, missing expiry default, hostname vs region semantics, source default field, perms_md5 index strategy). Unresolved TODOs in a merged schema definition create a migration burden — changes after the fact require a migration script for all existing rows.

Comment on lines +74 to +80
Authorization $authorization
): void {
$isAPIKey = $user->isApp($authorization->getRoles());
$isPrivilegedUser = $user->isPrivileged($authorization->getRoles());

if ($userId && !$isAPIKey && !$isPrivilegedUser) {
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, 'userId is not allowed for non-API key and non-privileged users');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Requiring userId on every PATCH is unnecessarily restrictive

The presence record already stores userId; a PATCH endpoint should update only the supplied fields. Forcing server-side callers to re-supply userId on every update (or receive a 400) makes partial updates awkward and diverges from the sparse-update pattern described in AGENTS.md. Consider removing the mandatory userId check from the update endpoint and only applying it to the upsert/create path.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that is what happening, we are not forcing server side to provide userId everytime but if its provided, it must be the server side and not client

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 13, 2026

🔄 PHP-Retry Summary

Flaky tests detected across commits:

Commit 05add10 - 7 flaky tests
Test Retries Total Time Details
UsageTest::testFunctionsStats 1 10.27s Logs
UsageTest::testPrepareSitesStats 1 8ms Logs
UsageTest::testEmbeddingsTextUsageDoesNotBreakProjectUsage 1 7ms Logs
LegacyCustomClientTest::testAttributeResponseModels 1 243.01s Logs
LegacyCustomServerTest::testAttributeResponseModels 1 242.96s Logs
VectorsDBConsoleClientTest::testGetCollectionLogs 1 13ms Logs
DatabasesConsoleClientTest::testGetCollectionLogs 1 8ms Logs
Commit 4fea92c - 6 flaky tests
Test Retries Total Time Details
LegacyCustomClientTest::testCreateIndexes 1 240.98s Logs
VectorsDBConsoleClientTest::testGetCollectionLogs 1 4ms Logs
DatabasesConsoleClientTest::testGetCollectionLogs 1 18ms Logs
UsageTest::testFunctionsStats 1 10.21s Logs
UsageTest::testPrepareSitesStats 1 7ms Logs
UsageTest::testEmbeddingsTextUsageDoesNotBreakProjectUsage 1 5ms Logs
Commit 70f42a0 - 4 flaky tests
Test Retries Total Time Details
UsageTest::testFunctionsStats 1 10.20s Logs
UsageTest::testPrepareSitesStats 1 8ms Logs
UsageTest::testEmbeddingsTextUsageDoesNotBreakProjectUsage 1 6ms Logs
TablesDBConsoleClientTest::testNotContains 1 240.56s Logs
Commit 1d32faf - 28 flaky tests
Test Retries Total Time Details
UsageTest::testFunctionsStats 1 10.26s Logs
UsageTest::testPrepareSitesStats 1 8ms Logs
UsageTest::testEmbeddingsTextUsageDoesNotBreakProjectUsage 1 6ms Logs
WebhooksCustomServerTest::testDeleteDeployment 1 16ms Logs
WebhooksCustomServerTest::testDeleteFunction 1 13ms Logs
WebhooksCustomServerTest::testCreateCollection 1 12ms Logs
WebhooksCustomServerTest::testCreateAttributes 1 14ms Logs
WebhooksCustomServerTest::testCreateDocument 1 37ms Logs
WebhooksCustomServerTest::testUpdateDocument 1 11ms Logs
WebhooksCustomServerTest::testDeleteDocument 1 14ms Logs
WebhooksCustomServerTest::testCreateTable 1 14ms Logs
WebhooksCustomServerTest::testCreateColumns 1 13ms Logs
WebhooksCustomServerTest::testCreateRow 1 10ms Logs
WebhooksCustomServerTest::testUpdateRow 1 15ms Logs
WebhooksCustomServerTest::testDeleteRow 1 17ms Logs
WebhooksCustomServerTest::testCreateStorageBucket 1 7ms Logs
WebhooksCustomServerTest::testUpdateStorageBucket 1 11ms Logs
WebhooksCustomServerTest::testCreateBucketFile 1 14ms Logs
WebhooksCustomServerTest::testUpdateBucketFile 1 8ms Logs
WebhooksCustomServerTest::testDeleteBucketFile 1 7ms Logs
WebhooksCustomServerTest::testDeleteStorageBucket 1 15ms Logs
WebhooksCustomServerTest::testCreateTeam 1 8ms Logs
WebhooksCustomServerTest::testUpdateTeam 1 14ms Logs
WebhooksCustomServerTest::testUpdateTeamPrefs 1 20ms Logs
WebhooksCustomServerTest::testDeleteTeam 1 7ms Logs
WebhooksCustomServerTest::testCreateTeamMembership 1 22ms Logs
WebhooksCustomServerTest::testDeleteTeamMembership 1 23ms Logs
WebhooksCustomServerTest::testWebhookAutoDisable 1 31ms Logs
Commit 7167fe2 - 8 flaky tests
Test Retries Total Time Details
UsageTest::testFunctionsStats 1 10.25s Logs
UsageTest::testPrepareSitesStats 1 7ms Logs
UsageTest::testEmbeddingsTextUsageDoesNotBreakProjectUsage 1 6ms Logs
LegacyConsoleClientTest::testListDocumentsWithCache 1 648ms Logs
LegacyConsoleClientTest::testListDocumentsCachePurgedByUpdate 1 1.26s Logs
LegacyCustomClientTest::testAttributeResponseModels 1 242.68s Logs
VectorsDBConsoleClientTest::testGetCollectionLogs 1 9ms Logs
DatabasesConsoleClientTest::testGetCollectionLogs 1 7ms Logs

Note: Flaky test results are tracked for the last 5 commits

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 13, 2026

✨ Benchmark results

  • Requests per second: 1,874
  • Requests with 200 status code: 337,369
  • P99 latency: 0.094530565

⚡ Benchmark Comparison

Metric This PR Latest version
RPS 1,874 1,427
200 337,369 256,828
P99 0.094530565 0.140369746

@blacksmith-sh

This comment has been minimized.

ArnabChatterjee20k and others added 14 commits April 16, 2026 17:47
…ests. Updated authorization roles synchronization in the realtime connection and added exception handling for user retrieval. Improved connection pool size logic for PubSub workers and added comprehensive tests for presence events to ensure correct message ordering and validation.
…eption handling for MongoDB adapter to update existing documents instead of creating new ones when presenceId mismatches occur.

try {
if ($this->getSupportForUniqueIndexBasedUpsert()) {
return $this->transactionalUpsertForUser($dbForProject, $presenceDocument, $presenceId, $userId);
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mongodb and postgres adapter doesn't work on upsert based on unique indexes so manual upsert logic for this to have consistent and deterministic result
Failing cases might be -> user sending an unique $id for an already present user which will lead to create but due to unique index it will throw duplicate exception
And due to duplicate exception we again have to do this manually

Better to do it from start for mongodb and postgres

'source' => 'rest',
'expiry' => $expiry ?? DateTime::addSeconds(new \DateTime(), 15 * 60),
// TODO: finding a way to find hostname
// 'hostname' => $hostname,
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

],
contentType: ContentType::NONE,
))
->param('presenceId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Presence unique ID.', false, ['dbForProject'])
Copy link
Copy Markdown
Member Author

@ArnabChatterjee20k ArnabChatterjee20k Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abnegate I think we should proivide here the bulk deletion rather? In case of upsert, update, etc it kind of not useful
But in the delete, the admin might want to delete in bulk
WDYT?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant